webrtc-设备管理及采集

enumeratedevices

获取所有有效的输入输出设备(麦克风、摄像头、穿戴设备等)音视频设备，返回promise其resolve值是所有设备的列表

type MediaDeviceInfo = {
	deviceID: "device's ID",
	label: "device's name",
	kind: "device's category"
	groupID: 'the groupID of several physical devices '
}

navigator.mediaDevices.enumerateDevices().then((deviceList: MediaDeviceInfo[]) => {
	deviceList.forEach(device => {
		console.log('device.label:', device.label, 'device.deviceID: ', device.deviceID);
	})
})

<Callout> 如果未能获取设备权限，返回的设备信息devideID和label则为空. <a target="_blank" href="https://stackoverflow.com/questions/46648645/navigator-mediadevices-enumeratedevices-not-display-device-label-on-firefox">参考StackOverflow</a> 需要下面要讲的api </Callout>

getusermediaconstraints

获取用户权限
constraints[audio: boolean, video: boolean]
用来设置要获取权限的设备媒体类型,如果不先获取权限enumerateDevices获取的设备信息将为空

function getDeviceInfo() {
  navigator.mediaDevices.enumerateDevices().then((deviceList) => {
		deviceList.forEach(device => {
			console.log({
				'label': device.label,
				'deviceID': device.deviceID
			});
		})
	})
}

// must be get user permission at first
navigator.mediaDevices.getUserMedia({audio: true, video: true})
	.then(stream => {
		getDeviceInfo();
		getMediaStream(stream);
	})
	.catch(error => {
		console.log('Error :', error)
	})

getusermedia的不同实现适配

使用属性探侦的方式获取可用方法

const getUserMedia = navigator.getUserMedia ||
					 navigator.webkitGetUserMedia ||
					 navigator.mozGetUserMedia;

使用Google开源库的adapter.js,实际项目中使用最近release的版本号而非latest版(可能会有break change功能，导致不同浏览器出现问题)

https://webrtc.github.io/adapter/adapter-latest.js

视频采集的约束

视频一般的分辨率(aspectRatio)是4：3和16：9
frameRate: 帧率（30、60、120），帧率越高流就越大
facingMode: user(前置摄像头)、enviroment(后置摄像头)、left(前置左侧摄像头)、right(前置右侧摄像头)
resizeMode: 是否需要裁剪

const constraints = {
	audio: false,
	video: {
		width: 320,
		height: 240,
		frameRate: 30,
		facingMode: 'enviroment'
	}
}

navigator.mediaDevices.getUserMedia(constraints).then(res => {
	const video = document.querySelector('video');
	video.srcObject = mediaStream;
	video.onloadedmetadata = (e) => {
	  video.play();
	};
}).catch(err => console.log(err.name + ':' + err.message))

音频采集的约束

volume(声音): number
sampleRate(采样率): number
sampleSize(采样大小): number
echoCancellation(回音消除): boolean
autoGainControl(自动增音): boolean
noiseSuppression(降噪): boolean
latency(延迟大小200ms - 500ms): number
channelCount(单双声道): number
deviceId(设备ID): string
groupId(物理设备类ID): string

<Callout type="info"> 参考MDN文档查看具体参数及用法 <a target="_target" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia">getUserMedia</a>、 <a target="_target" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getSupportedConstraints">getSupportedConstraints</a>、 <a target="_target" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints">MediaTrackConstraints</a> <p className="mt-2"> 所有参数默认都是Boolean类型，同时也可设置为包含了exact、ideal、min、max属性的对象（根据具体属性的不同而异），如果传入属性的值不正确则会取系统最接近的值替换具体查看。 <a target="_target" href="https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API/Constraints#example__constraint_exerciser">example__constraint_exerciser</a> </p> </Callout>

navigator.mediaDevices.getUserMedia({
  video: {
    width: { min: 640, ideal: 1920 },
    height: { min: 400, ideal: 1080 },
    aspectRatio: { ideal: 1.7777777778 }
  },
  audio: {
    sampleSize: 16,
    channelCount: 2
  }
}).then(stream => {
  videoElement.srcObject = stream;
}).catch(handleError);

浏览器视频特效

通过CSS filter、-webkit-filter/filter与video标签关联

grayscale 灰度
spepia 褐色
saturate 饱和度
hue-rotate 色相旋转
invert 反色
opacity 透明度
brightness 亮度
contrast 对比度
blur 模糊
drop-shadow 阴影
filter-effect具体文档： https://developer.mozilla.org/en-US/docs/Web/CSS/Filter_Effects

<Callout type="info">只需将对应过滤特效css应用到video标签样式属性即可</Callout>

从视频中获取图片

只需要通过捕捉当前视频的当前帧即可

pictureCanvas.getContext('2d').drawImage(
	video,
	0,
	0,
	picture.width,
	picture.height
);

音频录制

和视频录制同理，通过getUserMedia获取媒体流并放置到audio源中

html

<audio autoplay controls id='audioplayer'></audio>

navigator.mediaDevices.getUserMedia({audio: true, video: false}).then(res => {
	audioplayer.srcObject = mediaStream;
}).catch(err => console.log(err.name + ':' + err.message))

mediastream

MediaStream中的方法与事件

<Callout type="info"> Mdn文档 <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaStream">MediaStream</a>、 <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/addtrack_event">MediaStream event</a>、 <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack">MediaStreamTrack</a> </Callout>

属性

MediaStream.active => boolean 是否活动状态
MediaStream.id => boolean 一个包含了36个字节的uuid

方法

MediaStream.addTrack() 添加媒体轨
MediaStream.removeTrack() 移除媒体轨
MediaStream.clone() 根据传入的MediaStream返回其副本,但是副本具有其唯一id
MediaStream.getTrackById() 根据trackid获取媒体轨
MediaStream.getTracks() 获取所有媒体轨
MediaStream.getVideoTracks() 获取视频轨
MediaStream.getAudioTracks() 获取音频轨

事件

onaddtrack 当添加媒体轨时触发
onremovetrack 当移除媒体轨时触发
active(不推荐) 当激活状态时触发
inactive(不推荐) 当失活状态时触发
onended 当流的readyState结束时触发

获取视频流的配置

navigator.mediaDevices.getUserMedia({audio: false, video: true})
.then(mediaStream => {
	const videoTrack = mediaStream.getVideoTracks()[0];
	const videoConstraints = videoTrack.getSettings();

	// 预期和传入的constraints相同否则展示默认配置
	divConstraints.textContent = JSON.stringify(videoConstraints, null, 2);
})