Tutorial: Enable Video Mixer Plugin

Function Description

This article introduces how to use the videomixer plugin to mix local camera and screen streams into the main stream.

Prerequisites

• TRTC Web SDK version >= 5.11.0.

• Compatibility:

  Browser	Compatibility
  Desktop Chrome	✅
  Desktop Safari	✅
  Desktop Firefox	✅
  Desktop Edge	✅
  iOS Safari	❌ Does not support screen sharing
  Android Chrome	❌ Does not support screen sharing

Implementation Steps

Install VideoMixer Plugin

import { VideoMixer } from 'trtc-sdk-v5/plugins/video-effect/video-mixer';
let trtc = TRTC.create({ plugins: [VideoMixer] });

Start VideoMixer Plugin

await trtc.startPlugin('VideoMixer', {
    container: {
      width: 1920,
      height: 1080
    },
    camera: {
      x: 1280,
      y: 0,
      width: 640,
      height: 480
    },
    screen: {
      x: 0,
      y: 0,
      width: 1920,
      height: 1080
    }
  });

When starting the plugin, the width and height of the mixing canvas must be set. The parameters for the camera and screen are optional. If not set, the video will start from the top-left corner with the original video width and height.

Open Camera and Screen

After starting the plugin, both the camera and screen will be mixed into the main stream. Therefore, you need to call:

await trtc.startLocalVideo({
  view: 'local-video'
});

Then, start screen sharing, and you will see the camera and screen mixed together.

await trtc.startScreenShare();

Note: After starting the plugin, the auxiliary stream will automatically stop because the auxiliary stream has been mixed into the main stream.

Update VideoMixer Plugin

You can update the parameters of the mixed stream:

await trtc.updatePlugin('VideoMixer', {
  camera: {
    x: 50,
    y: 100,
    width: 960
  },
});

Here, only the width is specified, so the height of the camera image will be stretched according to the original video ratio. Conversely, if only the height is specified, the width will be stretched proportionally.

Stop VideoMixer Plugin

await trtc.stopPlugin('VideoMixer');

Mix Without Camera

If the camera will not be used when the mixing starts (e.g., first screen sharing, then later turning on the camera), you need to specify the capture parameter of startLocalVideo as false:

await trtc.startLocalVideo({
  view: 'local-video',
  capture: false
});

Subsequently, to turn on/off the camera, update the capture parameter:

await trtc.updateLocalVideo({
  capture: true
});

Note: After enabling mixing, switching the camera must be done through updateLocalVideo({ capture: true/false }). Do not call stopLocalVideo, as it will stop the main stream and prevent screen sharing from mixing.

API Description

trtc.startPlugin('VideoMixer', options)

Used to start the Mixing Plugin.

options

LayerOption

Resolution

Example:

await trtc.startPlugin('VideoMixer', {
  container: {
    width: 1920,
    height: 1080
  }
});
await trtc.startPlugin('VideoMixer', {
  container: {
    width: 640,
    height: 1000
  },
  camera: {
    x: 0,
    y: 0,
    width: 640,
    height: 480
  },
  screen: {
    x: 0,
    y: 480,
    width: 640,
    height: 500
  }
});

trtc.updatePlugin('VideoMixer', options)

Used to update the mixing parameters. Except for the container parameter, which becomes optional, the other parameters passed are the same as in startPlugin.

Example

await trtc.updatePlugin('VideoMixer', {
  camera: {
    x: 500,
    y: 0,
    width: 640,
    height: 480
  }
});

trtc.stopPlugin('VideoMixer')

Stops the Mixing Plugin.

Example:

await trtc.stopPlugin('VideoMixer');

Q&A

1. How to set resolution and bitrate?

• Resolution refers to the width and height of the canvas.
• Bitrate is set through the start/updateLocalVideo interface for the main stream parameters.