Tutorial: Enable Virtual Background

Enable Virtual Background

Function Description

This document will describe how to implement the virtual background feature during a call.

Original Camera Background Blur Background Image

Prerequisites

  • Pricing see RTC-Engine Packages.
  • TRTC SDK version must be >= 5.5.0
  • System and configuration requirements for various platforms are as follows:
Platform Operating System Browser Version FPS Recommended Configuration Remarks
Web Windows Chrome 90+
Firefox 90+
Edge 97+
30 Memory: 16GB
CPU: i5-10500
GPU: Dedicated 2GB
It is recommended to use the latest version of Chrome browser
(Enable hardware acceleration in the browser)
15 Memory: 8GB
CPU: i3-8300
GPU: Integrated Intel 1GB
Mac Chrome 98+
Firefox 96+
Safari 14+
30 2019 MacBook
Memory: 16GB (2667MHz)
CPU: i7 (6-core 2.60GHz)
GPU: AMD Radeon 5300M
Android Chrome
Firefox Browser
30 High-end devices (e.g., Qualcomm Snapdragon 8 Gen1) It is recommended to use Chrome, or Firefox browsers and other mainstream browsers
20 Mid-range devices (e.g., MediaTek Dimensity 8000-MAX)
10 Low-end devices (e.g., Qualcomm Snapdragon 660)
iOS Chrome
Safari
Firefox
30 iPhone 13 - Requires iOS 14.4 or above
- It is recommended to use Chrome or Safari browsers
20 iPhone XR

Implementation Steps

Registering the Plugin

import { VirtualBackground } from 'trtc-sdk-v5/plugins/video-effect/virtual-background';
let trtc = TRTC.create({ plugins: [VirtualBackground] });

Starting the Local Camera

await trtc.startLocalVideo();

Enabling the Virtual Background Plugin

await trtc.startPlugin('VirtualBackground', {
  sdkAppId: 123123,
  userId: 'userId_123',
  userSig: 'your_userSig'
});

Note: The first time you enable this feature, it requires loading computing resources, which has been automatically processed for you. For a more ultimate startup experience, please refer to the end of the document: FAQ > How to speed up startup?

Updating Parameters as Needed

// Changing to an image background
await trtc.updatePlugin('VirtualBackground', {
  type: 'image',
  src: 'https://picsum.photos/seed/picsum/200/300'
});

Disabling Virtual Background

await trtc.stopPlugin('VirtualBackground');

API Documentation

trtc.startPlugin('VirtualBackground', options)

Used to enable the virtual background.

options

Name Type Attributes Description
sdkAppId number Required Current application ID
userId string Required Current user ID
userSig string Required UserSig corresponding to the user ID
type string Optional - image for image background
- blur for background blur (default)
blurLevel number Optional - Set the level of blur when type is set to blur. Default value is 3, with a range from 1 to 10.
src string Required if type is image Image URL, such as https://picsum.photos/seed/picsum/200/300
onError (error) => {} Optional Callback for errors that occur during runtime
- error.code=10000003 indicates high rendering latency
- error.code=10000006 indicates insufficient browser feature support, which may lead to lagging. Recommended solutions can be found in the Common Issues section at the end of the document.

Example:

await trtc.startPlugin('VirtualBackground', {
  sdkAppId: 123123,
  userId: 'userId_123',
  userSig: 'your_userSig',
  type: 'image',
  src: 'https://picsum.photos/seed/picsum/200/300'
});

trtc.updatePlugin('VirtualBackground', options)

Allows modification of virtual background parameters.

options

Name Type Attributes Description
type string Required - image for image background
- blur for background blur
blurLevel number Optional - Set the level of blur when type is set to blur. Default value is 3, with a range from 1 to 10.
src string Required if type is image Image URL, such as https://picsum.photos/seed/picsum/200/300

Example:

await trtc.updatePlugin('VirtualBackground', {
  type: 'blur',
  blurLevel: 6
});

trtc.stopPlugin('VirtualBackground')

Disables the virtual background.

FAQs

1. When running the demo in Chrome, the video appears upside down and laggy

This plugin uses GPU acceleration, and you need to enable hardware acceleration mode in your browser settings. You can copy and paste chrome://settings/system into your browser's address bar and enable hardware acceleration mode.

2. When device performance is insufficient, causing high latency and rendering issues

You can reduce video resolution or frame rate by listening to events.

async function onError(error) {
  const { code } = error;
  if (code === 10000003 || code === 10000006) {
    // Reduce resolution and frame rate
    await trtc.updateLocalVideo({
      option: {
        profile: '480p_2'
      },
    });
    // await trtc.stopPlugin('VirtualBackground'); // Or disable the plugin
  }
}
await trtc.startPlugin('VirtualBackground', {
  ...// Other parameters
  onError,
});

2. When device performance is insufficient and causes high latency with long rendering times

You can reduce video resolution or frame rate by listening to events.

async function onError(error) {
  const { code } = error;
  if (code === 10000003 || code === 10000006) {
    // Reduce resolution and frame rate
    await trtc.updateLocalVideo({
      option: {
        profile: '480p_2'
      },
    });
    // await trtc.stopPlugin('VirtualBackground'); // Or disable the plugin
  }
}
await trtc.startPlugin('VirtualBackground', {
  ...// Other parameters
  onError,
});

3. How to start the plugin faster? (Manually deploy virtual background model files)

The principle of this plugin is: after capturing from the camera, perform portrait segmentation locally through machine learning models, and then upload the results to the backend.

When first enabled, it will automatically download machine learning models from Tencent Cloud CDN, but Tencent Cloud CDN and the static file address of your webpage are not from the same-origin, which will reduce some loading speeds. The best solution is to deploy model files on your own static resource server, such as deploying them together with web services.

The acceleration method is very simple, unzip assets.zip into your project, such as ./src/assets/, and specify this address when creating a TRTC instance:

const trtc = TRTC.create({ plugins: [VirtualBackground], assetsPath: './src/assets/' });