Tutorial: 42-advanced-face-detection

Function Description

This document details how to use the TRTC Web SDK's Face Detection plugin to monitor real-time face presence changes in camera video streams. The plugin employs intelligent debounce logic, triggering callbacks only when face appearance/disappearance states genuinely change, preventing false positives caused by lighting variations or brief obstructions.

Prerequisites

• TRTC Web SDK version >= 5.15.0
• Camera video stream must be enabled first.

Platform Compatibility

Implementation Steps

Deploying Static Resources

The plugin relies on static resource files. To ensure the browser can load and run these files correctly, you need to complete the following steps:

1. Publish the node_modules/trtc-sdk-v5/assets directory to your CDN or static resource server.
2. Pass your CDN address to the assetsPath parameter when creating the TRTC instance, e.g., TRTC.create({ assetsPath: 'https://xxxx/assets' }). The SDK will load the necessary resource files on demand.

• If your SDK version is lower than v5.10.0, you need to download and extract https://web.sdk.qcloud.com/trtc/webrtc/v5/assets/assets.zip, then publish it to your CDN or static resource server.
• If the Host URL of the files differs from your web application's Host URL, you need to enable CORS policy for the file domain.
• Do not host the directory files under an HTTP service, as loading HTTP resources under an HTTPS domain will be blocked by the browser's security policy.

Registering the Plugin

import { FaceDetection } from 'trtc-sdk-v5/plugins/video-effect/face-detection';
const trtc = TRTC.create({ 
  plugins: [FaceDetection], 
  assetsPath: 'https://xxxx/assets'  // Replace with your static resource address
});

Starting the Local Camera

await trtc.startLocalVideo();

Enabling the Face Detection Plugin

await trtc.startPlugin('FaceDetection', {
  onFaceDetectionStateChanged: (hasFace) => {
    if (hasFace) {
      console.log('Face detected');
      // Execute business logic for face appearance
    } else {
      console.log('Face disappeared');
      // Execute business logic for face disappearance
    }
  },
  detectionInterval: 300,    // Detection interval in ms (Optional, default: 300)
  minConfidence: 0.8,        // Minimum confidence threshold (Optional, default: 0.8)
  missingTimeout: 1000       // Disappearance confirmation time in ms (Optional, default: 1000)
});

Updating Parameters as Needed

// Update detection parameters - increase sensitivity
await trtc.updatePlugin('FaceDetection', {
  onFaceDetectionStateChanged: (hasFace) => {
    console.log('Updated detection state:', hasFace);
  },
  detectionInterval: 200,    // Shorter interval for higher real-time performance
  minConfidence: 0.7,        // Lower threshold for higher sensitivity
  missingTimeout: 1500       // Longer timeout to reduce false negatives
});

Disabling Face Detection

await trtc.stopPlugin('FaceDetection');

Core Features

Intelligent Debounce Logic

The plugin uses an asymmetric debounce mechanism to ensure state change accuracy:

• Face appears (true): Triggered immediately for real-time response.
• Face disappears (false): Triggered after a delay, requiring the missingTimeout condition to be met for confirmation.

This design effectively prevents false positives caused by:

• Brief head turns or looking down
• Sudden lighting changes
• Temporary obstructions

Video Stream State Synchronization

The plugin automatically handles local video stream state changes:

API Documentation

trtc.startPlugin('FaceDetection', options)

Used to start the Face Detection plugin.

options

Example:

await trtc.startPlugin('FaceDetection', {
  onFaceDetectionStateChanged: (hasFace) => {
    if (hasFace) {
      console.log('Face appeared');
    } else {
      console.log('Face disappeared');
    }
  },
  detectionInterval: 300,
  minConfidence: 0.8,
  missingTimeout: 1000
});

trtc.updatePlugin('FaceDetection', options)

Used to modify face detection parameters.

options

Example:

await trtc.updatePlugin('FaceDetection', {
  detectionInterval: 200,
  minConfidence: 0.7,
  missingTimeout: 1500
});

trtc.stopPlugin('FaceDetection')

Stops the Face Detection plugin.

FAQs

1. How to adjust detection sensitivity?

Adjust sensitivity by combining these parameters:

• Increase sensitivity: Lower minConfidence, shorten detectionInterval
• Decrease sensitivity: Raise minConfidence, lengthen missingTimeout

2. Do I need to restart detection after the video stream recovers?

No. The plugin automatically handles video stream state changes and resumes detection upon recovery.