Tutorial: Data Messages

In real-time audio/video calls, besides the media streams, you may also need to send application-level data messages to other users in the room, such as bullet comments, likes, quiz status, game commands, etc. TRTC Web SDK provides two ways to transmit data messages: Custom Message and SEI (Supplemental Enhancement Information) Message.

• Custom Message: Sent through an independent data channel, unrelated to audio/video streams. Suitable for transmitting control commands, status synchronization, and other text or binary data with high real-time requirements.
• SEI Message: Embedded in the header of video frames and sent along with the video stream. Suitable for information strongly associated with the video image, such as live quiz countdowns, video coordinate annotations, lyrics synchronization, etc.

Demo

Custom Message

• Supported SDK version: v5.6.0+

• Only TRTC.TYPE.ROLE_ANCHOR can call trtc.sendCustomMessage.

• You should call this api after TRTC.enterRoom successfully.

• The custom message will be sent in order and as reliably as possible, but it's possible to loss messages in a very bad network. The receiver will also receive the message in order.

Send Custom Message

const trtc = TRTC.create();
await trtc.enterRoom({ sdkAppId, userId, userSig, roomId: 12345 })
// send custom message
const data = new TextEncoder().encode('hello').buffer;
trtc.sendCustomMessage({ cmdId: 1, data });

Receive Custom Message

Listen for the event TRTC.EVENT.CUSTOM_MESSAGE to receive custom message.

// receive custom message
trtc.on(TRTC.EVENT.CUSTOM_MESSAGE, event => {
   // event.userId: remote userId.
   // event.cmdId: message cmdId.
   // event.seq: message sequence number.
   // event.data: custom message data, type is ArrayBuffer.
   console.log(`received custom msg from ${event.userId}, message: ${new TextDecoder().decode(event.data)}`)
})

SEI Message

The header of a video frame has a header block called supplemental enhancement information (SEI). It is additional data inserted into the video bitstream to convey extra information. You can add a lot of information into SEI, such as parameters of the camera or encoder; time; closed captions; lyrics; and copyright info.

Send SEI Message

You can use trtc.sendSEIMessage to send SEI message. Because the SEI is inserted to the video stream, you should call it after you started local video.

// 1. enable SEI
const trtc = TRTC.create({ enableSEI: true })
// 2. enter room & start local video
await trtc.enterRoom({ sdkAppId, userId, userSig, roomId: 12345 })
await trtc.startLocalVideo();
// 3. send SEI
const unit8Array = new Uint8Array([1, 2, 3]);
trtc.sendSEIMessage(unit8Array.buffer);

Note:

1. Supported SDK version: v5.3.0+. Supported browsers: Chrome 86+, Edge 86+, Opera 72+ browsers(Chromium Based Browser M86+).
2. Maximum 1KB(Byte) sent in a single call, maximum 30 calls per second, maximum 8KB sent per second.
3. Since SEI is sent along with video frames, there is a possibility that video frames may be lost due to poor network, and therefore SEI may be lost as well. The number of times it can be sent can be increased within the frequency limit, and the business side needs to do message de-duplication on the receiving side.
4. SEI cannot be sent without trtc.startLocalVideo and cannot be received without trtc.startRemoteVideo.
5. Only H264 encoder is supported to send and receive SEI.

Receive SEI Message

Refer to TRTC.EVENT.SEI_MESSAGE

// 1. enable SEI
const trtc = TRTC.create({ enableSEI: true })
// 2. receive SEI
trtc.on(TRTC.EVENT.SEI_MESSAGE, event => {
 console.log(`received sei message from ${event.userId}, data: ${event.data}, streamType: ${event.streamType}`)
})