Tutorial: Implement 3D spatial audio

Overview

In scenarios such as the social metaverse, users need some stereo sound effects in conjunction with the scenario to allow users to perceive the presence of other users around the game character and their corresponding distance and orientation to improve the fun of voice calls, in addition to simple voice communication. In this article, we describe how to enable 3D spatial audio in developing TRTC applications.

Prerequisites

Feature Description

Reference Quick Start Call to implement a basic audio/video call process.

Integrated SDK

Since the spatial audio feature is not required by all applications, we have added a trtc-spatial.js in the node_modules/trtc-js-sdk directory to specifically implement 3D spatial audio.

// When your application does not need spatial audio
import TRTC from 'trtc-js-sdk';
// // When your application needs spatial audio
import TRTC from 'trtc-js-sdk/trtc-spatial.js';

Init

// create client
const client = TRTC.createClient({ mode: 'rtc', sdkAppId, userId, userSig });
const { detail } = await TRTC.checkSystemRequirements();
// If you need to use spatial audio, you should enable it before joining the room.
if (detail.isSpatialAudioSupported) {
  client.enable3DSpatialAudioEffect(true);
}
// join room
await client.join({ roomId: 8888 });
client.on('stream-added', event => {
  const remoteStream = event.stream;
  console.log('New remote stream:' + remoteStream.getId());
  // subscribe remote user
  client.subscribe(remoteStream);
});
client.on('stream-subscribed', event => {
  const remoteStream = event.stream;
  console.log('subscribe succrss：' + remoteStream.getId());
  // play
  remoteStream.play('remote_stream-' + remoteStream.getId());
});
// When you successfully subscribe a user, you can update their location information in real time.
client.updateRemote3DSpatialPosition('tom', [100, 200, 0]);
// You can also update your location information
client.updateSelf3DSpatialPosition([100, 0, 0], [2, 1, 0], [-1, 2, 0], [0, 0, 1]);

API

Client.enable3DSpatialAudioEffect(enabled): void

Enable 3D spatial audio effects.

If you need to use spatial audio, you should enable it before you join the room.

eg：

// enable
// Good:
client.enable3DSpatialAudioEffect(true);
await client.join({ roomId: 8888 });
// Bad:
await client.join({ roomId: 8888 });
client.enable3DSpatialAudioEffect(true);
client.enable3DSpatialAudioEffect(false);

Client.updateSelf3DSpatialPosition(position, axisRight, axisForward, axisUp): void

Update the position and orientation of the listener.

eg：

client.updateSelf3DSpatialPosition([100, 0, 0], [2, 1, 0], [-1, 2, 0], [0, 0, 1])

Client.updateRemote3DSpatialPosition(userId, position): void

You can get the location information of remote users through your own synchronization service. and update the location information of the remote user.

eg：

client.updateRemote3DSpatialPosition('tom', [100, 200, 0]);

Client.set3DSpatialReceivingRange(range): void

Set the maximum range of audio reception distance.

eg：

client.set3DSpatialReceivingRange(10000);