Local Video Transcoding

startLocalVideoTranscoder

Starts local video compositing.

virtual int startLocalVideoTranscoder(const LocalTranscoderConfiguration& config) = 0;

You can call this method to locally merge multiple video streams into one, such as camera video, screen sharing, media player, remote video, video files, images, etc., and publish the mixed video stream to the channel. Here is a typical use case for implementing portrait picture-in-picture:

Call enableVirtualBackground and set the custom background image to BACKGROUND_NONE to separate the portrait from the background in the camera video.
Call startScreenCapture to start capturing the screen sharing video stream.
Call startLocalVideoTranscoder and set the portrait video source as one of the sources for local video compositing to achieve portrait picture-in-picture in the final composited video.

Note:

Local video compositing requires high device performance. It is recommended to use high-performance devices.
To mix locally captured video streams, the SDK supports the following combinations:
- Windows: Up to 4 camera streams + 4 screen sharing streams.
- macOS: Up to 4 camera streams + 1 screen sharing stream.
- Android and iOS: Up to 2 camera streams (requires dual-camera or external camera support) + 1 screen sharing stream.
When configuring local video mixing, make sure the layer index of the portrait video stream is higher than that of the screen sharing stream. Otherwise, the portrait will be covered and not visible in the final composited video.

Scenario

Applicable to scenarios such as remote meetings, live streaming, and online education. It helps users view and manage multiple video streams and supports features like portrait picture-in-picture.

Timing

To mix locally captured video streams, call this method after calling startCameraCapture or startScreenCapture.
To publish the composited video stream to the channel, set publishTranscodedVideoTrack to true in ChannelMediaOptions when calling joinChannel or updateChannelMediaOptions.

Related Callbacks

If this method fails, the SDK triggers the onLocalVideoTranscoderError callback to report the failure reason.

Parameters

config

Local video compositing configuration. See LocalTranscoderConfiguration.

Note:

The maximum resolution for each video stream participating in local compositing is 4096 × 2160. Exceeding this limit will cause compositing to fail.
The maximum resolution of the composited video stream is 4096 × 2160.

Return Values

0: Success.
< 0: Failure.

stopLocalVideoTranscoder

Stops local video mixing.

virtual int stopLocalVideoTranscoder() = 0;

After calling startLocalVideoTranscoder, if you want to stop local video mixing, you can call this method.

Return Values

0: Success.
< 0: Failure.

updateLocalTranscoderConfiguration

Updates the local video mixing configuration.

virtual int updateLocalTranscoderConfiguration(const LocalTranscoderConfiguration& config) = 0;

After calling startLocalVideoTranscoder, if you need to update the local video mixing configuration, you can call this method.

Note: If you need to update the video source type used for local video mixing, such as adding a second camera or screen for video capture, call this method after calling startCameraCapture or startScreenCapture.

Parameters

config: Local video mixing configuration. See LocalTranscoderConfiguration.

Return Values

0: Success.
< 0: Failure.

onLocalVideoTranscoderError

Callback triggered when an error occurs during local video mixing.

virtual void onLocalVideoTranscoderError(const TranscodingVideoStream& stream, VIDEO_TRANSCODER_ERROR error)

When calling startLocalVideoTranscoder or updateLocalTranscoderConfiguration fails, the SDK triggers this callback to report the failure reason.

Parameters

stream: The video stream that could not be mixed during video mixing. See TranscodingVideoStream.
error: The reason for local video mixing failure. See VIDEO_TRANSCODER_ERROR.