IVideoFrameObserver

Video frame observer.

You can call registerVideoFrameObserver to register or unregister the IVideoFrameObserver video observer.

onCaptureVideoFrame

Gets the video data captured by the local device.

onCaptureVideoFrame?(
    sourceType: VideoSourceType,
    videoFrame: VideoFrame
  ): void;

You can get the raw video data captured by the local device in this callback.

Note:
  • If the video data format you receive is RGBA, the SDK does not support processing the Alpha channel.
  • When modifying parameters in videoFrame, ensure they match the actual video frame data in the buffer. Otherwise, unexpected rotation or distortion may occur in the local preview or remote video.
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.

Trigger Timing

After successfully registering the video data observer, this callback is triggered when each video frame is captured by the SDK.

Parameters

sourceType
Type of video source, which may include camera, screen, or media player. See VideoSourceType.
videoFrame
Video frame data. See VideoFrame.
Note: The default format of the video frame data obtained through this callback is:
  • macOS: I420
  • Windows: YUV420

onPreEncodeVideoFrame

Gets the video data before local encoding.

onPreEncodeVideoFrame?(
    sourceType: VideoSourceType,
    videoFrame: VideoFrame
  ): void;

After successfully registering the video data observer, the SDK triggers this callback when each video frame is captured. You can get the video data before encoding and process it as needed. After processing, you can pass the processed video data back to the SDK in this callback.

Note:
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.
  • The video data obtained here has already been pre-processed, such as cropping, rotation, beautification, etc.
  • When modifying parameters in videoFrame, ensure they match the actual video frame data in the buffer. Otherwise, unexpected rotation or distortion may occur in the local preview or remote video.

Parameters

sourceType
Type of video source. See VideoSourceType.
videoFrame
Video frame data. See VideoFrame.
Note: The default format of the video frame data obtained through this callback is:
  • macOS: I420
  • Windows: YUV420

onRenderVideoFrame

Gets the video data sent by the remote user.

onRenderVideoFrame?(
    channelId: string,
    remoteUid: number,
    videoFrame: VideoFrame
  ): void;

After successfully registering the video data observer, the SDK triggers this callback when each video frame is captured. You can get the video data sent by the remote user before rendering and process it as needed.

Note:
  • If the video data format you receive is RGBA, the SDK does not support processing the Alpha channel.
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.
  • When modifying parameters in videoFrame, ensure they match the actual video frame data in the buffer. Otherwise, unexpected rotation or distortion may occur in the local preview or remote video.

Parameters

remoteUid
The user ID of the remote user who sent the video frame.
videoFrame
Video frame data. See VideoFrame.
Note: The default format of the video frame data obtained through this callback is:
  • macOS: I420
  • Windows: YUV420
channelId
Channel ID.