IVideoFrameObserver

Video observer.

You can call registerVideoFrameObserver to register or unregister the IVideoFrameObserver video observer.

onCaptureVideoFrame

Gets video data captured by the local device.

onCaptureVideoFrame?(
    sourceType: VideoSourceType,
    videoFrame: VideoFrame
  ): void;

You can obtain the raw video data captured by the local device in the callback.

Note:
  • If the video data you obtain is in RGBA format, the SDK does not support processing the Alpha channel value.
  • When modifying parameters in videoFrame, ensure the modified parameters match the actual video frame in the buffer. Otherwise, unexpected rotation, distortion, or other issues may occur in the local preview or remote video.
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.

Trigger Timing

After successfully registering the video data observer, when the SDK captures each video frame.

Parameters

sourceType
Video source type, which can be: camera, screen, or media player. See VideoSourceType.
videoFrame
Video frame data. See VideoFrame.
Note: The default format of the video frame data obtained through this callback is:
  • Android: I420
  • iOS: I420

onPreEncodeVideoFrame

Retrieves local video data before encoding.

onPreEncodeVideoFrame?(
    sourceType: VideoSourceType,
    videoFrame: VideoFrame
  ): void;

After successfully registering the video data observer, the SDK triggers this callback for each captured video frame. You can use this callback to retrieve the video data before encoding and process it as needed. After processing, you can pass the processed video data back to the SDK in this callback.

Note:
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.
  • The video data obtained here has been pre-processed, such as cropping, rotation, and beautification.
  • When modifying parameters in videoFrame, make sure the modified parameters match the actual video frame in the buffer. Otherwise, unexpected issues such as incorrect rotation or distortion may occur in the local preview or remote video.

Parameters

sourceType
Type of video source. See VideoSourceType.
videoFrame
Video frame data. See VideoFrame.
Note: The default video frame data format obtained through this callback is:
  • Android: I420
  • iOS: I420

onRenderVideoFrame

Retrieves video data sent by the remote user.

onRenderVideoFrame?(
    channelId: string,
    remoteUid: number,
    videoFrame: VideoFrame
  ): void;

After successfully registering the video data observer, the SDK triggers this callback for each captured video frame. You can use this callback to retrieve the video data sent by the remote user before rendering and process it as needed.

Note:
  • If the video data type is RGBA, the SDK does not support processing the Alpha channel.
  • It is recommended to implement this callback using the C++ API.
  • Due to framework limitations, this callback does not support sending the processed video data back to the SDK.
  • When modifying parameters in videoFrame, make sure the modified parameters match the actual video frame in the buffer. Otherwise, unexpected issues such as incorrect rotation or distortion may occur in the local preview or remote video.

Parameters

remoteUid
ID of the remote user who sent the video frame.
videoFrame
Video frame data. See VideoFrame.
Note: The default video frame data format obtained through this callback is:
  • Android: I420
  • iOS: I420
channelId
Channel ID.