IVideoFrameObserver

Video frame observer.

You can call RegisterVideoFrameObserver to register or unregister the IVideoFrameObserver video observer.

OnCaptureVideoFrame

Obtains video data captured by the local device.

public virtual bool OnCaptureVideoFrame(VIDEO_SOURCE_TYPE sourceType, VideoFrame videoFrame)

You can obtain the raw video data captured by the local device in this callback and perform pre-processing as needed. After completing the pre-processing, you can directly modify videoFrame in this callback and return true to send the modified video data to the SDK.

Note:
  • If the video data type you obtain is RGBA, the SDK does not support processing the Alpha channel values.
  • When modifying parameters in videoFrame, it is recommended to ensure that the modified parameters match the actual conditions of the video frame in the buffer. Otherwise, unexpected rotation, distortion, or other issues may occur in the local preview or the remote video.

Trigger Timing

Triggered by the SDK when each video frame is captured after successfully registering the video data observer.

Parameters

sourceType
Type of video source, which may include: camera, screen, or media player. See VIDEO_SOURCE_TYPE.
videoFrame
Video frame data. See VideoFrame.

Return Values

  • true: Instructs the SDK to receive the video frame.
  • false: Instructs the SDK to discard the video frame.

OnPreEncodeVideoFrame

Obtains local video data before encoding.

public virtual bool OnPreEncodeVideoFrame(VIDEO_SOURCE_TYPE sourceType, VideoFrame videoFrame)

After successfully registering the video data observer, the SDK triggers this callback when each video frame is captured. You can obtain the video data before encoding in this callback and process it as needed based on your scenario. After processing, you can pass the processed video data back to the SDK in this callback.

Note:
  • You need to set the position parameter of RegisterVideoFrameObserver to POSITION_PRE_ENCODER (1 << 2) to use this callback to obtain local video data before encoding.
  • The video data obtained here has already been pre-processed, such as cropping, rotation, and beautification.
  • When modifying parameters in videoFrame, it is recommended to ensure that the modified parameters match the actual conditions of the video frame in the buffer. Otherwise, unexpected rotation, distortion, or other issues may occur in the local preview or the remote video.

Parameters

sourceType
Type of video source. See VIDEO_SOURCE_TYPE.
videoFrame
Video frame data. See VideoFrame.

Return Values

  • true: Instructs the SDK to receive the video frame.
  • false: Instructs the SDK to discard the video frame.

OnRenderVideoFrame

Obtains video data sent by the remote user.

public virtual bool OnRenderVideoFrame(string channelId, uint remoteUid, VideoFrame videoFrame)

After successfully registering the video data observer, the SDK triggers this callback when each video frame is captured. You can obtain the video data sent by the remote user before rendering and process it as needed based on your scenario.

Note: Unity only supports sending YUV format video data back to the SDK. Make sure to set mode to INTPTR when calling RegisterVideoFrameObserver to register the raw video frame observer.

Parameters

remoteUid
Remote user ID who sent the video frame.
videoFrame
Video frame data. See VideoFrame.
channelId
Channel ID.

Return Values

  • true: Instructs the SDK to receive the video frame.
  • false: Instructs the SDK to discard the video frame.