Classes
class	VideoFrame

Public Member Functions
abstract boolean	onCaptureVideoFrame (VideoFrame videoFrame)

boolean	onPreEncodeVideoFrame (VideoFrame videoFrame)

abstract boolean	onRenderVideoFrame (int uid, VideoFrame videoFrame)

int	getVideoFormatPreference ()

boolean	getRotationApplied ()

boolean	getMirrorApplied ()

int	getObservedFramePosition ()

boolean	isMultipleChannelFrameWanted ()

boolean	onRenderVideoFrameEx (String channelId, int uid, VideoFrame videoFrame)

Static Public Attributes
static final int	FRAME_TYPE_YUV420 = 0

static final int	FRAME_TYPE_YUV422 = 1

static final int	FRAME_TYPE_RGBA = 2

static final int	POSITION_POST_CAPTURER = 1 << 0

static final int	POSITION_PRE_RENDERER = 1 << 1

static final int	POSITION_PRE_ENCODER = 1 << 2

Detailed Description

The IVideoFrameObserver class.

Member Function Documentation

◆ onCaptureVideoFrame()

abstract boolean io.agora.rtc.IVideoFrameObserver.onCaptureVideoFrame ( VideoFrame videoFrame )

abstract

Gets the video data captured by the local camera.

Since: v3.4.5

After you successfully register the raw video frame observer, the SDK triggers this callback each time a video frame is received. You can get the video data captured by the local camera in videoFrame and then process the video data according to your scenario. After processing, you can use videoFrame to pass the processed video data back to the SDK.

Note

If you get the video data in RGBA color encoding format, Agora does not support using this callback to send the processed data in RGBA color encoding format back to the SDK.
The video data obtained through this callback has not undergone preprocessing, such as watermarking, cropping content, rotating, or image enhancement.

Parameters

videoFrame The video frame. See VideoFrame.

Returns

true: Sets the SDK to receive the video frame.
false: Sets the SDK to discard the video frame.

◆ onPreEncodeVideoFrame()

boolean io.agora.rtc.IVideoFrameObserver.onPreEncodeVideoFrame ( VideoFrame videoFrame )

Gets the local pre-encoded video data.

Since: v3.4.5

After you successfully register the raw video frame observer and use getObservedFramePosition to set the observation POSITION_PRE_ENCODER (1 << 2), the SDK triggers this callback each time a video frame is received. You can get the local pre-encoded video data in videoFrame and then process the video data according to your scenario. After processing, you can use videoFrame to pass the processed video data back to the SDK.

Note

If you get the video data in RGBA color encoding format, Agora does not support using this callback to send the processed data in RGBA color encoding format back to the SDK.
The video data obtained through this callback has not undergone preprocessing, such as watermarking, cropping content, rotating, or image enhancement.

Parameters

videoFrame The video frame. See VideoFrame.

Returns

true: Sets the SDK to receive the video frame.
false: Sets the SDK to discard the video frame.

◆ onRenderVideoFrame()

abstract boolean io.agora.rtc.IVideoFrameObserver.onRenderVideoFrame	(	int	uid,
		VideoFrame	videoFrame
	)

abstract

Gets the incoming remote video data.

Since: v3.4.5

After you successfully register the raw video frame and set the return value of isMultipleChannelFrameWanted as false, the SDK triggers this callback each time a video frame is received. You can get the incoming remote video data in videoFrame and then process the video data according to your scenario. After processing, you can use videoFrame to pass the processed video data back to the SDK.

Note: If you get the video data in RGBA color encoding format, Agora does not support using this callback to send the processed data in RGBA color encoding format back to the SDK.

Parameters

uid	The user ID of the remote user.
videoFrame	The video frame. See `VideoFrame`.

Returns

true: Sets the SDK to receive the video frame.
false: Sets the SDK to discard the video frame.

◆ getVideoFormatPreference()

int io.agora.rtc.IVideoFrameObserver.getVideoFormatPreference ( )

Sets the format of the raw video data output by the SDK.

Since: v3.4.5

If you want to get raw video data in a color encoding format other than YUV 420, you need to implement the getVideoFormatPreference callback in the IVideoFrameObserver class. After you successfully register the raw video frame observer, the SDK triggers this callback each time a video frame is received. You need to set the desired color encoding format of the video data in the return value of this callback.

Returns

The color encoding format of the raw video data output by the SDK. The following values are supported:

FRAME_TYPE_YUV420 (0): (Default) YUV 420
FRAME_TYPE_YUV422 (1): YUV 422
FRAME_TYPE_RGBA (2): RGBA

◆ getRotationApplied()

boolean io.agora.rtc.IVideoFrameObserver.getRotationApplied ( )

Sets whether to rotate the raw video data output by the SDK.

Since: v3.4.5

If you want to get raw video data that has been rotated according to the value of rotation in VideoFrame, you need to implement the getRotationApplied callback in the IVideoFrameObserver class. After you successfully register the raw video frame observer, the SDK triggers this callback each time a video frame is received. You need to set whether to rotate the observed raw video data in the return value of this callback.

Note: This function supports video data in RGBA and YUV 420 color encoding formats only.

Returns

Whether to rotate the raw video data output by the SDK:

true: Rotate the raw video data.
false: (Default) Do not rotate the raw video data.

◆ getMirrorApplied()

boolean io.agora.rtc.IVideoFrameObserver.getMirrorApplied ( )

Sets whether to mirror the raw video data output by the SDK.

Since: v3.4.5

If you want to get the mirrored raw video data, you need to implement the getMirrorApplied callback in the IVideoFrameObserver class. After you successfully register the raw video frame observer, the SDK triggers this callback each time a video frame is received. You need to set whether to mirror the observed raw video data in the return value of this callback.

Note: This function supports video data in RGBA and YUV 420 color encoding formats only.

Returns

Whether to mirror the raw video data output by the SDK:

true: Mirror the raw video data.
false: (Default) Do not mirror the raw video data.

◆ getObservedFramePosition()

int io.agora.rtc.IVideoFrameObserver.getObservedFramePosition ( )

Sets the video observation positions.

Since: v3.4.5

After you successfully register the raw video frame observer, the SDK uses the getObservedFramePosition callback to determine at each specific video-frame processing node whether to trigger the following callbacks:

onCaptureVideoFrame
onPreEncodeVideoFrame
onRenderVideoFrame or onRenderVideoFrameEx

You can set the position or positions that you want to observe by modifying the return value of getObservedFramePosition according to your scenario.

Note

To observe multiple positions, use | (the OR operator).
The default return value of this callback is POSITION_POST_CAPTURER (1 << 0) and POSITION_PRE_RENDERER (1 << 1).
To conserve system resources of the device, you can reduce the number of observation positions appropriately according to your scenario.

Returns

The bit mask of the observation positions. The following values are supported:

POSITION_POST_CAPTURER (1 << 0): (Default) The position to observe the video data captured by the local camera, which corresponds to the onCaptureVideoFrame callback.
POSITION_PRE_RENDERER (1 << 1): (Default) The position to observe the incoming remote video data, which corresponds to the onRenderVideoFrame or onRenderVideoFrameEx callback.
POSITION_PRE_ENCODER (1 << 2): The position to observe the local pre-encoded video data, which corresponds to the onPreEncodeVideoFrame callback.

◆ isMultipleChannelFrameWanted()

boolean io.agora.rtc.IVideoFrameObserver.isMultipleChannelFrameWanted ( )

Sets whether the SDK outputs remote video data received in multiple channels.

Since: v3.4.5

In a multi-channel (RtcChannel) scenario, if you want to get the remote video data received in multiple channels, you need to implement the isMultipleChannelFrameWanted callback in the IVideoFrameObserver class and set the return value of this callback to true. The SDK triggers the onRenderVideoFrameEx callback each time a video frame is received, from which you can get the expected multi-channel video data.

Note: The SDK chooses to trigger either the onRenderVideoFrame or onRenderVideoFrameEx callback depending on the return value that you set in the isMultipleChannelFrameWanted callback. Agora recommends that you set the return value as true in a multi-channel scenario.

Returns

Whether the SDK outputs remote video data received in multiple channels:

true: The SDK uses onRenderVideoFrameEx to output remote video data received in multiple channels.
false: (Default) The SDK uses onRenderVideoFrame to output remote video data received in a single channel.

◆ onRenderVideoFrameEx()

boolean io.agora.rtc.IVideoFrameObserver.onRenderVideoFrameEx	(	String	channelId,
		int	uid,
		VideoFrame	videoFrame
	)

Gets the remote video data received in multiple channels.

Since: v3.4.5

After you successfully register the raw video frame observer and set the return value of isMultipleChannelFrameWanted as true, the SDK triggers this callback each time a video frame is received. You get the remote video data received in multiple channels in videoFrame and then process the video data according to your scenario. After processing, you can pass the processed video data back to the SDK.

Note: If you get the video data in RGBA color encoding format, Agora does not support using this callback to send the processed data in RGBA color encoding format back to the SDK.

Parameters

channelId	The channel name.
uid	The user ID of the remote user.
videoFrame	The video frame. See `VideoFrame`.

Returns

true: Sets the SDK to receive the video frame.
false: Sets the SDK to discard the video frame.

Member Data Documentation

◆ FRAME_TYPE_YUV420

final int io.agora.rtc.IVideoFrameObserver.FRAME_TYPE_YUV420 = 0

static

0: (Default) YUV 420

◆ FRAME_TYPE_YUV422

final int io.agora.rtc.IVideoFrameObserver.FRAME_TYPE_YUV422 = 1

static

1: YUV422

◆ FRAME_TYPE_RGBA

final int io.agora.rtc.IVideoFrameObserver.FRAME_TYPE_RGBA = 2

static

2: RGBA

◆ POSITION_POST_CAPTURER

final int io.agora.rtc.IVideoFrameObserver.POSITION_POST_CAPTURER = 1 << 0

static

1 << 0: (Default) The position to observe the video data captured by the local camera, which corresponds to the onCaptureVideoFrame callback.

◆ POSITION_PRE_RENDERER

final int io.agora.rtc.IVideoFrameObserver.POSITION_PRE_RENDERER = 1 << 1

static

1 << 1: (Default) The position to observe the incoming remote video data, which corresponds to the onRenderVideoFrame or onRenderVideoFrameEx callback.

◆ POSITION_PRE_ENCODER

final int io.agora.rtc.IVideoFrameObserver.POSITION_PRE_ENCODER = 1 << 2

static

1 << 2: The position to observe the local pre-encoded video data, which corresponds to the onPreEncodeVideoFrame callback.

Classes

Public Member Functions

Static Public Attributes

Detailed Description

Member Function Documentation

◆ onCaptureVideoFrame()

◆ onPreEncodeVideoFrame()

◆ onRenderVideoFrame()

◆ getVideoFormatPreference()

◆ getRotationApplied()

◆ getMirrorApplied()

◆ getObservedFramePosition()

◆ isMultipleChannelFrameWanted()

◆ onRenderVideoFrameEx()

Member Data Documentation

◆ FRAME_TYPE_YUV420

◆ FRAME_TYPE_YUV422

◆ FRAME_TYPE_RGBA

◆ POSITION_POST_CAPTURER

◆ POSITION_PRE_RENDERER

◆ POSITION_PRE_ENCODER