Video Features

Video Basic Features

disableVideo

Disables the video module.

public abstract int disableVideo();
Note: disableVideo affects the internal engine and can be called after leaving the channel. Calling this method resets the entire engine, which may slow down response time. Depending on your needs, you can use the following methods to control specific video module functions:

Timing

You can call this method before or after joining a channel.
  • If called before joining, it enables voice-only mode.
  • If called after joining, it switches from video to voice mode. You can call enableVideo again to switch back to video mode.

Related Callbacks

After this method is successfully called, the remote client triggers the onUserEnableVideo(false) callback.

Return Values

  • 0: Success.
  • < 0: Failure.

enableLocalVideo

Enables or disables local video capture.

public abstract int enableLocalVideo(boolean enabled);

This method disables or re-enables local video capture without affecting the reception of remote video streams. If you call enableLocalVideo(false) in the channel, it also stops publishing the video stream in the channel. To resume video capture, call enableLocalVideo(true), then call updateChannelMediaOptions to set the options parameter to publish the locally captured video stream in the channel. After successfully disabling or enabling local video capture, the remote side triggers the onRemoteVideoStateChanged callback.

Note:
  • You can call this method before or after joining a channel. If called before joining, the setting takes effect after joining the channel.
  • This method enables the internal engine and remains effective after leaving the channel.

Related Callbacks

After successfully disabling or re-enabling the local video capturer, the SDK triggers the onRemoteVideoStateChanged callback on the remote client.

Parameters

enabled
Whether to enable local video capture.
  • true: (default) Enables local video capture.
  • false: Disables local video capture. When disabled, remote users cannot receive the local user's video stream, but the local user can still receive remote users' video streams. No local camera is required when set to false.

Return Values

  • 0: Success.
  • < 0: Failure.

enableVideo

Enables the video module.

public abstract int enableVideo();

By default, the video module is disabled. Call this method to enable the video module. To disable it later, call disableVideo.

Note: This method enables the internal engine and remains effective after leaving the channel. Calling this method resets the entire engine, which may slow down response time. Depending on your needs, you can use the following methods to control specific video module functions: Successfully calling this method resets the settings of enableLocalVideo, muteRemoteVideoStream, and muteAllRemoteVideoStreams, so use it with caution.

Timing

You can call this method before or after joining a channel:
  • If called before joining, it enables the video module.
  • If called during a voice call, the call automatically switches to a video call.

Related Callbacks

Successfully calling this method triggers the onRemoteVideoStateChanged callback on the remote client.

Return Values

  • 0: Success.
  • < 0: Failure.

setVideoEncoderConfiguration

Sets the encoder configuration for the local video.

public abstract int setVideoEncoderConfiguration(VideoEncoderConfiguration config);
Note:
  • This method and getMirrorApplied both support setting mirror effects. Agora recommends using only one of them. Using both may cause mirror effect stacking, making the mirror setting ineffective.
  • The config specified in this method is the maximum value under ideal network conditions. If the video engine cannot render video using the specified config due to unstable network conditions, it will try lower parameters in the list until it finds a usable configuration.

Timing

You can call this method before or after joining a channel. If the user does not need to reset video encoding properties after joining, Agora recommends calling this method before enableVideo to reduce the time to render the first video frame.

Parameters

config
Video encoder configuration. See VideoEncoderConfiguration.

Return Values

  • 0: Success.
  • < 0: Failure.

setVideoEncoderConfigurationEx

Sets the encoder configuration for the local video.

public abstract int setVideoEncoderConfigurationEx(VideoEncoderConfiguration config, RtcConnection connection);

setVideoEncoderConfigurationEx sets the encoder configuration for the local video. Each configuration corresponds to a set of video parameters, including resolution, frame rate, and bitrate.

Note: The config parameter represents the upper limits under ideal network conditions. If the video engine cannot render video using the specified config due to unstable network conditions, it will try lower configurations in the list until a suitable one is found.

Scenario

This method applies to multi-channel scenarios.

Parameters

config
Video encoder configuration. See VideoEncoderConfiguration.
connection
Connection information. See RtcConnection.

Return Values

  • 0: Success.
  • < 0: Failure.

setVideoScenario

Sets the video application scenario.

public abstract int setVideoScenario(Constants.VideoScenario scenarioType);

After this method is successfully called, the SDK automatically enables best practice strategies and adjusts key performance indicators based on the specified video scenario to optimize the video experience.

Note: Call this method before joining a channel.

Parameters

scenarioType
Type of video application scenario. See VideoScenario.
  • APPLICATION_SCENARIO_MEETING (1): Suitable for meeting scenarios. The SDK automatically enables the following strategies:
    • When the low-quality video stream requires a higher bitrate, the SDK automatically enables various network congestion handling techniques to enhance the performance of the low-quality stream and ensure smooth reception on the subscriber side.
    • The SDK monitors the number of subscribers to the high-quality video stream in real time and dynamically adjusts its configuration:
      • If no one subscribes to the high-quality stream, the SDK automatically reduces its bitrate and frame rate to save upstream bandwidth.
      • If someone subscribes to the high-quality stream, the SDK resets it to the configuration set by the most recent call to setVideoEncoderConfiguration; if no configuration was set, the following defaults are used:
        • Resolution: 960 × 540.
        • Frame rate: 15 fps.
        • Bitrate: 1000 Kbps.
    • The SDK monitors the number of subscribers to the low-quality stream in real time and dynamically enables or disables the stream:
      • If you called setDualStreamMode and set it to never send the low-quality stream (DISABLE_SIMULCAST_STREAM), then dynamic adjustment of the low-quality stream in meeting scenarios will not take effect.
      • If no one subscribes to the low-quality stream, the SDK disables it to save upstream bandwidth.
      • If someone subscribes to the low-quality stream, the SDK enables it and resets it to the configuration set by the most recent call to setDualStreamMode; if no configuration was set, the following defaults are used:
        • Resolution: 480 × 272.
        • Frame rate: 15 fps.
        • Bitrate: 500 Kbps.
  • APPLICATION_SCENARIO_1V1 (2): Suitable for one-on-one live streaming scenarios. To meet the needs of low latency and high quality, the SDK optimizes strategies to improve video quality, first frame rendering speed, latency performance on mid- and low-end devices, and smoothness under poor network conditions.
    • Note: This enum value is only applicable to broadcaster-to-broadcaster scenarios.
  • APPLICATION_SCENARIO_LIVESHOW (3): Suitable for live show scenarios. In this scenario, fast video rendering and high quality are critical. The SDK performs various performance optimizations, including:
    • Automatically enabling audio and video frame accelerated rendering to minimize first-frame delay (no need to call enableInstantMediaRendering).
    • Enabling B-frame encoding to improve image quality and bandwidth efficiency.
    • Providing enhanced video quality and smooth playback even under poor network conditions or on low-end devices.

Return Values

  • 0: Success.
  • < 0: Failure.
    • -1: A general error occurred (no specific reason).
    • -4: Video scenario not supported. Possible reason: using the voice SDK instead of the video SDK.
    • -7: The RtcEngine object has not been initialized. Please initialize the RtcEngine object before calling this method.

startPreview [1/2]

Enables local video preview.

public abstract int startPreview();
Note:
  • Mirror mode is enabled by default for local preview.
  • After leaving the channel, the local preview remains active. You need to call stopPreview to stop the local preview.

Timing

This method must be called after calling enableVideo and setupLocalVideo.

Return Values

  • 0: Success.
  • < 0: Failure.

startPreview [2/2]

Enables local video preview and specifies the video source for preview.

public abstract int startPreview(Constants.VideoSourceType sourceType);
Note:
  • Mirror mode is enabled by default for local preview.
  • After leaving the channel, the local preview remains active. You need to call stopPreview to stop the local preview.

Timing

This method must be called after calling enableVideo and setupLocalVideo.

Parameters

sourceType
The type of video source. See VideoSourceType.

Return Values

  • 0: Success.
  • < 0: Failure.

stopPreview [1/2]

Stops local video preview.

public abstract int stopPreview();

After calling startPreview to enable preview, you can call this method to stop the local video preview.

Timing

Call this method before joining or after leaving a channel.

Return Values

  • 0: Success.
  • < 0: Failure.

stopPreview [2/2]

Stops local video preview.

public abstract int stopPreview(Constants.VideoSourceType sourceType);

After calling startPreview, if you need to stop the local video preview, call stopPreview.

Timing

Call this method before joining or after leaving a channel.

Parameters

sourceType
The type of video source. See VideoSourceType.

Return Values

  • 0: Success.
  • < 0: Failure.

onFirstLocalVideoFrame

Callback triggered when the first local video frame is displayed in the local view.

public void onFirstLocalVideoFrame(Constants.VideoSourceType source, int width, int height, int elapsed)

Parameters

source
Video source type. See VideoSourceType.
width
Width of the first local video frame in pixels.
height
Height of the first local video frame in pixels.
elapsed
Time elapsed (in milliseconds) from the local user calling joinChannel to the triggering of this callback. If startPreview was called before joining the channel, this parameter indicates the time from calling startPreview to the event trigger.

onFirstLocalVideoFramePublished

Callback for the first local video frame published.

public void onFirstLocalVideoFramePublished(Constants.VideoSourceType source, int elapsed)
The SDK triggers this callback when any of the following conditions are met:
  • The local client enables the video module and successfully joins the channel by calling joinChannel.
  • The local client calls muteLocalVideoStream(true) followed by muteLocalVideoStream(false).
  • The local client calls disableVideo followed by enableVideo.
  • The local client successfully pushes video frames to the SDK using pushExternalVideoFrameById.

Trigger Timing

This callback is triggered when the first local video frame is published.

Parameters

source
Video source type. See VideoSourceType.
elapsed
Time (in milliseconds) from the local user successfully calling joinChannel to the triggering of this callback.

onFirstRemoteVideoDecoded

Callback when the first remote video frame is received and decoded.

public void onFirstRemoteVideoDecoded(int uid, int width, int height, int elapsed)
Deprecated
This method is deprecated. Use REMOTE_VIDEO_STATE_STARTING(1) or REMOTE_VIDEO_STATE_DECODING(2) in the onRemoteVideoStateChanged callback instead.
The SDK triggers this callback in any of the following scenarios:
  • A remote user joins the channel and sends a video stream.
  • A remote user stops sending video and resumes after 15 seconds. Possible causes include:
    • The remote user leaves the channel.
    • The remote user goes offline.
    • The remote user calls disableVideo to disable video.

Trigger Timing

This callback is triggered when the first remote video frame is received and decoded.

Parameters

uid
Remote user ID.
width
Width of the video stream (in pixels).
height
Height of the video stream (in pixels).
elapsed
Time (in milliseconds) from the local user calling joinChannel to the triggering of this callback.

onFirstRemoteVideoFrame

Callback for the first remote video frame displayed.

public void onFirstRemoteVideoFrame(int uid, int width, int height, int elapsed)
Note: The onFirstRemoteVideoFrame callback is only triggered when the video frame is rendered by the SDK. If you use custom video rendering, this callback is not triggered. You need to implement this functionality outside of the SDK.

Trigger Timing

This callback is triggered when the renderer receives the first frame of remote video.

Parameters

uid
Remote user ID who sent the video stream.
width
Width of the video stream (pixels).
height
Height of the video stream (pixels).
elapsed
Time (in milliseconds) from the local user calling joinChannel to the SDK triggering this callback.

onLocalVideoEvent

onLocalVideoEvent callback.

public void onLocalVideoEvent(Constants.VideoSourceType source, int event)

You can use this callback to get the reason why a local video event occurred.

Parameters

source
Video source type. See VideoSourceType.
event
Local video event type:
  • LOCAL_VIDEO_EVENT_TYPE_SCREEN_CAPTURE_WINDOW_HIDDEN (1): The screen capture window is hidden.
  • LOCAL_VIDEO_EVENT_TYPE_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_HIDDEN (2): The screen capture window recovers from hidden.
  • LOCAL_VIDEO_EVENT_TYPE_SCREEN_CAPTURE_STOPPED_BY_USER (3): The screen capture is stopped by the user.
  • LOCAL_VIDEO_EVENT_TYPE_SCREEN_CAPTURE_SYSTEM_INTERNAL_ERROR (4): A system internal error occurs during screen capture.

onLocalVideoStateChanged

Callback for changes in local video stream state.

public void onLocalVideoStateChanged(Constants.VideoSourceType source, int state, int reason)

You can use this callback to monitor changes in the local video stream state and take appropriate actions based on the reason to better manage and troubleshoot video stream issues.

Note: Note that duplicate frame detection only applies to video frames with resolution greater than 200 × 200, frame rate ≥ 10 fps, and bitrate < 20 Kbps. In general, if a video capture error occurs, you can troubleshoot using the reason parameter in this callback. However, on some devices, Android does not throw any error callback when capture issues occur (e.g., stuttering), so the SDK cannot report the reason for local video state changes. In this case, you can determine whether video frames are not captured by checking if this callback reports LOCAL_VIDEO_STREAM_STATE_CAPTURING or LOCAL_VIDEO_STREAM_STATE_ENCODING, and captureFrameRate in the onLocalVideoStats callback is 0.

Trigger Timing

This callback is triggered in the following situations, with state as LOCAL_VIDEO_STREAM_STATE_FAILED and reason as LOCAL_VIDEO_STREAM_REASON_CAPTURE_FAILURE:
  • On Android 9 and above, the system may reclaim camera permissions after the app runs in the background for some time.
  • On Android 6 and above, if the camera is occupied by a third-party app and then released, the SDK triggers this callback with state as LOCAL_VIDEO_STREAM_STATE_CAPTURING and reason as LOCAL_VIDEO_STREAM_REASON_OK.
  • The camera starts normally but does not output video frames for four consecutive seconds.
  • If the camera outputs video frames, and the SDK detects 15 consecutive duplicate frames, this callback is triggered with state as LOCAL_VIDEO_STREAM_STATE_CAPTURING and reason as LOCAL_VIDEO_STREAM_REASON_CAPTURE_FAILURE.

Parameters

source
The type of video source. See VideoSourceType.
state
  • LOCAL_VIDEO_STREAM_STATE_STOPPED (0): The local video is in the initial state.
  • LOCAL_VIDEO_STREAM_STATE_CAPTURING (1): The local video capturing device is started successfully.
  • LOCAL_VIDEO_STREAM_STATE_ENCODING (2): The first frame of video is encoded successfully.
  • LOCAL_VIDEO_STREAM_STATE_FAILED (3): Failed to start the local video.
reason
  • LOCAL_VIDEO_STREAM_REASON_OK (0): The local video state is normal.
  • LOCAL_VIDEO_STREAM_REASON_FAILURE (1): The local video failed for an unspecified reason.
  • LOCAL_VIDEO_STREAM_REASON_DEVICE_BUSY (3): The local video capturing device is occupied. Prompt the user to check if the camera is used by another app or try rejoining the channel.
  • LOCAL_VIDEO_STREAM_REASON_CAPTURE_FAILURE (4): Failed to capture local video. Prompt the user to check if the video capturing device is working properly, if the camera is used by another app, or try rejoining the channel.
  • LOCAL_VIDEO_STREAM_REASON_CODEC_NOT_SUPPORT (5): Failed to encode local video.
  • LOCAL_VIDEO_STREAM_REASON_DEVICE_NOT_FOUND (8): No local video capturing device found. Prompt the user to check if the camera is properly connected or working, then rejoin the channel.
  • LOCAL_VIDEO_STREAM_REASON_DEVICE_INTERRUPT (14): Video capture interrupted. Possible reasons include:
    • The camera is used by another app. Prompt the user to check if the camera is in use.
    • The device is locked, or the app is sent to the background. You can use a foreground service notification to ensure video capture continues in the background.
  • LOCAL_VIDEO_STREAM_REASON_DEVICE_FATAL_ERROR (15): The video capturing device encountered a fatal error. Prompt the user to close and restart the camera. If the issue persists, check for hardware failure.
  • LOCAL_VIDEO_STREAM_REASON_SCREEN_CAPTURE_FAILURE (21): The current captured window has no data.

onLocalVideoStats

Callback for local video stream statistics.

public void onLocalVideoStats(Constants.VideoSourceType source, LocalVideoStats stats) {}

This callback is triggered every two seconds.

Trigger Timing

This callback is triggered every two seconds.

Parameters

source
Video source type. See VideoSourceType.
stats
Statistics of the local video stream. See LocalVideoStats.

onRemoteVideoStateChanged

Callback when the remote video state changes.

public void onRemoteVideoStateChanged(int uid, int state, int reason, int elapsed)
Note: This callback may not work properly when the number of users (in a communication channel) or hosts (in a live broadcasting channel) exceeds 32.

Trigger Timing

This callback is triggered when the state of the remote video changes.

Parameters

uid
The user ID of the remote user whose video state has changed.
state
The state of the remote video:
  • REMOTE_VIDEO_STATE_STOPPED (0): The remote video is in the initial state. This state is reported under REMOTE_VIDEO_STATE_REASON_LOCAL_MUTED (3), REMOTE_VIDEO_STATE_REASON_REMOTE_MUTED (5), or REMOTE_VIDEO_STATE_REASON_REMOTE_OFFLINE (7).
  • REMOTE_VIDEO_STATE_STARTING (1): The first frame of the remote video packet is received.
  • REMOTE_VIDEO_STATE_DECODING (2): The remote video stream is decoded and playing normally. This state is reported under REMOTE_VIDEO_STATE_REASON_NETWORK_RECOVERY (2), REMOTE_VIDEO_STATE_REASON_LOCAL_UNMUTED (4), REMOTE_VIDEO_STATE_REASON_REMOTE_UNMUTED (6), or REMOTE_VIDEO_STATE_REASON_AUDIO_FALLBACK_RECOVERY (9).
  • REMOTE_VIDEO_STATE_FROZEN (3): The remote video is frozen. This state is reported under REMOTE_VIDEO_STATE_REASON_NETWORK_CONGESTION (1) or REMOTE_VIDEO_STATE_REASON_AUDIO_FALLBACK (8).
  • REMOTE_VIDEO_STATE_FAILED (4): The remote video fails to start. This state is reported by the SDK under REMOTE_VIDEO_STATE_REASON_INTERNAL (0).
reason
The reason for the remote video state change:
  • REMOTE_VIDEO_STATE_REASON_INTERNAL (0): Internal reason.
  • REMOTE_VIDEO_STATE_REASON_NETWORK_CONGESTION (1): Network congestion.
  • REMOTE_VIDEO_STATE_REASON_NETWORK_RECOVERY (2): Network recovery.
  • REMOTE_VIDEO_STATE_REASON_LOCAL_MUTED (3): The local user stops receiving the remote video stream or disables the video module.
  • REMOTE_VIDEO_STATE_REASON_LOCAL_UNMUTED (4): The local user resumes receiving the remote video stream or enables the video module.
  • REMOTE_VIDEO_STATE_REASON_REMOTE_MUTED (5): The remote user stops sending the video stream or disables the video module.
  • REMOTE_VIDEO_STATE_REASON_REMOTE_UNMUTED (6): The remote user resumes sending the video stream or enables the video module.
  • REMOTE_VIDEO_STATE_REASON_REMOTE_OFFLINE (7): The remote user leaves the channel.
  • REMOTE_VIDEO_STATE_REASON_AUDIO_FALLBACK (8): The remote media stream switches to audio-only due to poor network conditions.
  • REMOTE_VIDEO_STATE_REASON_AUDIO_FALLBACK_RECOVERY (9): The remote media stream switches back to video after the network condition improves.
  • REMOTE_VIDEO_STATE_REASON_CODEC_NOT_SUPPORT (13): The local video decoder does not support decoding the remote video stream.
elapsed
Time (ms) from the local user calling joinChannel to the SDK triggering this callback.

onRemoteVideoStats

Callback that reports statistics of the video stream sent by each remote user.

public void onRemoteVideoStats(RemoteVideoStats stats) {}

The SDK triggers this callback every two seconds for each remote user. If there are multiple users or hosts sending video streams in the channel, the SDK triggers this callback multiple times accordingly.

Trigger Timing

This callback is triggered when the SDK collects video stream statistics from remote users every two seconds.

Parameters

stats
Statistics of the remote video stream. See RemoteVideoStats.

onRemoteVideoTransportStats

Callback for transport-layer statistics of remote video stream.

public void onRemoteVideoTransportStats(int uid, int delay, int lost, int rxKBitRate)
Deprecated
This method is deprecated. Use onRemoteVideoStats instead.

This callback reports transport-layer statistics such as packet loss rate and network delay. It is triggered every 2 seconds during the call when the local user receives video packets from a remote user or host.

Trigger Timing

This callback is triggered every 2 seconds during the call when the local user receives video packets from a remote user or host.

Parameters

uid
Remote user ID.
delay
Network delay from sender to receiver (ms).
lost
Packet loss rate (%) of video packets sent from the remote user.
rxKBitRate
Received video bitrate (Kbps).

onUserEnableLocalVideo

Callback triggered when a specified remote user enables or disables local video capture.

public void onUserEnableLocalVideo(int uid, boolean enabled)
Deprecated
This callback is deprecated. Use the onRemoteVideoStateChanged callback instead, with the following parameters:
  • REMOTE_VIDEO_STATE_STOPPED(0) and REMOTE_VIDEO_STATE_REASON_REMOTE_MUTED(5).
  • REMOTE_VIDEO_STATE_DECODING(2) and REMOTE_VIDEO_STATE_REASON_REMOTE_UNMUTED(6).

Trigger Timing

This callback is triggered when a remote user calls enableLocalVideo to enable or disable local video capture.

Parameters

uid
Remote user ID.
enabled
Whether the specified remote user enables local video capture:
  • true: Enables the video module. Other users in the channel can see the remote user's video.
  • false: Disables the video module. Other users in the channel cannot receive the remote user's video stream, but the remote user can still receive video streams from others.

onUserEnableVideo

Callback when a remote user enables or disables the video module.

public void onUserEnableVideo(int uid, boolean enabled)

This callback is triggered when a remote user calls enableVideo or disableVideo. After disabling the video module, the user can only make voice calls and cannot send or receive any video.

Trigger Timing

This callback is triggered when a remote user enables or disables the video module.

Parameters

uid
Remote user ID.
enabled
Whether the video module is enabled:
  • true: Video module is enabled.
  • false: Video module is disabled.

onUserMuteVideo

Callback when the remote video state changes.

public void onUserMuteVideo(int uid, boolean muted)

When a remote user calls muteLocalVideoStream, the SDK triggers this callback to report the remote user's video stream state to the local user.

Note: When the number of users in a communication scenario or hosts in an ultra-low latency live streaming scenario exceeds 32, this callback may be inaccurate.

Trigger Timing

This callback is triggered when a remote user calls muteLocalVideoStream to stop or resume publishing video streams.

Parameters

uid
Remote user ID.
muted
Whether the remote user stops publishing video streams:
  • true: The remote user stops publishing video streams.
  • false: The remote user resumes publishing video streams.

onVideoPublishStateChanged

Callback for video publish state change.

public void onVideoPublishStateChanged(Constants.VideoSourceType source, String channel, int oldState, int newState, int elapseSinceLastState)

Parameters

source
Type of video source. See VideoSourceType.
channel
Channel name.
oldState
Previous video publish state:
  • PUB_STATE_IDLE (0): Initial publish state after joining the channel.
  • PUB_STATE_NO_PUBLISHED (1): Failed to publish the local stream.
  • PUB_STATE_PUBLISHING (2): Publishing the local stream.
  • PUB_STATE_PUBLISHED (3): Successfully published the local stream.
newState
Current video publish state:
  • PUB_STATE_IDLE (0): Initial publish state after joining the channel.
  • PUB_STATE_NO_PUBLISHED (1): Failed to publish the local stream.
  • PUB_STATE_PUBLISHING (2): Publishing the local stream.
  • PUB_STATE_PUBLISHED (3): Successfully published the local stream.
elapseSinceLastState
Time elapsed (ms) from the previous state to the current state.

onVideoSizeChanged

Callback when the video size or rotation information of a specified user changes.

public void onVideoSizeChanged(Constants.VideoSourceType source, int uid, int width, int height, int rotation)

Trigger Timing

This callback is triggered when the video size or rotation angle of a specified user changes.

Parameters

source
Video source type. See VideoSourceType.
uid
User ID whose video size or rotation changed. For the local user, uid is 0, indicating the local video preview.
width
Width of the video stream (in pixels).
height
Height of the video stream (in pixels).
rotation
Video rotation angle, range [0, 360).

onVideoStopped

Callback triggered when the video stops playing.

public void onVideoStopped() {}
Deprecated
This method is deprecated. Use onLocalVideoStateChanged instead.

You can change the view configuration in this callback, such as displaying another image in the view.

Trigger Timing

This callback is triggered when the video stops playing.

Camera Capture

startCameraCapture

Starts video capture.

public abstract int startCameraCapture(Constants.VideoSourceType sourceType, CameraCapturerConfiguration config);

You can call this method and specify the sourceType to start capturing video from one or more cameras.

Note: On Android devices, if the device has multiple cameras or supports external cameras, you can capture video from up to 4 cameras.

Parameters

sourceType
The type of video source. See VideoSourceType.
Note: On Android devices, if the device has multiple cameras or supports external cameras, you can capture video from up to 4 cameras.
config
Video capture configuration. See CameraCapturerConfiguration.

Return Values

  • 0: Success.
  • < 0: Failure.

stopCameraCapture

Stops camera capture.

public abstract int stopCameraCapture(Constants.VideoSourceType sourceType);

You can call stopCameraCapture and set the sourceType parameter to stop video capture from the specified camera.

Note: If you are using local video composition, calling this method may interrupt the composition.

Parameters

sourceType
The type of video source. See VideoSourceType.

Return Values

  • 0: Success.
  • < 0: Failure.

Screen Capture

queryScreenCaptureCapability

Queries the maximum supported frame rate for screen sharing on the device.

public abstract int queryScreenCaptureCapability();
Since
Added since v4.2.0.

To ensure optimal screen sharing performance, especially when enabling high frame rates such as 60 fps, Agora recommends that you use this method to query the maximum supported frame rate before sharing. If the device does not support such high frame rates, you can adjust the screen sharing stream's frame rate accordingly to avoid affecting the sharing quality.

Return Values

  • 0: The device supports up to 15 fps.
  • 1: The device supports up to 30 fps.
  • 2: The device supports up to 60 fps.
  • < 0: Failure.

setExternalMediaProjection

Configures an external MediaProjection for the SDK to capture screen video streams.

public abstract int setExternalMediaProjection(MediaProjection mediaProjection);

After calling this method successfully, the provided external MediaProjection replaces the one requested by the SDK for capturing screen video streams. When screen sharing stops or RtcEngine is destroyed, the SDK automatically releases the MediaProjection.

Note: You must request [MediaProjection](https://developer.android.com/reference/android/media/projection/MediaProjection) permission before calling this method.

Scenario

If you have already obtained a MediaProjection, you can use your own MediaProjection directly without using the one requested by the SDK. Applicable scenarios include:
  • On customized system devices, avoid system pop-ups (such as user authorization for screen capture) and start capturing screen video streams directly.
  • In screen sharing processes involving one or more subprocesses, avoid potential errors when creating objects in subprocesses to prevent screen capture failure.
This method is applicable in multi-channel scenarios.

Timing

Call this method after startScreenCapture.

Parameters

mediaProjection
The object used to capture screen video streams.

Return Values

  • 0: Success.
  • < 0: Failure.

setScreenCaptureScenario

Sets the screen sharing scenario.

public abstract int setScreenCaptureScenario(Constants.ScreenScenarioType screenScenario);
Since
Available since v4.2.0.

The SDK automatically adjusts the quality and user experience of the shared video based on the selected scenario.

Note: Agora recommends calling this method before joining a channel.

Parameters

screenScenario
Screen sharing scenario. See ScreenScenarioType.

Return Values

  • 0: Success.
  • < 0: Failure.

startScreenCapture

Starts screen capture.

public abstract int startScreenCapture(ScreenCaptureParameters screenCaptureParameters);
Since
Added since v3.7.0.
You can call this method before or after joining a channel to start capturing the screen video stream as follows:
  • Call this method first, then call joinChannel and set publishScreenCaptureVideo to true to start screen sharing.
  • Call this method after joining the channel, then call updateChannelMediaOptions and set publishScreenCaptureVideo to true to start screen sharing.
Note:
  • On Android, if the user does not grant screen capture permission to the app, the SDK triggers the onPermissionError (error code 2) callback.
  • On Android 9 and above, to avoid the app being killed by the system when it goes to the background, Agora recommends adding the foreground service permission android.permission.FOREGROUND_SERVICE in the /app/Manifests/AndroidManifest.xml file.
  • Due to performance limitations, Android TV does not support screen sharing.
  • Due to system limitations, when using Huawei phones, do not change the video encoding resolution of the screen sharing stream during screen sharing, as it may cause a crash.
  • Due to system limitations, some Xiaomi devices do not support capturing system audio during screen sharing.
  • To avoid system audio capture failure during screen sharing, Agora recommends setting the audio scenario to AUDIO_SCENARIO_GAME_STREAMING using setAudioScenario before joining the channel.
  • The billing of the screen sharing stream is based on the dimensions in VideoCaptureParameters:
    • If the value is not provided, Agora charges based on 1280 × 720.
    • If the value is provided, Agora charges based on the provided value.

Scenario

Used in screen sharing scenarios.

Timing

You can call this method before or after joining a channel.

Parameters

screenCaptureParameters
Encoding parameters for screen sharing. See ScreenCaptureParameters.

Return Values

  • 0: Success.
  • < 0: Failure.
    • -2: System version too low. Make sure the Android API level is no lower than 21.
    • -3: Failed to capture system audio. Make sure the Android API level is no lower than 29.

stopScreenCapture

Stops screen capture.

public abstract int stopScreenCapture();
Since
Available since v3.7.0.

If you started screen capture by calling startScreenCapture, you need to call stopScreenCapture to stop it.

Timing

You can call this method before or after joining a channel.

Return Values

  • 0: Success.
  • < 0: Failure.

updateScreenCaptureParameters

Updates the screen capture parameters.

public abstract int updateScreenCaptureParameters(ScreenCaptureParameters screenCaptureParameters);
Since
Available since v3.7.0.
Note: Call this method after starting screen or window sharing.

Parameters

screenCaptureParameters
Encoding parameters for screen sharing. See ScreenCaptureParameters.
Note: The video properties of the screen sharing stream should only be set through this parameter and are not related to setVideoEncoderConfiguration.

Return Values

  • 0: Success.
  • < 0: Failure.
    • -2: Invalid parameter.
    • -8: Invalid screen sharing state. This may occur if you are already sharing another screen or window. Try calling stopScreenCapture to stop the current sharing and then restart screen sharing.

Video Pre-processing and Post-processing

Snapshot

enableContentInspect

Enables or disables the local snapshot upload feature.

public abstract int enableContentInspect(boolean enabled, ContentInspectConfig config);

The SDK captures and uploads snapshots of the local user's video based on the module type and frequency set in ContentInspectConfig. After completing the capture and upload, the Agora server sends a callback notification to your application server via HTTPS and uploads all snapshots to third-party cloud storage.

Note:
  • Before calling this method, make sure you have enabled the local snapshot upload feature in the Agora Console.
  • If the video moderation module uses the Agora proprietary extension (CONTENT_INSPECT_TYPE_SUPERVISE), you need to use the local snapshot upload dynamic library agora_content_inspect_extension.so. Removing this library disables the snapshot upload feature.
Timing

You can call this method before or after joining a channel.

Parameters
enabled
Whether to enable the local snapshot upload feature:
  • true: Enable the local snapshot upload feature.
  • false: Disable the local snapshot upload feature.
config
The snapshot upload configuration. See ContentInspectConfig.
Return Values
  • 0: Success.
  • < 0: Failure.

enableContentInspectEx

Enables or disables local snapshot upload.

public abstract int enableContentInspectEx(boolean enabled, ContentInspectConfig config, RtcConnection connection);

When enabled, the SDK captures and uploads snapshots of the local user's video based on the module type and frequency set in ContentInspectConfig. After uploading, the Agora server sends a callback notification to your app server via HTTPS and uploads all snapshots to a third-party cloud storage service.

Note: Before calling this method, make sure you have contacted technical support to enable the local snapshot upload service.
Scenario

This method is applicable in multi-channel scenarios.

Timing

You can call this method before or after joining a channel.

Parameters
enabled
Whether to enable local snapshot upload:
  • true: Enable local snapshot upload.
  • false: Disable local snapshot upload.
config
Configuration parameters for snapshot upload. See ContentInspectConfig.
connection
Connection information. See RtcConnection.
Return Values
  • 0: Success.
  • < 0: Failure.

takeSnapshot [1/2]

Captures a snapshot of the specified user's video stream and saves it as a JPG image.

public abstract int takeSnapshot(int uid, String filePath);
Note: This method is asynchronous. When the method returns, the SDK has not yet completed the snapshot operation. When used for a local video snapshot, this method captures the video stream specified in ChannelMediaOptions. If the video has been preprocessed (e.g., watermark or beautification), the snapshot will include the effects.
Timing

Call this method after joining a channel.

Related Callbacks

After calling this method successfully, the SDK triggers the onSnapshotTaken callback to report the snapshot result and details.

Parameters
uid
The user ID. To capture a snapshot of the local user's video, set uid to 0.
filePath
Note: Make sure the specified path exists and is writable.
The local path of the snapshot (including file extension). For example:
  • Android: /storage/emulated/0/Android/data/<package name>/files/example.jpg
Return Values
  • 0: Success.
  • < 0: Failure.

takeSnapshot [2/2]

Captures a video snapshot at the specified observation point.

public abstract int takeSnapshot(int uid, SnapshotConfig config);
Note:
  • This method is asynchronous. When the method returns, the SDK has not yet completed the snapshot operation.
  • When used for a local video snapshot, this method captures the video stream specified in ChannelMediaOptions.
Timing

Call this method after joining a channel.

Related Callbacks

After calling this method successfully, the SDK triggers the onSnapshotTaken callback to report the snapshot result and details.

Parameters
uid
The user ID. To capture a snapshot of the local user's video, set uid to 0.
config
The snapshot configuration. See SnapshotConfig.
Return Values
  • 0: Success.
  • < 0: Failure.

takeSnapshotEx [1/2]

Takes a snapshot of the video stream using the connection ID.

public abstract int takeSnapshotEx(RtcConnection connection, int uid, String filePath);
Note: takeSnapshotEx is an asynchronous call. The SDK does not complete the snapshot immediately. When used for local video, it captures the video stream specified in ChannelMediaOptions. If preprocessing (e.g., watermark, beautification) is applied, the snapshot includes those effects.
Scenario

Applicable in multi-channel scenarios.

Timing

Call this method after calling joinChannelEx.

Related Callbacks

After a successful call, the SDK triggers the onSnapshotTaken callback to report the result and details of the snapshot.

Parameters
connection
Connection information. See RtcConnection.
uid
User ID. To capture the local user's video, set uid to 0.
filePath
Note: Ensure the specified path exists and is writable.
Local path of the snapshot (including file extension). For example:
  • Android: /storage/emulated/0/Android/data/<package name>/files/example.jpg
Return Values
  • 0: Success.
  • < 0: Failure.

takeSnapshotEx [2/2]

Takes a snapshot of a specified user's video using the connection ID.

public abstract int takeSnapshotEx(RtcConnection connection, int uid, SnapshotConfig config);
Note:
  • This method is asynchronous. The SDK does not complete the snapshot immediately.
  • When used for local video, it captures the video stream specified in ChannelMediaOptions.
  • If preprocessing such as watermarking or beautification is applied, the snapshot includes those effects.
Scenario

Applicable in multi-channel scenarios.

Timing

Call this method after calling joinChannelEx.

Related Callbacks

After a successful call, the SDK triggers the onSnapshotTaken callback to report the result and details of the snapshot.

Parameters
connection
Connection information. See RtcConnection.
uid
User ID. To capture the local user's video, set uid to 0.
config
Snapshot configuration. See SnapshotConfig.
Return Values
  • 0: Success.
  • < 0: Failure.

onSnapshotTaken

Callback for video snapshot result.

public void onSnapshotTaken(int uid, String filePath, int width, int height, int errCode)

This callback is triggered after the takeSnapshot method is successfully called to report the details and success of the snapshot.

Trigger Timing

This callback is triggered after the takeSnapshot method is successfully called.

Parameters
uid
User ID. uid 0 indicates the local user.
filePath
Local path of the snapshot.
width
Width of the snapshot (in pixels).
height
Height of the snapshot (in pixels).
errCode
Error code of the snapshot result:
  • 0: Success.
  • < 0: Failure:
    • -1: SDK failed to write file or encode JPEG image.
    • -2: The specified user's video stream was not found within 1 second after calling takeSnapshot. Possible reasons include: local capture stopped, remote stopped publishing, or video data processing blocked.
    • -3: takeSnapshot method is called too frequently.

Virtual Background

enableVirtualBackground [1/2]

Enables or disables the virtual background feature.

public abstract int enableVirtualBackground(boolean enabled, VirtualBackgroundSource backgroundSource, SegmentationProperty segproperty);

The virtual background feature allows the local user to replace the original background with a static image, dynamic video, blurred background, or portrait segmentation to achieve picture-in-picture effects. Once enabled, all users in the channel can see the customized background. Call this method after enableVideo or startPreview.

Note:
  • Using a video as a virtual background may cause continuous memory growth, potentially leading to app crashes. It is recommended to reduce the resolution and frame rate of the video.
  • This feature requires high device performance. When this method is called, the SDK automatically detects the device's performance. Agora recommends using the following processors:
    • Snapdragon 700 series 750G or above.
    • Snapdragon 800 series 835 or above.
    • Dimensity 700 series 720 or above.
    • Kirin 800 series 810 or above.
    • Kirin 900 series 980 or above.
  • Agora recommends using this feature in the following scenarios:
    • When using high-definition camera devices with even lighting.
    • When there are few objects in the frame, and the portrait is a half-body shot with no obstructions.
    • When the background is a solid color and clearly different from the user's clothing.
  • This method depends on the virtual background dynamic library libagora_segmentation_extension.so. If the library is deleted, the feature cannot be enabled.
Parameters
enabled
Whether to enable the virtual background:
  • true: Enable virtual background.
  • false: Disable virtual background.
backgroundSource
The custom background. To adapt the resolution of the custom background image to the video resolution captured by the SDK, the SDK scales and crops the image to ensure no distortion. See VirtualBackgroundSource.
segproperty
The processing properties of the background image. See SegmentationProperty.
Return Values
  • 0: Success.
  • < 0: Failure.
    • -4: The device does not meet the performance requirements for the virtual background feature. Agora recommends using a higher-performance device.

enableVirtualBackground [2/2]

Enables virtual background and specifies the media source, or disables the virtual background.

public abstract int enableVirtualBackground(boolean enabled, VirtualBackgroundSource backgroundSource, SegmentationProperty segproperty, Constants.MediaSourceType sourceType);

This method enables or disables the virtual background feature. You can replace the original background with a static image, dynamic video, blurred background, or portrait segmentation to achieve a picture-in-picture effect. Once enabled, all users in the channel can see the customized background. This method provides the same functionality as enableVirtualBackground [1/2], but allows you to specify the media source type for applying the virtual background. Call this method after calling enableVideo or startPreview.

Note:
  • Using video as a virtual background may cause continuous memory growth, potentially leading to application crashes. It is recommended to reduce the resolution and frame rate of the video.
  • This feature requires high device performance. When calling this method, the SDK automatically checks the device's capability. Agora recommends using this feature on the following processors:
    • Snapdragon 700 series 750G and above.
    • Snapdragon 800 series 835 and above.
    • Dimensity 700 series 720 and above.
    • Kirin 800 series 810 and above.
    • Kirin 900 series 980 and above.
  • Agora recommends using this feature in the following scenarios:
    • Using a high-definition camera device with even lighting.
    • Few objects in the frame, with a half-body portrait unobstructed.
    • A solid color background that contrasts with the user’s clothing.
  • This method depends on the virtual background dynamic library libagora_segmentation_extension.so. If the library is deleted, the feature cannot be enabled properly.
Parameters
enabled
Whether to enable the virtual background:
  • true: Enable the virtual background.
  • false: Disable the virtual background.
backgroundSource
The custom background. The SDK scales and crops the custom background image to fit the captured video resolution and ensures the image content is not distorted. See VirtualBackgroundSource.
segproperty
The processing properties of the background image. See SegmentationProperty.
sourceType
The media source type to apply the filter effect. See MediaSourceType.
Note: In this method, the parameter only supports the following two values:
  • Default is PRIMARY_CAMERA_SOURCE.
  • If using a custom video source, set this parameter to CUSTOM_VIDEO_SOURCE.
Return Values
  • 0: Success.
  • < 0: Failure.
    • -4: The device does not meet the requirements for the virtual background feature. Agora recommends using this feature on higher-performance devices.

Raw Video Data

getMirrorApplied

Callback triggered each time the SDK receives a video frame, used to determine whether to mirror the captured video.

@CalledByNative boolean getMirrorApplied();

If you want the obtained video data to be a mirrored image of the original video, you need to register this callback when calling registerVideoFrameObserver. After successfully registering the video frame observer, the SDK triggers this callback each time it receives a video frame. You must set whether to mirror the video frame in the return value of this callback.

Note:
  • On Android, this callback supports the following video data formats: I420, RGBA, and Texture.
  • This method and setVideoEncoderConfiguration both support setting mirror effects. It is recommended to use only one method. Using both may result in overlapping mirror effects, causing the setting to fail.

Timing

This callback is triggered each time the SDK receives a video frame.

Return Values

Whether to mirror the captured video:
  • true: Mirror the captured video.
  • false: (default) Do not mirror the captured video.

getObservedFramePosition

Sets the frame position for the video observer.

@CalledByNative int getObservedFramePosition();
After successfully registering the video data observer, the SDK uses this callback to determine whether to trigger the onCaptureVideoFrame, onRenderVideoFrame, and onPreEncodeVideoFrame callbacks at specific video frame processing positions. You can set one or more positions to observe by modifying the return value based on your business scenario:
  • VIDEO_MODULE_POSITION_POST_CAPTURER (1 << 0): After video capture, corresponds to onCaptureVideoFrame callback.
  • VIDEO_MODULE_POSITION_PRE_RENDERER (1 << 1): Before rendering remote video, corresponds to onRenderVideoFrame callback.
  • VIDEO_MODULE_POSITION_PRE_ENCODER (1 << 2): Before encoding, corresponds to onPreEncodeVideoFrame callback.

Return Values

If the method call succeeds, returns the bitmask used to control the frame position for the video observer:
  • VIDEO_MODULE_POSITION_POST_CAPTURER (1 << 0): After video capture.
  • VIDEO_MODULE_POSITION_PRE_RENDERER (1 << 1): Before rendering remote video.
  • VIDEO_MODULE_POSITION_PRE_ENCODER (1 << 2): Before encoding.

getRotationApplied

getRotationApplied callback is triggered each time the SDK receives a video frame, used to determine whether to rotate the captured video.

@CalledByNative boolean getRotationApplied();

If you want to rotate the captured video based on the rotation member in VideoFrame, you need to register this callback when calling registerVideoFrameObserver. After successful registration, the SDK triggers this callback each time it receives a video frame. You must set whether to rotate the video frame in the return value of this callback.

Note: On Android, this callback supports the following video data formats: I420, RGBA, and Texture.

Timing

This callback is triggered each time the SDK receives a video frame.

Return Values

  • true: Rotate the captured video.
  • false: (default) Do not rotate the captured video.

getVideoFormatPreference

Sets the format of the raw video data output by the SDK.

@CalledByNative int getVideoFormatPreference();

After calling registerVideoFrameObserver to register the callback, the SDK triggers this callback each time it receives a video frame. You need to set the expected video data format in the return value of this callback.

Note: VIDEO_PIXEL_DEFAULT indicates the default pixel format of the raw video, as follows:
  • On Android, the default video frame type may be I420Buffer or TextureBuffer. The texture format of TextureBuffer may be OES or RGB. If the video frame type returned by getVideoFormatPreference is VIDEO_PIXEL_DEFAULT, you must handle both I420Buffer and TextureBuffer.
Cases where the video frame type is fixed to I420Buffer include but are not limited to:
  • Specific devices, such as: LG G5 SE (H848), Google Pixel 4a, Samsung Galaxy A7, or Xiaomi Max.
  • When beauty effect extension is integrated and video denoising or low light enhancement is enabled.

Return Values

If the method call succeeds, returns the raw data format output by the SDK:
  • VIDEO_PIXEL_DEFAULT (0): Original video format, must support I420Buffer or TextureBuffer.
  • VIDEO_PIXEL_I420 (1): I420 format.
  • VIDEO_PIXEL_RGBA (4): RGBA format.
  • VIDEO_PIXEL_I422 (16): I422 format.

getVideoFrameProcessMode

getVideoFrameProcessMode callback. Triggered each time the SDK receives a video frame, prompting you to set the processing mode.

@CalledByNative int getVideoFrameProcessMode();

After successfully registering a video frame observer, the SDK triggers the getVideoFrameProcessMode callback each time it receives a video frame. You need to set the preferred processing mode in the return value of this callback.

Return Values

Video frame processing mode:
  • PROCESS_MODE_READ_ONLY (0): (default) Read-only.
  • PROCESS_MODE_READ_WRITE (1): Read and write.

registerVideoFrameObserver

Registers the raw video frame observer object.

public abstract int registerVideoFrameObserver(IVideoFrameObserver observer);

It is recommended to register an IVideoFrameObserver instance using this method and implement the callbacks as needed. After successful registration, the SDK triggers the corresponding callbacks whenever a video frame is received. You can use this raw video data in video preprocessing scenarios such as virtual background and beauty effects.

Note: When processing the video data returned in the callbacks, note that the width and height parameters may change under the following conditions:
  • When network conditions deteriorate, the video resolution will gradually decrease.
  • When the user adjusts the video configuration, the resolution in the callback will also change accordingly.

Timing

Call this method before joining a channel.

Parameters

observer
Observer instance. See IVideoFrameObserver.
Note: To release this instance, set this parameter to null.

Return Values

  • 0: Success.
  • < 0: Failure.

onCaptureVideoFrame

Callback triggered when a video frame is captured by the local device.

@CalledByNative boolean onCaptureVideoFrame(int sourceType, VideoFrame videoFrame);

You can use this callback to obtain the raw video data captured by the local device and preprocess it as needed. After preprocessing, you can directly modify the videoFrame in this callback and return true to send the modified video data to the SDK. If you want to send the preprocessed video data to the SDK, you must first call getVideoFrameProcessMode to set the video processing mode to read-write (PROCESS_MODE_READ_WRITE).

Note:
  • If the obtained video data type is RGBA, the SDK does not support processing alpha channel data.
  • It is recommended to ensure that the modified parameters in videoFrame match the actual video frame in the buffer. Otherwise, issues such as abnormal rotation or stretching may occur in local preview and remote video display.
  • The default video format obtained through this callback may be I420Buffer or TextureBuffer. The texture format of TextureBuffer may be OES or RGB. If you need other video formats, you can set the desired data format in the return value of the getVideoFormatPreference callback.

Trigger Timing

This callback is triggered each time the SDK captures a video frame after successfully registering the video data observer.

Parameters

sourceType
Video source type, including camera, screen, or media player. See VideoSourceType.
videoFrame
Video frame.
Note: The default format of the video frame data obtained through this callback is as follows:
  • Android: I420 or RGB (GLES20.GL_TEXTURE_2D)

Return Values

  • When the video processing mode is PROCESS_MODE_READ_ONLY:
    • true: Retain for subsequent use.
    • false: Retain for subsequent use.
  • When the video processing mode is PROCESS_MODE_READ_WRITE:
    • true: The SDK accepts the video frame.
    • false: The SDK discards the video frame.

onPreEncodeVideoFrame

Callback triggered each time the SDK receives a video frame before encoding.

@CalledByNative boolean onPreEncodeVideoFrame(int sourceType, VideoFrame videoFrame);

After you successfully register a video frame observer, the SDK triggers this callback each time it receives a video frame. In this callback, you can obtain the video data before encoding and process it according to your specific scenario. After processing, you can return the processed video data to the SDK.

Note:
  • If you want to send the preprocessed video data back to the SDK, you must first call getVideoFrameProcessMode to set the video processing mode to read-write (PROCESS_MODE_READ_WRITE).
  • To obtain pre-encoded video data from the second screen capture, you need to set the frame position to VIDEO_MODULE_POSITION_PRE_ENCODER (1 << 2) via getObservedFramePosition.
  • The video data obtained from this callback has already been preprocessed, including cropping, rotation, and beauty effects.
  • It is recommended to ensure that the modified parameters in videoFrame match the actual video frame in the buffer. Otherwise, issues such as abnormal rotation or image stretching may occur in local preview and remote video display.

Trigger Timing

This callback is triggered each time the SDK receives a video frame before encoding.

Parameters

sourceType
Video source type. See VideoSourceType.
videoFrame
Video frame.
Note: The default format of the video frame data obtained through this callback is as follows:
  • Android: I420 or RGB (GLES20.GL_TEXTURE_2D).

Return Values

  • When the video processing mode is PROCESS_MODE_READ_ONLY:
    • true: Retain for subsequent use.
    • false: Retain for subsequent use.
  • When the video processing mode is PROCESS_MODE_READ_WRITE:
    • true: The SDK accepts the video frame.
    • false: The SDK discards the video frame.

onRenderVideoFrame

Callback triggered when a remote user sends a video frame.

boolean onRenderVideoFrame(String channelId, int uid, VideoFrame videoFrame)

After you successfully register a video frame observer, the SDK triggers this callback each time it receives a video frame sent by a remote user. You can obtain the video data sent by the remote user before rendering and process it according to your specific scenario. The default format of the video frame obtained via this callback may be I420Buffer or TextureBuffer, and the texture format of TextureBuffer may be OES or RGB. If you need other video formats, you can set the desired data format in the getVideoFormatPreference callback.

Note:
  • If you want to send the preprocessed video data back to the SDK, you must first call getVideoFrameProcessMode to set the video processing mode to read-write (PROCESS_MODE_READ_WRITE).
  • If the obtained video data type is RGBA, the SDK does not support processing alpha channel data.
  • It is recommended to ensure that the modified parameters in videoFrame match the actual video frame in the buffer. Otherwise, issues such as abnormal rotation or image stretching may occur in local preview and remote video display.

Trigger Timing

This callback is triggered when the SDK receives a video frame sent by a remote user.

Parameters

channelId
Channel ID.
uid
Remote user ID who sent the current video frame.
videoFrame
Video frame.

Return Values

  • When the video processing mode is PROCESS_MODE_READ_ONLY:
    • true: Retain for subsequent use.
    • false: Retain for subsequent use.
  • When the video processing mode is PROCESS_MODE_READ_WRITE:
    • true: The SDK accepts the video frame.
    • false: The SDK discards the video frame.

Encoded Video Data

registerVideoEncodedFrameObserver

Registers an observer object to receive encoded video frames.

public abstract int registerVideoEncodedFrameObserver(IVideoEncodedFrameObserver receiver);

It is recommended to register an IVideoEncodedFrameObserver instance using this method to observe encoded video frames (e.g., H.264 format) without decoding and rendering.

Note: Call this method before joining a channel.

Parameters

receiver
The observer object for encoded video frames. See IVideoEncodedFrameObserver.

Return Values

  • 0: Success.
  • < 0: Failure.

onEncodedVideoFrameReceived

Callback when the receiver receives a remote encoded video frame.

boolean onEncodedVideoFrameReceived(String channelId, int remoteUid, ByteBuffer buffer, EncodedVideoFrameInfo info)
Since
Available since v4.6.0.

If you call the setRemoteVideoSubscriptionOptions method and set encodedFrameOnly to true, the SDK triggers this callback locally to report the received encoded video frame information.

Parameters

channelId
Channel name.
remoteUid
Remote user ID.
buffer
Encoded video image buffer.
info
Information of the encoded video frame. See EncodedVideoFrameInfo.

Return Values

  • true: Callback processed successfully.
  • false: Callback processing failed.

Custom Video Capture and Rendering

createCustomVideoTrack

Creates a custom video track.

public abstract int createCustomVideoTrack();
Call this method to create a video track and obtain the video track ID. This ID can be used to publish a custom video source. To publish a custom video source, follow these steps:
  1. Call this method to create a video track and obtain the video track ID.
  2. Call joinChannel to join a channel. In ChannelMediaOptions, set customVideoTrackId to the video track ID to be published and set publishCustomVideoTrack to true.
  3. Call pushExternalVideoFrameById and set videoTrackId to the video track ID specified in step 2 to publish the corresponding custom video source in the channel.

Return Values

  • If the method call succeeds, returns the unique identifier of the video track, i.e., the video track ID.
  • If the method call fails, returns 0xffffffff.

destroyCustomVideoTrack

Destroys the specified video track.

public abstract int destroyCustomVideoTrack(int video_track_id);

Parameters

video_track_id
The video track ID returned by createCustomVideoTrack.

Return Values

  • 0: Success.
  • < 0: Failure.

isTextureEncodeSupported

Checks whether Texture encoding is supported for video.

public abstract boolean isTextureEncodeSupported();

Return Values

  • true: Texture encoding is supported.
  • false: Texture encoding is not supported.

pushExternalVideoFrameById [1/2]

Pushes external raw video frame to the SDK via video track.

public abstract int pushExternalVideoFrameById(VideoFrame frame, int videoTrackId);
To publish a custom video source, follow these steps:
  1. Call createCustomVideoTrack to create a video track and get the video track ID.
  2. Call joinChannel to join the channel. In ChannelMediaOptions, set customVideoTrackId to the video track ID and set publishCustomVideoTrack to true.
  3. Call pushExternalVideoFrameById and set videoTrackId to the ID from step 2 to publish the corresponding custom video source in the channel.
You can push video frames by calling pushExternalVideoFrameById or pushExternalVideoFrameById (with AgoraVideoFrame parameter). The difference is that this method supports video data in Texture format.
Note: If you only need to push one custom video source to the channel, you can directly call setExternalVideoSource, and the SDK will automatically create a video track with videoTrackId 0. Danger: After calling this method, even if you stop pushing external video frames to the SDK, the custom video stream will still be counted toward video duration and incur charges. Agora recommends taking appropriate actions based on your use case to avoid such charges:
  • If you no longer need to capture external video, call destroyCustomVideoTrack to destroy the custom video track.
  • If you only want to use external video for local preview and not publish it to the channel, call muteLocalVideoStream to stop sending the video stream, or call updateChannelMediaOptions and set publishCustomVideoTrack to false.

Parameters

frame
The video frame to be pushed.
videoTrackId
The video track ID returned by createCustomVideoTrack.
Note: If you only need to push one custom video source, set videoTrackId to 0.

Return Values

  • 0: Success.
  • < 0: Failure.

pushExternalVideoFrame [1/2]

Pushes external raw video frame to the SDK.

public abstract boolean pushExternalVideoFrame(VideoFrame frame)
Deprecated
This method is deprecated.

Call setExternalVideoSource and set enabled to true, and encodedFrame to false to use this method to push external raw video frames to the SDK. You can call this method or pushExternalVideoFrame to push video frames. The difference is that this method supports video data in Texture format.

Parameters

frame
The video frame to be pushed.

Return Values

  • true: Success.
  • false: Failure.

pushExternalVideoFrameById [2/2]

Pushes external raw video frame to the SDK via video track.

public abstract int pushExternalVideoFrameById(AgoraVideoFrame frame, int videoTrackId);
To publish a custom video source, follow these steps:
  1. Call createCustomVideoTrack to create a video track and get the video track ID.
  2. Call joinChannel to join the channel. In ChannelMediaOptions, set customVideoTrackId to the video track ID and set publishCustomVideoTrack to true.
  3. Call pushExternalVideoFrameById and set videoTrackId to the ID from step 2 to publish the corresponding custom video source in the channel.
You can push video frames by calling pushExternalVideoFrameById or pushExternalVideoFrame. The difference is that pushExternalVideoFrameById does not support Texture format video data.
Note: If you only need to push one custom video source to the channel, you can directly call setExternalVideoSource, and the SDK will automatically create a video track with videoTrackId 0. Warning: After calling pushExternalVideoFrameById, even if you stop pushing external video frames to the SDK, the custom video stream will still be counted toward video duration and incur charges. Agora recommends taking appropriate actions based on your use case to avoid such charges:
  • If you no longer need to capture external video, call destroyCustomVideoTrack to destroy the custom video track.
  • If you only want to use external video for local preview and not publish it to the channel, call muteLocalVideoStream to stop sending the video stream, or call updateChannelMediaOptions and set publishCustomVideoTrack to false.

Parameters

frame
The external raw video frame to be pushed. See AgoraVideoFrame.
videoTrackId
The video track ID returned by createCustomVideoTrack.
Note: If you only need to push one custom video source, set videoTrackId to 0.

Return Values

  • 0: Success.
  • < 0: Failure.

pushExternalVideoFrame [2/2]

Pushes external raw video frame to the SDK.

public abstract boolean pushExternalVideoFrame(AgoraVideoFrame frame)
Deprecated
Deprecated since v4.5.0.

Call setExternalVideoSource and set enabled to true and encodedFrame to false to use pushExternalVideoFrame to push external raw video frames to the SDK. You can also call pushExternalVideoFrame to push video frames. The difference is that this method does not support Texture format video data.

Parameters

frame
The external raw video frame to be pushed. See AgoraVideoFrame.

Return Values

  • true: Success.
  • false: Failure.

setExternalRemoteEglContext

Sets the EGL context used to render remote video streams.

public abstract int setExternalRemoteEglContext(Object eglContext);

This method replaces the SDK's default remote EGL context, allowing you to manage the EGL context yourself. The SDK automatically releases the context when the engine is destroyed.

Scenario

This method is applicable when using custom rendering (e.g., Texture format) to replace the SDK's default rendering of remote video frames.

Timing

Call this method before joining a channel.

Parameters

eglContext
The EGL context used to render remote video streams.

Return Values

  • 0: Success.
  • < 0: Failure.

setExternalVideoSource

Configures external video source.

public abstract int setExternalVideoSource(boolean enable, boolean useTexture, Constants.ExternalVideoSourceType sourceType);

After calling this method, you can call pushExternalVideoFrameById to push external video data to the SDK.

Note: Dynamic switching of video sources is not supported in the channel. To switch from external to internal video source, you must leave the channel, disable the external video source with this method, and rejoin the channel.

Timing

Call this method before joining a channel.

Parameters

enable
Whether to use an external video source:
  • true: Use external video source. SDK prepares to receive external video frames.
  • false: (Default) Do not use external video source.
useTexture
Whether to use Texture format for external video frames:
  • true: Use Texture format.
  • false: (Default) Do not use Texture format.
sourceType
Whether the external video frame is encoded. See ExternalVideoSourceType.

Return Values

  • 0: Success.
  • < 0: Failure.