Public Member Functions
abstract boolean	onRecordFrame (AudioFrame audioFrame)

abstract boolean	onPlaybackFrame (AudioFrame audioFrame)

abstract boolean	onPlaybackFrameBeforeMixing (AudioFrame audioFrame, int uid)

abstract boolean	onMixedFrame (AudioFrame audioFrame)

abstract boolean	isMultipleChannelFrameWanted ()

abstract boolean	onPlaybackFrameBeforeMixingEx (AudioFrame audioFrame, int uid, String channelId)

abstract int	getObservedAudioFramePosition ()

abstract AudioParams	getRecordAudioParams ()

abstract AudioParams	getPlaybackAudioParams ()

abstract AudioParams	getMixedAudioParams ()

Static Public Attributes
static int	POSITION_PLAYBACK = 0x01

static int	POSITION_RECORD = 0x01 << 1

static int	POSITION_MIXED = 0x01 << 2

static int	POSITION_BEFORE_MIXING = 0x01 << 3

Detailed Description

Callbacks of the raw audio data (IAudioFrameObserver).

Member Function Documentation

◆ onRecordFrame()

abstract boolean io.agora.rtc.IAudioFrameObserver.onRecordFrame ( AudioFrame audioFrame )

abstract

Occurs when the recorded audio frame of the local user is received.

Since: v3.4.5

After you successfully register the audio observer, the SDK triggers the onRecordFrame callback every 10 ms by default, and the reported audio frame format is the same as the recorded audio frame format.

Note: To ensure that the received audio frame has the expected format, you can register the getRecordAudioParams callback when calling the registerAudioFrameObserver method and set the audio recording format in the return value of getRecordAudioParams. The SDK calculates the sample interval according to the AudioParams you set in the return value of getRecordAudioParams and triggers the onRecordFrame callback at the sample interval.

Parameters

audioFrame The raw audio data. See AudioFrame.

Returns

true: Reserved for future use.
false: Reserved for future use.

◆ onPlaybackFrame()

abstract boolean io.agora.rtc.IAudioFrameObserver.onPlaybackFrame ( AudioFrame audioFrame )

abstract

Occurs when the playback audio frame of all remote users is received.

Since: v3.4.5

After you successfully register the audio observer, the SDK triggers the onPlaybackFrame callback every 10 ms by default, and the reported audio frame format is the same as the playback audio frame format.

Note: To ensure that the received audio frame has the expected format, you can register the getPlaybackAudioParams callback when calling the registerAudioFrameObserver method and set the audio playback format in the return value of getPlaybackAudioParams. The SDK calculates the sample interval according to the AudioParams you set in the return value of getPlaybackAudioParams and triggers the onPlaybackFrame callback at the sample interval.

Parameters

audioFrame The raw audio data. See AudioFrame.

Returns

true: Reserved for future use.
false: Reserved for future use.

◆ onPlaybackFrameBeforeMixing()

abstract boolean io.agora.rtc.IAudioFrameObserver.onPlaybackFrameBeforeMixing	(	AudioFrame	audioFrame,
		int	uid
	)

abstract

Occurs when the audio frame of a remote user is received before mixing.

Since: v3.4.5

To trigger this callback, you need to register the following callbacks and set their return values when calling the registerAudioFrameObserver method:

getObservedAudioFramePosition: Set its return value as POSITION_BEFORE_MIXING(0x01 << 3).
isMultipleChannelFrameWanted: Set its return value as false.

After the setting is successful, the SDK triggers the onPlaybackFrameBeforeMixing callback every 10 ms by default.

Parameters

audioFrame	The raw audio data. See `AudioFrame`.
uid	The user ID of the remote user.

Returns

true: Reserved for future use.
false: Reserved for future use.

◆ onMixedFrame()

abstract boolean io.agora.rtc.IAudioFrameObserver.onMixedFrame ( AudioFrame audioFrame )

abstract

Occurs when the mixed audio frame of the local user and all remote users is received.

Since: v3.4.5

To trigger this callback, you need to register the getObservedAudioFramePosition callback when calling the registerAudioFrameObserver method and set the return value of getObservedAudioFramePosition as POSITION_MIXED(0x01 << 2). After the setting is successful, the SDK triggers the onMixedFrame callback every 10 ms by default, and the reported audio frame format is the same as the mixed audio frame format.

Note: To ensure that the received audio frame has the expected format, you can register the getMixedAudioParams callback when calling the registerAudioFrameObserver method and set the audio mixing format in the return value of getMixedAudioParams. The SDK calculates the sample interval according to the AudioParams you set in the return value of getMixedAudioParams and triggers the onMixedFrame callback at the sample interval.

Parameters

audioFrame The raw audio data. See AudioFrame.

Returns

true: Reserved for future use.
false: Reserved for future use.

◆ isMultipleChannelFrameWanted()

abstract boolean io.agora.rtc.IAudioFrameObserver.isMultipleChannelFrameWanted ( )

abstract

Determines whether to receive raw audio data from multiple channels.

Since: 3.3.0

After you register the audio frame observer, the SDK triggers this callback every time it captures an audio frame.

In a multi-channel scenario, if you want to get audio data from multiple channels, set the return value of this callback as true, and set the return value of getObservedAudioFramePosition as POSITION_BEFORE_MIXING (0x01 << 3). After that, the SDK triggers the onPlaybackFrameBeforeMixingEx callback to send you the before-mixing audio data from various channels. You can also get the channel ID of each audio frame.

Note

Once you set the return value of this callback as true, the SDK triggers only the onPlaybackFrameBeforeMixingEx callback to send the before-mixing audio frame. onPlaybackFrameBeforeMixing is not triggered. In a multi-channel scenario, Agora recommends setting the return value as true.
If you set the return value of this callback as false, the SDK triggers only the onPlaybackFrameBeforeMixing callback to send the audio data.

Returns

true: Receive audio data from multiple channels.
false: Do not receive audio data from multiple channels.

◆ onPlaybackFrameBeforeMixingEx()

abstract boolean io.agora.rtc.IAudioFrameObserver.onPlaybackFrameBeforeMixingEx	(	AudioFrame	audioFrame,
		int	uid,
		String	channelId
	)

abstract

Gets the audio frame of a remote user before mixing from one of the multiple channels.

Since: v3.4.5

To trigger this callback, you need to register the following callbacks and set their return values when calling the registerAudioFrameObserver method:

getObservedAudioFramePosition: Set its return value as POSITION_BEFORE_MIXING(0x01 << 3).
isMultipleChannelFrameWanted: Set its return value as true.

After the setting is successful, the SDK triggers the onPlaybackFrameBeforeMixingEx callback every 10 ms by default.

Parameters

audioFrame	The raw audio data. See `AudioFrame`.
uid	The user ID of the remote user.
channelId	The channel name.

Returns

true: Reserved for future use.
false: Reserved for future use.

◆ getObservedAudioFramePosition()

abstract int io.agora.rtc.IAudioFrameObserver.getObservedAudioFramePosition ( )

abstract

Sets the audio observation positions.

Since: v3.4.5

After you successfully register the audio observer, the SDK uses the getObservedAudioFramePosition callback to determine at each specific audio-frame processing node whether to trigger the following callbacks:

onRecordFrame
onPlaybackFrame
onPlaybackFrameBeforeMixing or onPlaybackFrameBeforeMixingEx
onMixedFrame

You can set the positions that you want to observe by modifying the return value of getObservedAudioFramePosition according to your scenario.

Note

To observe multiple positions, use | (the OR operator).
The default return value of getObservedAudioFramePosition is POSITION_PLAYBACK (0x01) and POSITION_RECORD (0x01 << 1).
To conserve system resources, you can reduce the number of frame positions that you want to observe.

Returns

The bit mask that controls the audio observation positions:

POSITION_PLAYBACK (0x01): The position for observing the playback audio of all remote users after mixing, which enables the SDK to trigger the onPlaybackFrame callback.
POSITION_RECORD (0x01 << 1): The position for observing the recorded audio of the local user, which enables the SDK to trigger the onRecordFrame callback.
POSITION_MIXED (0x01 << 2): The position for observing the mixed audio of the local user and all remote users, which enables the SDK to trigger the onMixedFrame callback.
POSITION_BEFORE_MIXING (0x01 << 3): The position for observing the audio of a single remote user before mixing, which enables the SDK to trigger the onPlaybackFrameBeforeMixing or onPlaybackFrameBeforeMixingEx callback.

◆ getRecordAudioParams()

abstract AudioParams io.agora.rtc.IAudioFrameObserver.getRecordAudioParams ( )

abstract

Sets the audio recording format for the onRecordFrame callback.

Since: v3.4.5

Register the getRecordAudioParams callback when calling the registerAudioFrameObserver method. After you successfully register the audio observer, the SDK triggers this callback, and you can set the audio recording format in the return value of this callback.

Note: The SDK calculates the sample interval according to the AudioParams you set in the return value of this callback and triggers the onRecordFrame callback at the calculated sample interval.

Sample interval (seconds) = samplePerCall/(sampleRate × channelCnt). Ensure that the value of sample interval is equal to or greater than 0.01.

Returns: Sets the audio format. See AudioParams.

◆ getPlaybackAudioParams()

abstract AudioParams io.agora.rtc.IAudioFrameObserver.getPlaybackAudioParams ( )

abstract

Sets the audio playback format for the onPlaybackFrame callback.

Since: v3.4.5

Register the getPlaybackAudioParams callback when calling the registerAudioFrameObserver method. After you successfully register the audio observer, the SDK triggers this callback, and you can set the audio playback format in the return value of this callback.

Note: The SDK calculates the sample interval according to the AudioParams you set in the return value of this callback and triggers the onPlaybackFrame callback at the calculated sample interval.

Sample interval (seconds) = samplePerCall/(sampleRate × channelCnt). Ensure that the value of sample interval is equal to or greater than 0.01.

Returns: Sets the audio format. See AudioParams.

◆ getMixedAudioParams()

abstract AudioParams io.agora.rtc.IAudioFrameObserver.getMixedAudioParams ( )

abstract

Sets the audio mixing format for the onMixedFrame callback.

Since: v3.4.5

Register the getMixedAudioParams callback when calling the registerAudioFrameObserver method. After you successfully register the audio observer, the SDK triggers this callback, and you can set the audio mixing format in the return value of this callback.

Note: The SDK calculates the sample interval according to the AudioParams you set in the return value of this callback and triggers the onMixedFrame callback at the calculated sample interval.

Sample interval (seconds) = samplePerCall/(sampleRate × channelCnt). Ensure that the value of sample interval is equal to or greater than 0.01.

Returns: Sets the audio format. See AudioParams.

Member Data Documentation

◆ POSITION_PLAYBACK

int io.agora.rtc.IAudioFrameObserver.POSITION_PLAYBACK = 0x01

static

0x01: The position for observing the playback audio of all remote users after mixing, which enables the SDK to trigger the onPlaybackFrame callback.

◆ POSITION_RECORD

int io.agora.rtc.IAudioFrameObserver.POSITION_RECORD = 0x01 << 1

static

0x01 << 1: The position for observing the recorded audio of the local user, which enables the SDK to trigger the onRecordFrame callback.

◆ POSITION_MIXED

int io.agora.rtc.IAudioFrameObserver.POSITION_MIXED = 0x01 << 2

static

0x01 << 2: The position for observing the mixed audio of the local user and all remote users, which enables the SDK to trigger the onMixedFrame callback.

◆ POSITION_BEFORE_MIXING

int io.agora.rtc.IAudioFrameObserver.POSITION_BEFORE_MIXING = 0x01 << 3

static

0x01 << 3: The position for observing the audio of a single remote user before mixing, which enables the SDK to trigger the onPlaybackFrameBeforeMixing or onPlaybackFrameBeforeMixingEx callback.

Public Member Functions

Static Public Attributes

Detailed Description

Member Function Documentation

◆ onRecordFrame()

◆ onPlaybackFrame()

◆ onPlaybackFrameBeforeMixing()

◆ onMixedFrame()

◆ isMultipleChannelFrameWanted()

◆ onPlaybackFrameBeforeMixingEx()

◆ getObservedAudioFramePosition()

◆ getRecordAudioParams()

◆ getPlaybackAudioParams()

◆ getMixedAudioParams()

Member Data Documentation

◆ POSITION_PLAYBACK

◆ POSITION_RECORD

◆ POSITION_MIXED

◆ POSITION_BEFORE_MIXING