agora::media::IMediaEngine Class Reference

Public Member Functions
virtual	~IMediaEngine ()
virtual void	release ()=0
virtual int	registerAudioFrameObserver (IAudioFrameObserver *observer)=0
virtual int	registerVideoFrameObserver (IVideoFrameObserver *observer)=0
virtual int	registerVideoRenderFactory (IExternalVideoRenderFactory *factory) AGORA_DEPRECATED_ATTRIBUTE=0
virtual int	pushAudioFrame (MEDIA_SOURCE_TYPE type, IAudioFrameObserver::AudioFrame *frame, bool wrap) AGORA_DEPRECATED_ATTRIBUTE=0
virtual int	pushAudioFrame (IAudioFrameObserver::AudioFrame *frame) AGORA_DEPRECATED_ATTRIBUTE=0
virtual int	pushAudioFrame (int32_t sourcePos, IAudioFrameObserver::AudioFrame *frame)=0
virtual int	setExternalAudioSourceVolume (int32_t sourcePos, int32_t volume)=0
virtual int	pullAudioFrame (IAudioFrameObserver::AudioFrame *frame)=0
virtual int	setExternalVideoSource (bool enable, bool useTexture)=0
virtual int	pushVideoFrame (ExternalVideoFrame *frame)=0
virtual int	registerVideoEncodedFrameObserver (IVideoEncodedFrameObserver *observer)=0

Constructor & Destructor Documentation

~IMediaEngine()

virtual agora::media::IMediaEngine::~IMediaEngine ( )

inlinevirtual

Member Function Documentation

release()

virtual void agora::media::IMediaEngine::release ( )

pure virtual

registerAudioFrameObserver()

virtual int agora::media::IMediaEngine::registerAudioFrameObserver ( IAudioFrameObserver *  observer )

pure virtual

Registers an audio frame observer object.

This method is used to register an audio frame observer object (register a callback). This method is required to register callbacks when the engine is required to provide an onRecordAudioFrame or onPlaybackAudioFrame callback.

Note

Ensure that you call this method before joining a channel.

Parameters

observer

Audio frame observer object instance. See IAudioFrameObserver. Set the value as NULL to release the audio observer object. Agora recommends calling registerAudioFrameObserver(NULL) after receiving the onLeaveChannel callback.

Returns

• 0: Success.
• < 0: Failure.

registerVideoFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoFrameObserver ( IVideoFrameObserver *  observer )

pure virtual

Registers a video frame observer object.

You need to implement the IVideoFrameObserver class in this method, and register callbacks according to your scenarios.

After you successfully register the video frame observer, the SDK triggers the registered callbacks each time a video frame is received.

Note

• When handling the video data returned in the callbacks, pay attention to the changes in the width and height parameters, which may be adapted under the following circumstances:
  • When the network condition deteriorates, the video resolution decreases incrementally.
  • If the user adjusts the video profile, the resolution of the video returned in the callbacks also changes.
• Ensure that you call this method before joining a channel.

Parameters

observer

Video frame observer object instance. If NULL is passed in, the registration is canceled.

Returns

• 0: Success.
• < 0: Failure.

registerVideoRenderFactory()

virtual int agora::media::IMediaEngine::registerVideoRenderFactory ( IExternalVideoRenderFactory *  factory )

pure virtual

Deprecated:

pushAudioFrame() [1/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( MEDIA_SOURCE_TYPE  type, IAudioFrameObserver::AudioFrame *  frame, bool  wrap  )

pure virtual

Pushes the external audio frame.

Deprecated:

This method is deprecated. Use pushAudioFrame [3/3] instead.

Parameters

type	Type of audio capture device: MEDIA_SOURCE_TYPE.
frame	Audio frame pointer: AudioFrame.
wrap	Whether to use the placeholder. We recommend setting the default value. true: Use. false: (Default) Not use.

Returns

• 0: Success.
• < 0: Failure.

pushAudioFrame() [2/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( IAudioFrameObserver::AudioFrame *  frame )

pure virtual

Pushes the external audio frame.

Deprecated:

This method is deprecated. Use pushAudioFrame [3/3] instead.

Parameters

frame

Pointer to the audio frame: AudioFrame.

Returns

• 0: Success.
• < 0: Failure.

pushAudioFrame() [3/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( int32_t  sourcePos, IAudioFrameObserver::AudioFrame *  frame  )

pure virtual

Pushes the external audio frame to a specified position.

Since

v3.5.1

According to your needs, you can push the external audio frame to one of three positions: after audio capture, before audio encoding, or before local playback. You can call this method multiple times to push one audio frame to multiple positions or multiple audio frames to different positions. For example, in the KTV scenario, you can push the singing voice to after audio capture, so that the singing voice can be processed by the SDK audio module and you can obtain a high-quality audio experience; you can also push the accompaniment to before audio encoding, so that the accompaniment is not affected by the audio module of the SDK.

Note

Call this method after joining a channel.

Parameters

sourcePos	The push position of the external audio frame. See AUDIO_EXTERNAL_SOURCE_POSITION.
frame	The external audio frame. See AudioFrame. The value range of the audio frame length (ms) is [10,60].

Returns

• 0: Success.
• < 0: Failure.
  • -2 (ERR_INVALID_ARGUMENT): The parameter is invalid.
  • -12 (ERR_TOO_OFTEN): The call frequency is too high, causing the internal buffer to overflow. Call this method again after 30-50 ms.

setExternalAudioSourceVolume()

virtual int agora::media::IMediaEngine::setExternalAudioSourceVolume ( int32_t  sourcePos, int32_t  volume  )

pure virtual

Sets the volume of the external audio frame in the specified position.

Since

v3.5.1

You can call this method multiple times to set the volume of external audio frames in different positions. The volume setting takes effect for all external audio frames that are pushed to the specified position.

Note

Call this method after joining a channel.

Parameters

sourcePos	The push position of the external audio frame. See AUDIO_EXTERNAL_SOURCE_POSITION.
volume	The volume of the external audio frame. The value range is [0,100]. The default value is 100, which represents the original value.

Returns

• 0: Success.
• < 0: Failure.
  • -2 (ERR_INVALID_ARGUMENT): The parameter is invalid.

pullAudioFrame()

virtual int agora::media::IMediaEngine::pullAudioFrame ( IAudioFrameObserver::AudioFrame *  frame )

pure virtual

Pulls the remote audio data.

Before calling this method, call the setExternalAudioSink(enabled: true) method to enable and set the external audio sink.

After a successful method call, the app pulls the decoded and mixed audio data for playback.

Note

• Ensure that you call this method after joining a channel.
• When using the SDK with versions earlier than v3.5.0, you will not receive the onPlaybackAudioFrame callback after using the Pull method to set the external audio sink.
• The difference between the onPlaybackAudioFrame callback and the pullAudioFrame method is as follows:
  • onPlaybackAudioFrame: The SDK sends the audio data to the app through this callback. Any delay in processing the audio frames may result in audio jitter.
  • pullAudioFrame: The app pulls the remote audio data. After setting the audio data parameters, the SDK adjusts the frame buffer and avoids problems caused by jitter in the external audio playback.

Parameters

frame

Pointers to the audio frame. See: AudioFrame.

Returns

• 0: Success.
• < 0: Failure.

setExternalVideoSource()

virtual int agora::media::IMediaEngine::setExternalVideoSource ( bool  enable, bool  useTexture  )

pure virtual

Configures the external video source.

Note

Ensure that you call this method before joining a channel.

Parameters

enable	Sets whether to use the external video source: true: Use the external video source. false: (Default) Do not use the external video source.
useTexture	Sets whether to use texture as an input: true: Use texture as an input. false: (Default) Do not use texture as an input.

Returns

• 0: Success.
• < 0: Failure.

pushVideoFrame()

virtual int agora::media::IMediaEngine::pushVideoFrame ( ExternalVideoFrame *  frame )

pure virtual

Pushes the video frame using the ExternalVideoFrame and passes the video frame to the Agora SDK.

Parameters

frame

Video frame to be pushed. See ExternalVideoFrame.

Note

In the COMMUNICATION profile, this method does not support video frames in the Texture format.

Returns

• 0: Success.
• < 0: Failure.

registerVideoEncodedFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoEncodedFrameObserver ( IVideoEncodedFrameObserver *  observer )

pure virtual

Registers a local encoded video frame observer.

Since

v3.4.5

After you successfully register the local encoded video frame observer, the SDK triggers the callbacks that you have implemented in the IVideoEncodedFrameObserver class each time a video frame is received.

Note

• Ensure that you call this method before joining a channel.
• The width and height of the video obtained through the observer may change due to poor network conditions and user-adjusted resolution.

Parameters

observer

The local encoded video frame observer. See IVideoEncodedFrameObserver. If null is passed, the observer registration is canceled.

Returns

• 0: Success.
• < 0: Failure.