Agora C++ API Reference for All Platforms
Public Member Functions | List of all members
agora::media::IMediaEngine Class Referenceabstract

#include <IAgoraMediaEngine.h>

Public Member Functions

virtual ~IMediaEngine ()
 
virtual void release ()=0
 
virtual int registerAudioFrameObserver (IAudioFrameObserver *observer)=0
 
virtual int registerVideoFrameObserver (IVideoFrameObserver *observer)=0
 
virtual int registerVideoRenderFactory (IExternalVideoRenderFactory *factory) AGORA_DEPRECATED_ATTRIBUTE=0
 
virtual int pushAudioFrame (MEDIA_SOURCE_TYPE type, IAudioFrameObserver::AudioFrame *frame, bool wrap) AGORA_DEPRECATED_ATTRIBUTE=0
 
virtual int pushAudioFrame (IAudioFrameObserver::AudioFrame *frame) AGORA_DEPRECATED_ATTRIBUTE=0
 
virtual int pushAudioFrame (int32_t sourcePos, IAudioFrameObserver::AudioFrame *frame)=0
 
virtual int setExternalAudioSourceVolume (int32_t sourcePos, int32_t volume)=0
 
virtual int pullAudioFrame (IAudioFrameObserver::AudioFrame *frame)=0
 
virtual int setExternalVideoSource (bool enable, bool useTexture)=0
 
virtual int pushVideoFrame (ExternalVideoFrame *frame)=0
 
virtual int registerVideoEncodedFrameObserver (IVideoEncodedFrameObserver *observer)=0
 

Constructor & Destructor Documentation

◆ ~IMediaEngine()

virtual agora::media::IMediaEngine::~IMediaEngine ( )
inlinevirtual

Member Function Documentation

◆ release()

virtual void agora::media::IMediaEngine::release ( )
pure virtual

◆ registerAudioFrameObserver()

virtual int agora::media::IMediaEngine::registerAudioFrameObserver ( IAudioFrameObserver *  observer)
pure virtual

Registers an audio frame observer object.

This method is used to register an audio frame observer object (register a callback). This method is required to register callbacks when the engine is required to provide an onRecordAudioFrame or onPlaybackAudioFrame callback.

Note
Ensure that you call this method before joining a channel.
Parameters
observerAudio frame observer object instance. See IAudioFrameObserver. Set the value as NULL to release the audio observer object. Agora recommends calling registerAudioFrameObserver(NULL) after receiving the onLeaveChannel callback.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerVideoFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoFrameObserver ( IVideoFrameObserver *  observer)
pure virtual

Registers a video frame observer object.

You need to implement the IVideoFrameObserver class in this method, and register callbacks according to your scenarios.

After you successfully register the video frame observer, the SDK triggers the registered callbacks each time a video frame is received.

Note
  • When handling the video data returned in the callbacks, pay attention to the changes in the width and height parameters, which may be adapted under the following circumstances:
    • When the network condition deteriorates, the video resolution decreases incrementally.
    • If the user adjusts the video profile, the resolution of the video returned in the callbacks also changes.
  • Ensure that you call this method before joining a channel.
  • When you use the SDK v3.0.1 or later, note the following:
    • The SDK no longer guarantees that callback functions in IVideoFrameObserver are reported in the same thread. The SDK only guarantees the sequence of these callbacks.
    • If you are using OpenGL to perform image enhancement on the raw video data, you need to actively switch the OpenGL context in the callback function in IVideoFrameObserver to adapt to a multi-threaded scenario; otherwise, the image enhancement cannot work.
Parameters
observerVideo frame observer object instance. If NULL is passed in, the registration is canceled.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerVideoRenderFactory()

virtual int agora::media::IMediaEngine::registerVideoRenderFactory ( IExternalVideoRenderFactory *  factory)
pure virtual
Deprecated:

◆ pushAudioFrame() [1/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( MEDIA_SOURCE_TYPE  type,
IAudioFrameObserver::AudioFrame *  frame,
bool  wrap 
)
pure virtual

Pushes the external audio frame.

Deprecated:
This method is deprecated. Use pushAudioFrame [3/3] instead.
Parameters
typeType of audio capture device: MEDIA_SOURCE_TYPE.
frameAudio frame pointer: AudioFrame.
wrapWhether to use the placeholder. We recommend setting the default value.
  • true: Use.
  • false: (Default) Not use.
Returns
  • 0: Success.
  • < 0: Failure.

◆ pushAudioFrame() [2/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( IAudioFrameObserver::AudioFrame *  frame)
pure virtual

Pushes the external audio frame.

Deprecated:
This method is deprecated. Use pushAudioFrame [3/3] instead.
Parameters
framePointer to the audio frame: AudioFrame.
Returns
  • 0: Success.
  • < 0: Failure.

◆ pushAudioFrame() [3/3]

virtual int agora::media::IMediaEngine::pushAudioFrame ( int32_t  sourcePos,
IAudioFrameObserver::AudioFrame *  frame 
)
pure virtual

Pushes the external audio frame to a specified position.

Since
v3.5.1

According to your needs, you can push the external audio frame to one of three positions: after audio capture, before audio encoding, or before local playback. You can call this method multiple times to push one audio frame to multiple positions or multiple audio frames to different positions. For example, in the KTV scenario, you can push the singing voice to after audio capture, so that the singing voice can be processed by the SDK audio module and you can obtain a high-quality audio experience; you can also push the accompaniment to before audio encoding, so that the accompaniment is not affected by the audio module of the SDK.

Note
Call this method after joining a channel.
Parameters
sourcePosThe push position of the external audio frame. See AUDIO_EXTERNAL_SOURCE_POSITION.
frameThe external audio frame. See AudioFrame. The value range of the audio frame length (ms) is [10,60].
Returns
  • 0: Success.
  • < 0: Failure.
    • -2 (ERR_INVALID_ARGUMENT): The parameter is invalid.
    • -12 (ERR_TOO_OFTEN): The call frequency is too high, causing the internal buffer to overflow. Call this method again after 30-50 ms.

◆ setExternalAudioSourceVolume()

virtual int agora::media::IMediaEngine::setExternalAudioSourceVolume ( int32_t  sourcePos,
int32_t  volume 
)
pure virtual

Sets the volume of the external audio frame in the specified position.

Since
v3.5.1

You can call this method multiple times to set the volume of external audio frames in different positions. The volume setting takes effect for all external audio frames that are pushed to the specified position.

Note
Call this method after joining a channel.
Parameters
sourcePosThe push position of the external audio frame. See AUDIO_EXTERNAL_SOURCE_POSITION.
volumeThe volume of the external audio frame. The value range is [0,100]. The default value is 100, which represents the original value.
Returns
  • 0: Success.
  • < 0: Failure.
    • -2 (ERR_INVALID_ARGUMENT): The parameter is invalid.

◆ pullAudioFrame()

virtual int agora::media::IMediaEngine::pullAudioFrame ( IAudioFrameObserver::AudioFrame *  frame)
pure virtual

Pulls the remote audio data.

Before calling this method, call the setExternalAudioSink(enabled: true) method to enable and set the external audio sink.

After a successful method call, the app pulls the decoded and mixed audio data for playback.

Note
  • Ensure that you call this method after joining a channel.
  • When using the SDK with versions earlier than v3.5.0, you will not receive the onPlaybackAudioFrame callback after using the Pull method to set the external audio sink.
  • The difference between the onPlaybackAudioFrame callback and the pullAudioFrame method is as follows:
    • onPlaybackAudioFrame: The SDK sends the audio data to the app through this callback. Any delay in processing the audio frames may result in audio jitter.
    • pullAudioFrame: The app pulls the remote audio data. After setting the audio data parameters, the SDK adjusts the frame buffer and avoids problems caused by jitter in the external audio playback.
Parameters
framePointers to the audio frame. See: AudioFrame.
Returns
  • 0: Success.
  • < 0: Failure.

◆ setExternalVideoSource()

virtual int agora::media::IMediaEngine::setExternalVideoSource ( bool  enable,
bool  useTexture 
)
pure virtual

Configures the external video source.

Note
Ensure that you call this method before joining a channel.
Parameters
enableSets whether to use the external video source:
  • true: Use the external video source.
  • false: (Default) Do not use the external video source.
useTextureSets whether to use texture as an input:
  • true: Use texture as an input.
  • false: (Default) Do not use texture as an input.
Returns
  • 0: Success.
  • < 0: Failure.

◆ pushVideoFrame()

virtual int agora::media::IMediaEngine::pushVideoFrame ( ExternalVideoFrame *  frame)
pure virtual

Pushes the video frame using the ExternalVideoFrame and passes the video frame to the Agora SDK.

Parameters
frameVideo frame to be pushed. See ExternalVideoFrame.
Note
In the COMMUNICATION profile, this method does not support video frames in the Texture format.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerVideoEncodedFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoEncodedFrameObserver ( IVideoEncodedFrameObserver *  observer)
pure virtual

Registers a local encoded video frame observer.

Since
v3.4.5

After you successfully register the local encoded video frame observer, the SDK triggers the callbacks that you have implemented in the IVideoEncodedFrameObserver class each time a video frame is received.

Note
  • Ensure that you call this method before joining a channel.
  • The width and height of the video obtained through the observer may change due to poor network conditions and user-adjusted resolution.
Parameters
observerThe local encoded video frame observer. See IVideoEncodedFrameObserver. If null is passed, the observer registration is canceled.
Returns
  • 0: Success.
  • < 0: Failure.