Agora C++ API Reference for All Platforms
Public Member Functions | Protected Member Functions | List of all members
agora::media::IMediaEngine Class Referenceabstract

Inherited by agora::media::IMediaEngineEx.

Public Member Functions

virtual int registerAudioFrameObserver (IAudioFrameObserver *observer)=0
 
virtual int registerVideoFrameObserver (IVideoFrameObserver *observer)=0
 
virtual int registerVideoEncodedFrameObserver (IVideoEncodedFrameObserver *observer)=0
 
virtual int registerFaceInfoObserver (IFaceInfoObserver *observer)=0
 
virtual int pushAudioFrame (IAudioFrameObserverBase::AudioFrame *frame, rtc::track_id_t trackId=0)=0
 
virtual int pullAudioFrame (IAudioFrameObserverBase::AudioFrame *frame)=0
 
virtual int setExternalVideoSource (bool enabled, bool useTexture, EXTERNAL_VIDEO_SOURCE_TYPE sourceType=VIDEO_FRAME, rtc::SenderOptions encodedVideoOption=rtc::SenderOptions())=0
 
virtual int setExternalAudioSource (bool enabled, int sampleRate, int channels, bool localPlayback=false, bool publish=true) __deprecated=0
 
virtual rtc::track_id_t createCustomAudioTrack (rtc::AUDIO_TRACK_TYPE trackType, const rtc::AudioTrackConfig &config)=0
 
virtual int destroyCustomAudioTrack (rtc::track_id_t trackId)=0
 
virtual int setExternalAudioSink (bool enabled, int sampleRate, int channels)=0
 
virtual int enableCustomAudioLocalPlayback (rtc::track_id_t trackId, bool enabled)=0
 
virtual int pushVideoFrame (base::ExternalVideoFrame *frame, unsigned int videoTrackId=0)=0
 
virtual int pushEncodedVideoImage (const unsigned char *imageBuffer, size_t length, const agora::rtc::EncodedVideoFrameInfo &videoEncodedFrameInfo, unsigned int videoTrackId=0)=0
 
virtual int addVideoFrameRenderer (IVideoFrameObserver *renderer)=0
 
virtual int removeVideoFrameRenderer (IVideoFrameObserver *renderer)=0
 
virtual void release ()=0
 

Protected Member Functions

virtual ~IMediaEngine ()
 

Detailed Description

The IMediaEngine class.

Constructor & Destructor Documentation

◆ ~IMediaEngine()

virtual agora::media::IMediaEngine::~IMediaEngine ( )
inlineprotectedvirtual

Member Function Documentation

◆ registerAudioFrameObserver()

virtual int agora::media::IMediaEngine::registerAudioFrameObserver ( IAudioFrameObserver observer)
pure virtual

Registers an audio frame observer object.

Note
Ensure that you call this method before joinChannel.
Parameters
observerA pointer to the audio frame observer object: IAudioFrameObserver, nullptr means unregistering observer instead.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerVideoFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoFrameObserver ( IVideoFrameObserver observer)
pure virtual

Registers a video frame observer object.

Note
  • Ensure that you call this method before joining the channel.
  • If you register an observer for video raw video data, you cannot register an IVideoEncodedFrameObserver object.
Parameters
observerA pointer to the video frame observer: IVideoFrameObserver.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerVideoEncodedFrameObserver()

virtual int agora::media::IMediaEngine::registerVideoEncodedFrameObserver ( IVideoEncodedFrameObserver observer)
pure virtual

Registers a receiver object for the encoded video image.

Note
  • Ensure that you call this method before joining the channel.
Parameters
observerA pointer to the observer of the encoded video image: IVideoEncodedFrameObserver.
Returns
  • 0: Success.
  • < 0: Failure.

◆ registerFaceInfoObserver()

virtual int agora::media::IMediaEngine::registerFaceInfoObserver ( IFaceInfoObserver observer)
pure virtual

Registers a face info observer object.

Note
Ensure that you call this method before joinChannel.
Parameters
observerA pointer to the face info observer object: IFaceInfoObserver.
Returns
  • 0: Success.
  • < 0: Failure.

◆ pushAudioFrame()

virtual int agora::media::IMediaEngine::pushAudioFrame ( IAudioFrameObserverBase::AudioFrame frame,
rtc::track_id_t  trackId = 0 
)
pure virtual

Pushes the external audio data to the app.

Parameters
frameThe audio buffer data.
trackIdThe audio track ID.
Returns
  • 0: Success.
  • < 0: Failure.

◆ pullAudioFrame()

virtual int agora::media::IMediaEngine::pullAudioFrame ( IAudioFrameObserverBase::AudioFrame frame)
pure virtual

Pulls the remote audio data.

After a successful method call, the app pulls the decoded and mixed audio data for playback.

The difference between this method and the onPlaybackAudioFrame is as follows:

  • onPlaybackAudioFrame: The SDK sends the audio data to the app once every 10 ms. Any delay in processing the audio frames may result in audio jitter.
  • pullAudioFrame: The app pulls the remote audio data. After setting the audio data parameters, the SDK adjusts the frame buffer and avoids problems caused by jitter in the external audio playback.
Parameters
frameThe pointer to the audio frame: AudioFrame.
Returns
  • 0: Success.
  • < 0: Failure.

◆ setExternalVideoSource()

virtual int agora::media::IMediaEngine::setExternalVideoSource ( bool  enabled,
bool  useTexture,
EXTERNAL_VIDEO_SOURCE_TYPE  sourceType = VIDEO_FRAME,
rtc::SenderOptions  encodedVideoOption = rtc::SenderOptions() 
)
pure virtual

Sets the external video source.

Once the external video source is enabled, the SDK prepares to accept the external video frame.

Parameters
enabledDetermines whether to enable the external video source.
  • true: Enable the external video source. Once set, the SDK creates the external source and prepares video data from pushVideoFrame or pushEncodedVideoImage.
  • false: Disable the external video source.
useTextureDetermines whether to use textured video data.
  • true: Use texture, which is not supported now.
  • False: Do not use texture.
sourceTypeDetermines the type of external video source frame.
  • ENCODED_VIDEO_FRAME: The external video source is encoded.
  • VIDEO_FRAME: The external video source is not encoded.
encodedVideoOptionVideo encoded track option, which is only used for ENCODED_VIDEO_FRAME.
Returns
  • 0: Success.
  • < 0: Failure.

◆ setExternalAudioSource()

virtual int agora::media::IMediaEngine::setExternalAudioSource ( bool  enabled,
int  sampleRate,
int  channels,
bool  localPlayback = false,
bool  publish = true 
)
pure virtual

Sets the external audio source.

Note
Ensure that you call this method before joining the channel.
Deprecated:
This method is deprecated. Use createCustomAudioTrack(rtc::AUDIO_TRACK_TYPE trackType, const rtc::AudioTrackConfig& config) instead.
Parameters
enabledDetermines whether to enable the external audio source:
  • true: Enable the external audio source.
  • false: (default) Disable the external audio source.
sampleRateThe Sample rate (Hz) of the external audio source, which can set be as 8000, 16000, 32000, 44100, or 48000.
channelsThe number of channels of the external audio source, which can be set as 1 or 2:
  • 1: Mono.
  • 2: Stereo.
localPlaybackEnable/Disables the local playback of external audio track:
  • true: Enable local playback
  • false: (Default) Do not enable local playback
publishDetermines whether to publish the external audio track:
  • true: (Default) Publish the external audio track.
  • false: Don`t publish the external audio track.
Returns
  • 0: Success.
  • < 0: Failure.

◆ createCustomAudioTrack()

virtual rtc::track_id_t agora::media::IMediaEngine::createCustomAudioTrack ( rtc::AUDIO_TRACK_TYPE  trackType,
const rtc::AudioTrackConfig config 
)
pure virtual

Create a custom audio track and get the audio track id.

Note
Ensure that you call this method before calling joinChannel.
Parameters
trackTypeThe type of custom audio track See AUDIO_TRACK_TYPE.
configThe config of custom audio track See AudioTrackConfig.
Returns
  • If the call is successful, SDK returns audio track id.
  • If the call fails, SDK returns 0xffffffff.

◆ destroyCustomAudioTrack()

virtual int agora::media::IMediaEngine::destroyCustomAudioTrack ( rtc::track_id_t  trackId)
pure virtual

Destroy custom audio track by trackId

Parameters
trackIdThe custom audio track id.
Returns
  • 0: Success.
  • < 0: Failure.

◆ setExternalAudioSink()

virtual int agora::media::IMediaEngine::setExternalAudioSink ( bool  enabled,
int  sampleRate,
int  channels 
)
pure virtual

Sets the external audio sink.

This method applies to scenarios where you want to use external audio data for playback. After calling the initialize method and pass value of false in the enableAudioDevice member in the RtcEngineContext struct, you can call the pullAudioFrame method to pull the remote audio data, process it, and play it with the audio effects that you want.

Note
Once you call the initialize method and pass value of false in the enableAudioDevice member in the RtcEngineContext struct, the app will not retrieve any audio data from the onPlaybackAudioFrame callback.
Parameters
enabledSets whether or not to the external audio sink
  • true: Enables the external audio sink.
  • false: Disables the external audio sink.
sampleRateSets the sample rate (Hz) of the external audio sink, which can be set as 16000, 32000, 44100 or 48000.
channelsSets the number of audio channels of the external audio sink:
  • 1: Mono.
  • 2: Stereo.
Returns
  • 0: Success.
  • < 0: Failure.

◆ enableCustomAudioLocalPlayback()

virtual int agora::media::IMediaEngine::enableCustomAudioLocalPlayback ( rtc::track_id_t  trackId,
bool  enabled 
)
pure virtual

Sets the external audio track.

Note
Ensure that you call this method before joining the channel.
Parameters
trackIdThe custom audio track id.
enabledEnable/Disables the local playback of external audio track:
  • true: Enable local playback
  • false: Do not enable local playback
Returns
  • 0: Success.
  • < 0: Failure.

◆ pushVideoFrame()

virtual int agora::media::IMediaEngine::pushVideoFrame ( base::ExternalVideoFrame frame,
unsigned int  videoTrackId = 0 
)
pure virtual

Pushes the external video frame to the app.

Parameters
frameThe external video frame: ExternalVideoFrame.
videoTrackIdThe id of the video track.
  • 0: Success.
  • < 0: Failure.

◆ pushEncodedVideoImage()

virtual int agora::media::IMediaEngine::pushEncodedVideoImage ( const unsigned char *  imageBuffer,
size_t  length,
const agora::rtc::EncodedVideoFrameInfo videoEncodedFrameInfo,
unsigned int  videoTrackId = 0 
)
pure virtual

Pushes the encoded video image to the app.

Parameters
imageBufferA pointer to the video image.
lengthThe data length.
videoEncodedFrameInfoThe reference to the information of the encoded video frame: EncodedVideoFrameInfo.
videoTrackIdThe id of the video track.
  • 0: Success.
  • < 0: Failure.

◆ addVideoFrameRenderer()

virtual int agora::media::IMediaEngine::addVideoFrameRenderer ( IVideoFrameObserver renderer)
pure virtual

@hide For internal usage only

◆ removeVideoFrameRenderer()

virtual int agora::media::IMediaEngine::removeVideoFrameRenderer ( IVideoFrameObserver renderer)
pure virtual

@hide For internal usage only

◆ release()

virtual void agora::media::IMediaEngine::release ( )
pure virtual