技術(shù)背景

RTSP轉(zhuǎn)RTMP推送，好多開發(fā)者第一想到的是采用ffmpeg命令行的形式，如果對(duì)ffmpeg比較熟，而且產(chǎn)品不要額外的定制和更高階的要求，未嘗不可，如果對(duì)產(chǎn)品穩(wěn)定性、時(shí)延、斷網(wǎng)重連等有更高的技術(shù)訴求，比較好的辦法，還是采用我們的技術(shù)實(shí)現(xiàn)。

技術(shù)實(shí)現(xiàn)

以大牛直播SDK的多路RTSP轉(zhuǎn)RTMP推送模塊為例，首先拉取RTSP流，把未解碼的H.264/H.265、AAC/PCMA/PCMU數(shù)據(jù)回調(diào)上來，然后通過調(diào)用推送模塊的編碼后數(shù)據(jù)接口，同步轉(zhuǎn)發(fā)出去，整體下來，幾無多少延遲。如果需要把數(shù)據(jù)投遞到輕量級(jí)RTSP服務(wù)也可以。系統(tǒng)設(shè)計(jì)架構(gòu)圖如下：

1. 拉流：通過RTSP直播播放SDK的數(shù)據(jù)回調(diào)接口，拿到音視頻數(shù)據(jù)；

2. 轉(zhuǎn)推：通過RTMP直播推送SDK的編碼后數(shù)據(jù)輸入接口，把回調(diào)上來的數(shù)據(jù)，傳給RTMP直播推送模塊，實(shí)現(xiàn)RTSP數(shù)據(jù)流到RTMP服務(wù)器的轉(zhuǎn)發(fā)；

3. 錄像：如果需要錄像，借助RTSP直播播放SDK，拉到音視頻數(shù)據(jù)后，直接存儲(chǔ)MP4文件即可；

4. 快照：如果需要實(shí)時(shí)快照，拉流后，解碼調(diào)用播放端快照接口，生成快照，因?yàn)榭煺丈婕暗絭ideo數(shù)據(jù)解碼，如無必要，可不必開啟，不然會(huì)額外消耗性能。

5. 拉流預(yù)覽：如需預(yù)覽拉流數(shù)據(jù)，只要調(diào)用播放端的播放接口，即可實(shí)現(xiàn)拉流數(shù)據(jù)預(yù)覽；

6. 數(shù)據(jù)轉(zhuǎn)AAC后轉(zhuǎn)發(fā)：考慮到好多監(jiān)控設(shè)備出來的音頻可能是PCMA/PCMU的，如需要更通用的音頻格式，可以轉(zhuǎn)AAC后，在通過RTMP推送；

7. 轉(zhuǎn)推RTMP實(shí)時(shí)靜音：只需要在傳audio數(shù)據(jù)的地方，加個(gè)判斷即可；

8. 拉流速度反饋：通過RTSP播放端的實(shí)時(shí)碼率反饋event，拿到實(shí)時(shí)帶寬占用即可；

9. 整體網(wǎng)絡(luò)狀態(tài)反饋：考慮到有些攝像頭可能會(huì)臨時(shí)或異常關(guān)閉，RTMP服務(wù)器亦是，可以通過推拉流的event回調(diào)狀態(tài)，查看那整體網(wǎng)絡(luò)情況，如此界定：是拉不到流，還是推不到RTMP服務(wù)器。

多路RTMP/RTSP轉(zhuǎn)RTMP推送模塊功能支持：

1. 支持拉取rtmp流；
2. 支持拉取rtsp流；
3. Windows支持本地flv文件轉(zhuǎn)發(fā)(支持制定文件位置轉(zhuǎn)發(fā)，或轉(zhuǎn)發(fā)過程中seek)；
4. 支持本地預(yù)覽；
5. 支持轉(zhuǎn)發(fā)過程中，實(shí)時(shí)靜音；
6. 支持轉(zhuǎn)發(fā)過程中，切換rtmp/rtsp url，此外，windows平臺(tái)還支持切換本地flv文件；
7. 支持錄像模塊擴(kuò)展，可邊轉(zhuǎn)發(fā)邊錄制，每個(gè)文件錄制開始結(jié)束，均有狀態(tài)回饋；
8. 支持內(nèi)網(wǎng)RTSP網(wǎng)關(guān)模塊擴(kuò)展，拉取的流數(shù)據(jù)，可以流入到內(nèi)網(wǎng)RTSP網(wǎng)關(guān)模塊，對(duì)外微型RTSP媒體流服務(wù)（RTSP url），便于內(nèi)網(wǎng)訪問；
9. 音頻：AAC，并支持拉流后的音頻(PCMU/PCMA,Speex等)轉(zhuǎn)AAC后再轉(zhuǎn)發(fā)；
10. 視頻：H.264、H.265，支持h265轉(zhuǎn)發(fā)(rtsp/rtmp h265轉(zhuǎn)rtmp h265推送)；

上述實(shí)現(xiàn)，2016年我們已經(jīng)非常成熟，本次要談的，是開發(fā)者實(shí)際場景用到的一個(gè)技術(shù)需求，如何實(shí)現(xiàn)視頻用RTSP數(shù)據(jù)源獲取到的，音頻采集麥克風(fēng)的數(shù)據(jù)。

廢話不多說，上代碼：

先說開始拉流、停止拉流設(shè)計(jì)如下，如果是用rtsp的audio，那么我們就開啟audio數(shù)據(jù)的回調(diào)，如果采用麥克風(fēng)的，這里只要開video的即可。

/** SmartRelayDemo.java* Created by daniusdk.com* weChat: xinsheng120*/
private boolean StartPull()
{if ( isPulling )return false;if(!isPlaying){if (!OpenPullHandle())return false;}if(audio_opt_ == 2){libPlayer.SmartPlayerSetAudioDataCallback(player_handle_, new PlayerAudioDataCallback(stream_publisher_));}if(video_opt_ == 2){libPlayer.SmartPlayerSetVideoDataCallback(player_handle_, new PlayerVideoDataCallback(stream_publisher_));}int is_pull_trans_code  = 1;libPlayer.SmartPlayerSetPullStreamAudioTranscodeAAC(player_handle_, is_pull_trans_code);int startRet = libPlayer.SmartPlayerStartPullStream(player_handle_);if (startRet != 0) {Log.e(TAG, "Failed to start pull stream!");if(!isPlaying){releasePlayerHandle();}return false;}isPulling = true;return true;
}private void StopPull()
{if ( !isPulling )return;isPulling = false;if (null == libPlayer || 0 == player_handle_)return;libPlayer.SmartPlayerStopPullStream(player_handle_);if ( !isPlaying){releasePlayerHandle();}
}

OpenPullHandle()實(shí)現(xiàn)邏輯如下，常規(guī)的參數(shù)設(shè)置，和event callback設(shè)置等。

private boolean OpenPullHandle()
{//playbackUrl可自定義playbackUrl = "rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream";if (playbackUrl == null) {Log.e(TAG, "playback URL is null...");return false;}player_handle_ = libPlayer.SmartPlayerOpen(context_);if (player_handle_ == 0) {Log.e(TAG, "playerHandle is null..");return false;}libPlayer.SetSmartPlayerEventCallbackV2(player_handle_,new EventHandlePlayerV2());libPlayer.SmartPlayerSetBuffer(player_handle_, playBuffer);// set report download speedlibPlayer.SmartPlayerSetReportDownloadSpeed(player_handle_, 1, 2);//設(shè)置RTSP超時(shí)時(shí)間int rtsp_timeout = 10;libPlayer.SmartPlayerSetRTSPTimeout(player_handle_, rtsp_timeout);//設(shè)置RTSP TCP/UDP模式自動(dòng)切換int is_auto_switch_tcp_udp = 1;libPlayer.SmartPlayerSetRTSPAutoSwitchTcpUdp(player_handle_, is_auto_switch_tcp_udp);libPlayer.SmartPlayerSaveImageFlag(player_handle_, 1);// It only used when playback RTSP stream..//libPlayer.SmartPlayerSetRTSPTcpMode(playerHandle, 1);libPlayer.SmartPlayerSetUrl(player_handle_, playbackUrl);return true;
}

拉流后，轉(zhuǎn)推RTMP的設(shè)計(jì)如下：

btnRTMPPusher.setOnClickListener(new Button.OnClickListener() {// @Overridepublic void onClick(View v) {if (stream_publisher_.is_rtmp_publishing()) {stopPush();btnRTMPPusher.setText("推送RTMP");return;}Log.i(TAG, "onClick start push rtmp..");InitAndSetConfig();String rtmp_pusher_url = "rtmp://192.168.0.104:1935/hls/stream1";//String rtmp_pusher_url = relayStreamUrl;if (!stream_publisher_.SetURL(rtmp_pusher_url))Log.e(TAG, "Failed to set publish stream URL..");boolean start_ret = stream_publisher_.StartPublisher();if (!start_ret) {stream_publisher_.try_release();Log.e(TAG, "Failed to start push stream..");return;}startAudioRecorder();btnRTMPPusher.setText("停止推送");}
});

InitAndSetConfig()設(shè)計(jì)如下：

private void InitAndSetConfig() {if (null == libPublisher)return;if (!stream_publisher_.empty())return;Log.i(TAG, "InitAndSetConfig video width: " + video_width_ + ", height" + video_height_);long handle = libPublisher.SmartPublisherOpen(context_, audio_opt_, video_opt_,  video_width_, video_height_);if (0==handle) {Log.e(TAG, "sdk open failed!");return;}Log.i(TAG, "publisherHandle=" + handle);int fps = 25;int gop = fps * 3;initialize_publisher(libPublisher, handle, video_width_, video_height_, fps, gop);stream_publisher_.set(libPublisher, handle);
}

這里可以看到，我們在轉(zhuǎn)推RTMP的時(shí)候，調(diào)用了startAudioRecorder()來做麥克風(fēng)的采集：

void startAudioRecorder() {if(audio_opt_ != 1)return;if (audio_recorder_ != null)return;audio_recorder_ = new NTAudioRecordV2(this);Log.i(TAG, "startAudioRecorder call audio_recorder_.start()+++...");audio_recorder_callback_ = new NTAudioRecordV2CallbackImpl(stream_publisher_, null);audio_recorder_.AddCallback(audio_recorder_callback_);if (!audio_recorder_.Start(is_pcma_ ? 8000 : 44100, 1) ) {audio_recorder_.RemoveCallback(audio_recorder_callback_);audio_recorder_callback_ = null;audio_recorder_ = null;Log.e(TAG, "startAudioRecorder start failed.");}else {Log.i(TAG, "startAudioRecorder call audio_recorder_.start() OK---...");}
}void stopAudioRecorder() {if (null == audio_recorder_)return;Log.i(TAG, "stopAudioRecorder+++");audio_recorder_.Stop();if (audio_recorder_callback_ != null) {audio_recorder_.RemoveCallback(audio_recorder_callback_);audio_recorder_callback_ = null;}audio_recorder_ = null;Log.i(TAG, "stopAudioRecorder---");
}

采集到的audio回調(diào)上來后，我們調(diào)RTMP推送接口，把數(shù)據(jù)投遞下去即可：

private static class NTAudioRecordV2CallbackImpl implements NTAudioRecordV2Callback {private WeakReference<LibPublisherWrapper> publisher_0_;private WeakReference<LibPublisherWrapper> publisher_1_;public NTAudioRecordV2CallbackImpl(LibPublisherWrapper publisher_0, LibPublisherWrapper publisher_1) {if (publisher_0 != null)publisher_0_ = new WeakReference<>(publisher_0);if (publisher_1 != null)publisher_1_ = new WeakReference<>(publisher_1);}private final LibPublisherWrapper get_publisher_0() {if (publisher_0_ !=null)return publisher_0_.get();return null;}private final LibPublisherWrapper get_publisher_1() {if (publisher_1_ != null)return publisher_1_.get();return null;}@Overridepublic void onNTAudioRecordV2Frame(ByteBuffer data, int size, int sampleRate, int channel, int per_channel_sample_number) {//Log.i(TAG, "onNTAudioRecordV2Frame size=" + size + " sampleRate=" + sampleRate + " channel=" + channel//			 + " per_channel_sample_number=" + per_channel_sample_number);LibPublisherWrapper publisher_0 = get_publisher_0();if (publisher_0 != null)publisher_0.OnPCMData(data, size, sampleRate, channel, per_channel_sample_number);LibPublisherWrapper publisher_1 = get_publisher_1();if (publisher_1 != null)publisher_1.OnPCMData(data, size, sampleRate, channel, per_channel_sample_number);}
}

編碼后的視頻投遞設(shè)計(jì)如下：

class PlayerVideoDataCallback implements NTVideoDataCallback
{private WeakReference<LibPublisherWrapper> publisher_;private int video_buffer_size = 0;private ByteBuffer video_buffer_ = null;public PlayerVideoDataCallback(LibPublisherWrapper publisher) {if (publisher != null)publisher_ = new WeakReference<>(publisher);}@Overridepublic ByteBuffer getVideoByteBuffer(int size){if( size < 1 ){return null;}if ( size <= video_buffer_size &&  video_buffer_ != null ){return  video_buffer_;}video_buffer_size = size + 1024;video_buffer_size = (video_buffer_size+0xf) & (~0xf);video_buffer_ = ByteBuffer.allocateDirect(video_buffer_size);return video_buffer_;}public void onVideoDataCallback(int ret, int video_codec_id, int sample_size, int is_key_frame, long timestamp, int width, int height, long presentation_timestamp){if ( video_buffer_ == null)return;LibPublisherWrapper publisher = publisher_.get();if (null == publisher)return;if (!publisher.is_publishing())return;video_buffer_.rewind();publisher.PostVideoEncodedData(video_codec_id, video_buffer_, sample_size, is_key_frame, timestamp, presentation_timestamp);}
}

總結(jié)

從我發(fā)的Android平臺(tái)RTSP轉(zhuǎn)RTMP推送的demo界面，可以看到，這個(gè)demo，不是單純的RTSP轉(zhuǎn)RTMP推送的，還可以實(shí)現(xiàn)RTSP流獲取后，回調(diào)上來解碼后的數(shù)據(jù)，然后添加動(dòng)態(tài)水印或其他處理后，把video數(shù)據(jù)二次編碼推送出去。或者audio數(shù)據(jù)二次處理。

此外，還可以實(shí)現(xiàn)拉流的數(shù)據(jù)預(yù)覽播放、把數(shù)據(jù)注入到輕量級(jí)RTSP服務(wù)模塊，然后二次編碼的數(shù)據(jù)，本地錄像、快照等。一個(gè)好的RTSP轉(zhuǎn)RTMP推送的模塊，一定要足夠的靈活，擴(kuò)展性好，才能很快的實(shí)現(xiàn)客戶的技術(shù)訴求。以上拋磚引玉，感興趣的開發(fā)者，可以跟我單獨(dú)探討。