Re: [FFmpeg-user] Unable to record VP8+Opus from RTP+SDP

2019-09-03 Thread Juan Navarro 

Thanks for the comments;

in case anyone ends up having a similar issue, it all turned out to be
an issue in VLC itself; version 2.2.2 (from Ubuntu 16.04 "Xenial") won't
play OPUS audio correctly, at least not in the WEBM and MP4 sample files
I have been able to record with GStreamer. However, updating VLC to
3.0.7 fixed this and it is now able to play the files back, without issues.

I'm now trying to do the same type of recording with FFmpeg but using
MP4 as container, and there are some issues, but I'm opening a different
email thread for that ("mp4 opus invalid extradata size").

___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Unable to record VP8+Opus from RTP+SDP

2019-08-15 Thread Ted Park 
> If you notice, there is a lot of metadata about the RTP session (ssrc,
> cname), some about the ICE protocol (for WebRTC), but there isn't much
> information there about the specifics of the codecs.

I think that statement is rather specious… WebRTC relies on some other app 
(perhaps a browser webapp implemented in js) to negotiate those specifics.

> In this case, the
> preferred audio payload type is 111, which corresponds to the OPUS
> codec. The specification "opus/48000/2" is expected to already convey
> enough information to let the other peer know what codec it is going to
> receive, and it allows for some (much) variance in the actual encoding
> parameters that are going to be used; the receiver is just able to
> auto-detect those and decode the stream.

That’s just “opus/48000/2” being the only string you’re allowed to put for the 
subtype/clock/channel fields of rtpmap. The actual encoding might differ, but 
as you said, in most cases the receiver is able to use the in-band signaling to 
decode. But in this case you’re trying to codec copy using parameters ffmpeg 
determined solely from the SDP file.

>   v=0
>   o=- 6323209318560568134 2 IN IP4 127.0.0.1
>   s=-
>   t=0 0
>   a=group:BUNDLE 0 1
>   a=msid-semantic: WMS XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
>   m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
...
>   a=mid:0
>   a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
>   a=extmap:2
>   http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
>   a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
>   a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
>   a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
>   a=sendrecv
>   a=msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
>   816c42db-7d97-4053-86a0-bae0e5010985
>   a=rtcp-mux
>   a=rtpmap:111 opus/48000/2
>   a=rtcp-fb:111 transport-cc
>   a=fmtp:111 minptime=10;useinbandfec=1
...
>   a=ssrc:3169356447 cname:XZAw22pKY7Y9bCph
>   a=ssrc:3169356447 msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
>   816c42db-7d97-4053-86a0-bae0e5010985
>   a=ssrc:3169356447 mslabel:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
>   a=ssrc:3169356447 label:816c42db-7d97-4053-86a0-bae0e5010985


WebRTC is far more complex than simple RTP. A separate application signals 
metadata about the connection and whether streams/tracks change or are added.
The extmap attributes declare the mapping between the RTP header extensions and 
the corresponding parameters that will be used in the session, which may well 
include framing parameters (for VBR, DTX, etc.)

And the fmtp attribute actually does specify formatting info...

> For the cases where the codec has really incompatible profiles, to the
> point where trying to decode with the wrong profile would fail, then
> that information is also transmitted in the SDP; this is the case of the
> H.264 profile levels (Baseline, Main, High, etc.), which are then
> accordingly transmitted in the SDP Offer as totally different Payload
> Types. But you'll notice that other codecs, like VP8, don't really need
> such distinction.

As with opus, the “extradata” that ffmpeg complains is missing would be 
provided from a separate signaler to WebRTC.

>   m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125
>   107 108 109 124 120 123
...
>   a=rtpmap:96 VP8/9
>   a=rtcp-fb:96 goog-remb
>   a=rtcp-fb:96 transport-cc
>   a=rtcp-fb:96 ccm fir
>   a=rtcp-fb:96 nack
>   a=rtcp-fb:96 nack pli
>   a=rtpmap:97 rtx/9
>   a=fmtp:97 apt=96

Also, as this session uses the AVPF profile, it can do things, like make sure 
the timestamp increments are synchronized with the RTP clock, get a keyframe at 
the start, have the server retransmit lost packets, etc.

> Of course, such an SDP Offer is not really missing any crucial detail;
> another browser such as Chrome, Firefox, Safari, etc. will be able to
> receive and decode the incoming audio and video. So what I'm really
> trying to do is doing the same with FFmpeg... now if you compare
> Chrome's SDP with mine, it turns out mine does already have all relevant
> details that are included in a "full-blown" SDP message.

You mentioned mediasoup. Can you have it serve the sdp or get the parameters 
and use those to write a proper one? I don’t know how much access you have to 
the media soup router, is it actually on localhost? 
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Unable to record VP8+Opus from RTP+SDP

2019-08-14 Thread Juan Navarro 

>> So I’m not sure how ffmpeg parses SDP files or if it uses a library
but when you step back and take a look, you haven’t put in any specifics
for either codec besides from maybe their names.
>>
>>   v=0
>>   o=- 0 0 IN IP4 127.0.0.1
>>   s=-
>>   c=IN IP4 127.0.0.1
>>   t=0 0
>>   m=audio 5006 RTP/AVP 109
>>   a=rtpmap:109 opus/48000/2
>>   m=video 5004 RTP/AVP 120
>>   a=rtpmap:120 VP8/9
>
> You can try reading the rfc’s and filling in the extra codec info it
needs, but if you’re the one setting up the session, why not just use a
simpler protocol? (I’m assuming the localhost is just a stand-in)
>
>> However, the issue persists when trying to record Opus audio; it is just
>> not possible to play it back, supposedly due to missing headers (but
>> then how is it possible to record incoming RTP Opus audio?)
>
> Like, you told ffmpeg it was going to be 48khz, 2 chs it believed
your copy codec and just dumped everything into a file. But when you
open up the file, does it really have two channels? is the audio
interleaved? Nobody knows.
>

I just want to have a working command to record RTP media received from
an RTP router, in this case it is MediaSoup, which receives a WebRTC
stream from Chrome, then redistributes it as plain RTP. I first tried
using plain "rtp://" URL, but the ffmpeg command itself told me that an
SDP input file would be needed to use dynamic Payload Types (which is
totally sensible).

This is a standard SDP Offer as generated by Chrome to create a WebRTC
session:

   v=0
   o=- 6323209318560568134 2 IN IP4 127.0.0.1
   s=-
   t=0 0
   a=group:BUNDLE 0 1
   a=msid-semantic: WMS XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
   m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
   c=IN IP4 0.0.0.0
   a=rtcp:9 IN IP4 0.0.0.0
   a=ice-ufrag:Ao6s
   a=ice-pwd:h3NoIquJSZ8L/yQUSy7nme5Y
   a=ice-options:trickle
   a=fingerprint:sha-256
   
21:D2:14:1E:49:95:70:0B:A6:0B:93:5C:E5:A9:35:B0:8A:48:3C:95:75:06:62:74:5A:2B:99:29:0F:D0:3E:FB
   a=setup:actpass
   a=mid:0
   a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
   a=extmap:2
   http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
   a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
   a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
   a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
   a=sendrecv
   a=msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
   816c42db-7d97-4053-86a0-bae0e5010985
   a=rtcp-mux
   a=rtpmap:111 opus/48000/2
   a=rtcp-fb:111 transport-cc
   a=fmtp:111 minptime=10;useinbandfec=1
   a=rtpmap:103 ISAC/16000
   a=rtpmap:104 ISAC/32000
   a=rtpmap:9 G722/8000
   a=rtpmap:0 PCMU/8000
   a=rtpmap:8 PCMA/8000
   a=rtpmap:106 CN/32000
   a=rtpmap:105 CN/16000
   a=rtpmap:13 CN/8000
   a=rtpmap:110 telephone-event/48000
   a=rtpmap:112 telephone-event/32000
   a=rtpmap:113 telephone-event/16000
   a=rtpmap:126 telephone-event/8000
   a=ssrc:3169356447 cname:XZAw22pKY7Y9bCph
   a=ssrc:3169356447 msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
   816c42db-7d97-4053-86a0-bae0e5010985
   a=ssrc:3169356447 mslabel:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
   a=ssrc:3169356447 label:816c42db-7d97-4053-86a0-bae0e5010985
   m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125
   107 108 109 124 120 123
   c=IN IP4 0.0.0.0
   a=rtcp:9 IN IP4 0.0.0.0
   a=ice-ufrag:Ao6s
   a=ice-pwd:h3NoIquJSZ8L/yQUSy7nme5Y
   a=ice-options:trickle
   a=fingerprint:sha-256
   
21:D2:14:1E:49:95:70:0B:A6:0B:93:5C:E5:A9:35:B0:8A:48:3C:95:75:06:62:74:5A:2B:99:29:0F:D0:3E:FB
   a=setup:actpass
   a=mid:1
   a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
   a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
   a=extmap:12 urn:3gpp:video-orientation
   a=extmap:2
   http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
   a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
   a=extmap:6
   http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
   a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
   a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
   a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
   a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
   a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
   a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
   a=sendrecv
   a=msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
   0c05ddde-b52e-4776-97f0-755ded96e15d
   a=rtcp-mux
   a=rtcp-rsize
   a=rtpmap:96 VP8/9
   a=rtcp-fb:96 goog-remb
   a=rtcp-fb:96 transport-cc
   a=rtcp-fb:96 ccm fir
   a=rtcp-fb:96 nack
   a=rtcp-fb:96 nack pli
   a=rtpmap:97 rtx/9
   a=fmtp:97 apt=96
   a=rtpmap:98 VP9/9
   a=rtcp-fb:98 goog-remb
   a=rtcp-fb:98 transport-cc
   a=rtcp-fb:98 ccm fir
   a=rtcp-fb:98 nack
   a=rtcp-fb:98 nack pli
   a=fmtp:98 profile-id=0
   a=rtpmap:99 rtx/9
   a=fmtp:99 apt=98
   a=rtpmap:100 VP9/9
   a=rtcp-fb:100 goog-remb
   a=rtcp-fb:

Re: [FFmpeg-user] Unable to record VP8+Opus from RTP+SDP

2019-08-13 Thread Ted Park 
> Some updates: recording video actually works fine in both cases of only
> video and video+audio. A mistake in the video router was channeling
> audio frames to what should have been the video stream.

I think every instance of this working is because ffmpeg and the media players 
you tried are good at making educated guesses. 

The RTP payload format spec for opus is a proposed standard (rfc7587 
) and the one for vp8 (rfc7741 
) uses a profile of RTP that SDP 
hasn’t been updated to implement yet.

So I’m not sure how ffmpeg parses SDP files or if it uses a library but when 
you step back and take a look, you haven’t put in any specifics for either 
codec besides from maybe their names.

>   v=0
>   o=- 0 0 IN IP4 127.0.0.1
>   s=-
>   c=IN IP4 127.0.0.1
>   t=0 0
>   m=audio 5006 RTP/AVP 109
>   a=rtpmap:109 opus/48000/2
>   m=video 5004 RTP/AVP 120
>   a=rtpmap:120 VP8/9

You can try reading the rfc’s and filling in the extra codec info it needs, but 
if you’re the one setting up the session, why not just use a simpler protocol? 
(I’m assuming the localhost is just a stand-in)

> However, the issue persists when trying to record Opus audio; it is just
> not possible to play it back, supposedly due to missing headers (but
> then how is it possible to record incoming RTP Opus audio?)

Like, you told ffmpeg it was going to be 48khz, 2 chs it believed your copy 
codec and just dumped everything into a file. But when you open up the file, 
does it really have two channels? is the audio interleaved? Nobody knows.

___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-user] Unable to record VP8+Opus from RTP+SDP

2019-08-13 Thread Juan Navarro 

Some updates: recording video actually works fine in both cases of only
video and video+audio. A mistake in the video router was channeling
audio frames to what should have been the video stream.

However, the issue persists when trying to record Opus audio; it is just
not possible to play it back, supposedly due to missing headers (but
then how is it possible to record incoming RTP Opus audio?)

Cheers
___
ffmpeg-user mailing list
ffmpeg-user@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".