Well I wouldn't go so far as to say you are missing something big and
obvious.
Actually, it seems like I am. I wasn't aware until now that the project
provides for standing up rtsp feeds for consumption.
I was assuming that you were using a different library to start up the
rtsp feed, consuming rtp and passing the nals to the library decoding
calls yourself.
My questions were based on this assumption, so if you aren't doing these
things yourself, then disregard.
Ideally, then, you shouldn't have to care about these details, but to
close the loop...
H264 takes frames of a video and turns them into nals, packets of
encoded data that are designed to be passed over a network. In order to
decode a frame from these nals, you need to collect enough of the right
ones to decode a complete frame. Usually this requires an sps, pps, and
some combination of other nals that compose one frame. Usually this is
an idr nal alone (which is a nal that has all the data necessary to
decode one frame) or an idr nal plus delta nals, which is to say, one
nal that decribes a whole frame, and one or more nals who use the idr
nal as a baseline and (presumably) carry only information that differs
from the reference frame.
h264 servers (at the encoder, or at any machinery between it and the
network) can be configured to provide the sps and pps nals in the middle
of the normal bitstream, that is, you ask for data and get an sps, pps,
and then a series of the aforementioned frame nals, with the sps and pps
usually reintroduced periodically in order to allow new clients to begin
decoding frames.
Sometimes though, the sps and pps nals are also/only provided in the
rtsp setup request, and the other frame nals are only provided in the
bitstream. In the case of rtsp, the sps and pps nals are provided in
the sprops-parameter-set attribute of the sdp file, you can find them in
the wireshark paste that you provided in an earlier email. This is
base64 encoded binary data, which you can append immediately to an h264
annex b (add the startcodes yourself), and then begin appending whatever
you get from the bitstream, and you should be able to play that file in
vlc. or the ffplay or equivalent tool provided by the libav project.
I'll save you a headache, the h264 annex b page is ridiculously...
specified. All it calls for is a bitstream that starts with an sps and
pps nal, and has any other nal appended, all with 0x00000001 (so-called
startcode) added at the beginning of each nal. This is all you need to
do to produce such a stream.
To consume one, you need to account for the fact that the startcode may
be 3 or 4 null bytes followed by a 01 byte. I forget when the startcode
is allowed to be 3 bytes vs 4, but since I don't consume these myself, I
just produce 4 byte versions and go about my day. Its something like
"all nals of type x must have a 4 byte startcode, all others may be 3 or
4 bytes."
If you write this stream to a file and name it something.264, you should
be able to play it in vlc. I hear the .264 file extension is looked for
by some players in order to figure out that it is an annex b bitstream.
Joshua Kordani
LSA Autonomy
On 6/10/14 12:17 PM, Manuel Torres wrote:
I have to admit that I barely understood that reply so I assume that I am
missing something big and obvious.
Answers to the questions:
"Are you aware that the sprop-parameter-sets contain the sps and pps nal
packets encoded in base64?"
I think that the answer would be no because I do not know what those are
but I will look it up.
"Is your code looking for the sps and pps nals in the rtp stream itself?"
Hard to say. The library is doing everything for me. I have no clue of what
is going on there. I set up the context, open the input, set up the codec
and grab the frames. Nothing more advanced than that.
"Are you accounting for whether or not the sps and pps are coming in-stream
vs in the sdp?"
I guess I am not. I do not even know how to get the data from the SDP. I
will look it up too but I would appreciate if anybody could tell me.
"If your rtp reception code can produce an h264 annex b file, can vlc play
it?"
I suppose you are referring to the Annex B of the H.264 specification. I
have not read the specification but I have a copy of it so I will check
that annex and try to create that file to test as described.
Regardless of my answers to those questions, I understand from the reply
that the data is there somewhere. I would be really grateful if someone
could tell me where to find it in the library structures and, if it is not
too much to ask, a link to how to parse "the sps and pps nal packets",
whatever those are. Base64 is the only thing I understood from that first
question. Shame on me.
On Tue, Jun 10, 2014 at 5:28 PM, Joshua Kordani <[email protected]> wrote:
Are you aware that the sprop-parameter-sets contain the sps and pps nal
packets encoded in base64? Is your code looking for the sps and pps nals
in the rtp stream itself? Are you accounting for whether or not the sps
and pps are coming in-stream vs in the sdp?
If your rtp reception code can produce an h264 annex b file, can vlc play
it? Making your code create this file will tell you that you have
everything you need in order to pass data to the decoding library. If vlc
can play the h264 annex b that you create, then you know that all you have
left to do is figure out how to pass this data to the library to decode it
yourself, most likely.
Joshua Kordani
LSA Autonomy
On 6/10/14 8:11 AM, Manuel Torres wrote:
Captured with Wireshark:
v=0
o=- 1293840088672 1293840088672 IN IP4 192.168.2.93
s=Live
t=0 0
m=audio 0 RTP/AVP 0
c=IN IP4 192.168.2.93
a=control:rtsp://192.168.2.93/defaultPrimary/mic0/trackID=1
m=video 0 RTP/AVP 96
c=IN IP4 192.168.2.93
a=control:rtsp://192.168.2.93/defaultPrimary/cam0/trackID=1
a=fmtp:96 packetization-mode=0; profile-level-id=42A01E;
sprop-parameter-sets=J01AH42NKBaHt/4AQAA21BgYGQAAAwPoAADDUOhAB08A
AQcau8uNCADp4AAg41d5cE+iwA==,KO48gA==
a=rtpmap:96 H264/90000
a=x-avg-params:96 source-height=1080; source-width=1920
m=application 0 RTP/AVP 98
c=IN IP4 192.168.2.93
a=control:rtsp://192.168.2.93/defaultPrimary/metadata/trackID=1
a=rtpmap:98 vnd.onvif.metadata/90000
On Tue, Jun 10, 2014 at 12:58 PM, Luca Barbato <[email protected]>
wrote:
On 10/06/14 12:44, Manuel Torres wrote:
Would there be any other way to get the missing data? I guess that there
must be a way since VLC and MPlayer open the stream.
Could you please provide the sdp?
lu
_______________________________________________
libav-api mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-api
_______________________________________________
libav-api mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-api
_______________________________________________
libav-api mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-api
_______________________________________________
libav-api mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-api
_______________________________________________
libav-api mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-api
