Hi Sven
I made a test on my side by removing the IDR. Thus, I am now sending to
ffmpeg the following sequence:
[7 8] [1] [1] ... [1]
Huy is right (I was sure about this, but just wanted to check). Without
IDR, ffmpeg can't decode anything. Thus, I think you wil have to find
the IDR type (5), bundle it along with 7(sps) and 8(pps) types and offer
this to ffmpeg. After this, offer each NAL as a whole frame to ffmpeg.
Thus, as Huy said previously, if you are having slices, you will have to
collect them all and give it to ffmpeg. Ig you are having a frame per
NAL, you are in luck, just put the start sequence before each NAL and
feed ffmpeg.
Regards,
Alex
Huy Tran wrote:
Hi, Sven,
On Wed, Sep 15, 2010 at 7:11 PM, Sven Wasmer <[email protected]> wrote:
Hi Alex,
thank you for your help! YES, network streamed h264 decoding is a big
secret on the net, but h264 is far more complex than h263.
Well, I think I can't use "av_open_input_file", because my server and
client use live555 for the rtsp/rtp handling. Thus I get (not
fragmented) nalus by each RTP-Frame and get them into a buffer.
The current procedere is that I get a frame (first SPS, second PPS)
and hand them over to the decoder (error: no frame).
all the other frames are handled the same way.
As I said in my previous post, SPS and PPS must have an IDR slice followed
them, if not you get errors.
Another point is, that I think that "ffplay" works the same way you
described in your code, but ffplay does also not play the streamed
h264 correctly for me (only grey difference pictures are played, might
be missing the IDR frame?). It can on the other hand play the "file"
very well, thus I think, the decoder is not initialized correctly.
FFplay can work with streaming. Do you check this ?
http://www.wu.ece.ufl.edu/projects/wirelessVideo/project/realTimeCoding/
When you pass an IDR slice to avcodec_decode_video, FFmpeg will decode that
frame, and store it for reference.
FFMpeg will not output this IDR until you feed another frame to
avcodec_decode_video.
Hope it helps.
Huy.
Maybe I need to backbuffer some of these RTP-frames and let
av_read_frame work through that package? That would make only sense,
if av_read_frame would write some data to the AVFormatContext, right?
I will try it and come back later.
cheers
Sven
2010/9/15 Alex Grosu <[email protected]>:
Hi Sven
AFAIK, there isn't any other possibility when decoding raw NALs is
wanted.
The correct manner to handle this kind of issue is very vague discussed
over
the net. At least, I couldn't find proper information for this.
Anyway, you said you want to decode streams which are received through
rtsp
protocol. Why don't you use:
if(av_open_input_file(&pFormatCtx, "rtsp_address", NULL, 0, NULL)!=0)
return -1; // Couldn't open file
videoStream=-1;
audioStream=-1;
for(i=0; i<pFormatCtx->nb_streams; i++) {
if(pFormatCtx->streams[i]->codec->codec_type==CODEC_TYPE_VIDEO &&
videoStream < 0) {
videoStream=i;
}
if(pFormatCtx->streams[i]->codec->codec_type==CODEC_TYPE_AUDIO &&
audioStream < 0) {
audioStream=i;
}
}
vCodecCtxp = FormatCtx->streams[videoStream]->codec;
aCodecCtx = pFormatCtx->streams[audioStream]->codec;
After this, use av_read_frame:
while(av_read_frame(pFormatCtx, &packet)>=0) {
// Is this a packet from the video stream?
if(packet.stream_index==videoStream) {
// Decode video frame
avcodec_decode_video(pCodecCtx, pFrame, &frameFinished,
packet.data, packet.size);
if(frameFinished) {
//do something with decoded data
}
}
}
The board I am using sends the streams in two ways:
1. using rtsp protocol (for testing purpose)
2. sending raw NALs in order when they appear (which I have to use)
Using the above code, I can properly decode the rtsp protocol.
Hope this helps
Regards, Alex
Sven Wasmer wrote:
Hey,
I got the same. I want to stream raw h264 streams via RTSP/RTP. In
another post "Re: [libav-user] Decode h264/rtp video -> problem with
SPS/PPS and extradata" I mentioned already my problem.
Did you get any further? I tried to concatenate multiple received
RTP-Frames, always prepending 0x00, 0x00,0x00,0x01. But I still get
"-1" after avcodec_video_decode.
Is there no other way to initialize the decoder correctly for h264
using SDP (SPS,PPS)?
Cheers and thx for your help!
Sven
2010/9/14 Alex Grosu <[email protected]>:
Hello Huy
I will check your information. Great advices, thanks a lot
Regards,
Alex
Nhat Huy wrote:
On Fri, Sep 10, 2010 at 5:38 PM, Alex Grosu <[email protected]> wrote:
Hello
The information you passed will help me later, thanks. In this
moment,
my
application communicates with an embedded product which sends only
one
frame
per NAL in this moment. Later, it will send inceed multiple slices,
and
we
said that we will see what we are going to do then. Is there any
possibility
to find which slice is the start of a frame sequence and which slice
is
the
end of it? In you case for example, is there a possibility to find
the
first
5 from I frame and the last 5 from I frame (or, the first 1 and last
1
from
B frame / P frame) ?
Regards,
Alex
Nhat Huy wrote:
On Thu, Sep 9, 2010 at 9:00 PM, Alex Grosu <[email protected]> wrote:
Hello Huy
Thanks a lot for the answer. I found this solution 2 days ago, but
first
I
thought it was a dirty one. I wanted to check this before posting
here
again, but you took it ahead. Your pointing shows me that in fact
this
is
not a dirty solution, so thanks a lot for support.
So, what I had to do (as Huy suggested), was to couple 7 8 and 5
NAL
types
into only one buffer and to present it to libavcodec (in
avcodec_decode_video2 function).
Thus, using the [ and ] characters to delimit buffer boundaries, I
was
coupling following before send them to libavcodec:
[7 8 5] [1] [1] [1] [1] [7 8 5] [1] [1] [1] and so on.
I don't receive errors anymore. Everything works now
Thank you
Nhat Huy wrote:
On Thu, Sep 2, 2010 at 3:58 PM, Alex Grosu <[email protected]> wrote:
Hello
I am currently decoding h264 streams (with libavcodec) and I am
stuck
with
the logic for SPS (sequence parameter set) and PPS (picture
parameter
set).
The board from which I am receiving the packages is sending a NAL
unit
in
each packet. The sequence for NAL types is:
7(SPS) 8(PPS) 5 1 1 1 1 7 8 5 1 1 1...
I am putting in front of each NAL the start sequence 0x000001 and
sending
this packet to avcodec_decode_video2. When 7 and 8 types are
coming,
I
am
doing the same logic, but avcodec_decode_video2 is returning -1.
After
this,
all the received NALs are decoded and the images are displayed.
Every
time
when SPS and PPS are coming, avcodec_decode_video2 returns -1
(fails).
I
searched all over the net , and I still can't understand how to
fill
up
the
extradata and extradata_size from AVCodecContext used. All I
found
is
this
link:
http://www.mail-archive.com/[email protected]/msg04939.html.
As I saw from here:
"To decode H.264 stream you need to have SPS and PPS NAL units
also
.".
Ok,
I have them, but frankly I don't know how to use them. At first,
I
thought
that avcodec_decode_video2 will "automatically" use them. But
returning
-1,
I don't see how. Also, If I am discarding SPS and PPS from
sending
them
to
libavcodec, nothing is decoded anymore.
Can you please give me a hint?
Thank you a lot !
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
Hi,
I think you should read H.264 bit stream structure and use these
tools
to
analyze a NAL unit of H.264 bitstream.
http://www.codecian.com/
http://tsviatko.jongov.com/index_prj_h264videoesviewer.htm
When FFmpeg decode a NAL unit, if the nal_unit_type equal 7,8 it
will
continue decode to find the IDR nal_unit. In your case, I think
that
FFmpeg
found SPS, PPS but it can not find IDR thus it returns -1.
Hope it helps.
Huy.
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
Hi,
If you coupled H.264 slice as [7 8 5] [1] [1] [1] [1] [7 8 5] [1]
[1]
[1]
and so on, it can work well only in single slice/ frame case.
If you use multiple slice , it goes wrong.
For ex. my sequence has 5 frame I B B B P, and number of slices is 4
per
frame. The NAL_unit_type of bit stream is:
[7 8 5 5 5 5][1 1 1 1][1 1 1 1][1 1 1 1][1 1 1 1]
I use [ and ] to mark the frame boundary.
When you feed avcodec_decode_video2, the input buffer must contains
whole
frame data.
Thus, the problem that you have to calculate exactly the input
buffer
length, and feed it enough data plus some padding if necessary
before
pass
it to avcodec_decode_video2.
Huy.
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
Hi,
There is a syntax element named first_mb_in_slice in slice header. It
describes the position of the first macroblock in that slice. It
equals
0
to
indicate a new frame.
I am not sure that this information can help or not.
I just know that when I call av_read_frame, I can receive a buffer
contained
whole frame data. But I do not know how can it do this ?
Huy.
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
