2009/6/24 Erik Van Grunderbeeck <[email protected]> > >Thanks again. Just to make sure I'm clear can we run through an example? > > >So I have file with audio and video. > >The audio stream has a start time of 56994 > >The video stream has a start time of 64194 > >Both have a time base of 1/90000 > > >The first audio packet I get has a pts of 56994 > >The first video packet I get has a pts of 64194 > >The next video packets have a pts of: (coming out in this order) > >56994 > >60594 > >74994 <- need packet re-order here, if all these are video? > >67794 <- need packet re-order here, if all these are video? > >71394 > > >I get a frame on the 2nd call to avcodec_decode_video2() (i.e on the > packet > >with a pts of 56994). > > >So given this, should I be playing the first set of decoded audio at the > >same time that I show the first decoded frame? or is there some delay I > need > >to add to one of the output streams. The fact that my lowest pts for video > >is 56994, but the start_time is 64194 is a little confusing to me, just > I'm > >just trying to understand the offsets here. > > Your video decoder pulls data from the video queue. You have the correlate > the timestamp from the video packet to the audio. In your case, video with > pts 64194 needs to be played when audio around that is played. I don't > know > what the timestamp is after audio 56994, but your video prob needs to be > played between audio pts 56994 and the next one (say audio pts 8000). Now, > audio usually comes in a AVPacket that when decompressed is one second or > so > long. So, you need to interpolate your timestamp across the audio buffer > (use channel count, frequency, etc), ffplay does it with is->audio_clock += > (double)data_size / (double)(n * dec->sample_rate); in > audio_decode_frame(). > > > For video we need to build a "wait/sleep" until we around pts 64194 (the > pts > of the video). Code that does that can be found in ffplay, in the function > compute_frame_delay(). Referring to that, since you might be able to step > through it, and see what it does. > > Seeing your video queue, I see timestamps that are non-lineair? > > In a nutshell, > > Add audio and video to separate queues. > > Poll both queues, each in a separate thread (with enough locking to protect > the queues of course). > > Start playing the audio (by feeding the packets you stored in the audio > queue to the decoder, and from there your WAVE/PCM player). > > When you feed to audio to the PCM player, keep its timestamp for the > packet, > call that the clock. Remember that one AVPacket can contain several audio > buffers when decoded, so you may need to interpolate your timestamps over > the size of the buffer. > > Sleep/delay your video thread until the timestamp of the audio packet that > is playing matches your video (bigger then). > > Display your video frame. > > Keep feeding packets to both queues (and threads) until stop. > > > _______________________________________________ > libav-user mailing list > [email protected] > https://lists.mplayerhq.hu/mailman/listinfo/libav-user >
I use audio to sync, pretty much like you say. I have 2 queues, 1 video, 1 audio. The audio is streamed to audio output buffer (hardware). >From the audio outputbuffer I get the played samples and I can calculate the played time in MS. From there I am able to determine the right video frame to display. This works quite well with lot of codecs. Most of them have frame rate information, packets that contain PTS and audio and video starting at the same time. However: I have a WMV with: - VideoStream r_frame_rate = 0/0 - VideoStream time_stamp = 1/1000 - VIdeoStream->start_time == AV_NOPTS_VALUE and the first videopacket pts = 4101 I don't know how to calculate frame rate out of this, so I assume it is 23.976/s. If it is higher: no problem, since can drop frames. If it is lower I have a problem. The WMV is converted with FFMpeg from a .MOV file. The MOV file works excellent in my player. My questions: - How should I deal with this WMV? How do I determine frame rate and start times, when half of the information does not to seem there (or I look at the wrong members) - Is there a generic way to do this, or does anyone have examples of dealing with frame rates and start times? Further: _______________________________________________ libav-user mailing list [email protected] https://lists.mplayerhq.hu/mailman/listinfo/libav-user
