I'm using libav to import video data into an imagery analysis workbench. The
data is generally mpeg2/mpegts but could be mpeg1 or mpeg4 possibly in an
avi....right now I'm only worried about the mpeg2/mpegts case.
The workbench is a band/frame based workbench. I create a data structure
with the number of rows, cols, and bands/frames. Actual loading of the data
is similar to an OS VM system...a request for a band/frame causes a load of
at least that much data. When I advance one frame at a time forward (the
case when the data is completely loaded into memory) everything seems to be
ok but when I page in on request I occasionally get an assert() in mpeg12.c
I suspect it's because I'm trying to decode a P frame without the
corresponding I frame. Here's some of the code.
The first problem is that I need to know the number of frames before loading
the data. I try checking the AVStream->nb_frames member (is this correct?)
but it is often 0 so I need to calculate the number of frames. I do this by
multipling the minimum frame rate (AVStream->r_frame_rate) by the duration.
I'm guessing this isn't 100% correct but I figure it will at least give me
an upper bound on the number of frames in the video. I also need to
calculate the frame times in seconds for each frame. Again, I try and
estimate this based on the r_frame_rate member. Here's the code for all this
std::vector<double> frameTimes;
boost::rational<int>
minFrameTime(mpFormatCtx->streams[streamId]->r_frame_rate.den,
mpFormatCtx->streams[streamId]->r_frame_rate.num);
boost::rational<int> timeBase(mpFormatCtx->streams[streamId]->time_base.num,
mpFormatCtx->streams[streamId]->time_base.den);
unsigned int frameCnt = mpFormatCtx->streams[streamId]->nb_frames;
if(frameCnt == 0)
{
// need to estimate the number of frames
// this might be too many but it should be sufficient
boost::rational<int> duration = mpFormatCtx->streams[streamId]->duration
* timeBase;
boost::rational<int> minFrameRate = 1 / minFrameTime;
frameCnt = static_cast<unsigned
int>(boost::rational_cast<double>(minFrameRate * duration) + 0.5);
}
frameTimes.reserve(frameCnt);
double frameTime = 0.0;
for(unsigned int frameNum = 0; frameNum < frameCnt; frameNum++)
{
frameTimes.push_back(frameTime);
frameTime += boost::rational_cast<double>(minFrameTime);
}
I'm using the following code to decode the next frame in the stream:
bool getNextFrame()
{
int bytesDecoded = 0;
int frameFinished = 0;
for(bool streamFinished = false; !streamFinished; ) // decode packets
until we have a complete frame
{
while(mBytesRemaining > 0) // decode a packet
{
bytesDecoded = avcodec_decode_video(mpCodecCtx, &mNextFrame,
&frameFinished, mpRawPacketData, mBytesRemaining);
if(bytesDecoded < 0)
{
MessageResource msg("FFMPEG Raster Pager", "temporal",
"c83d182b-4e54-49e5-83fa-beff11f8c99c");
msg->addProperty("detail", "Unable to decode video frame.");
return false;
}
mBytesRemaining -= bytesDecoded;
mpRawPacketData += bytesDecoded;
if(frameFinished != 0)
{
return true;
}
}
do // read the next packet in the stream
{
if(mPacket.data != NULL)
{
av_free_packet(&mPacket);
}
if(av_read_packet(mpFormatCtx, &mPacket) < 0)
{
streamFinished = true;
break;
}
}
while(mPacket.stream_index != mStreamId);
if(!streamFinished)
{
mBytesRemaining = mPacket.size;
mpRawPacketData = mPacket.data;
}
}
// decode the remainder of the last frame
bytesDecoded = avcodec_decode_video(mpCodecCtx, &mNextFrame,
&frameFinished, mpRawPacketData, mBytesRemaining);
if(mPacket.data != NULL)
{
av_free_packet(&mPacket);
}
return frameFinished != 0;
}
Ok, the last interesting bit is the code that gets a decoded frame given a
frame number. First, I'm decoding an entire GOP at a time. I assume I'm at
the beginning of a GOP and I decode AVCodecContext->gop_size frames at a
time. When I need any frame in that GOP I can offset into the memory holding
the decoded frame. If I need something outside that, I decode another GOP.
If the start of the requested GOP is the current location (based on a frame
counter set to the starting frame number of the previously decoded GOP +
gop_size - 1) I just decode. If it's not, I call
av_seek_frame(mpFormatCtx, mStreamId, calculateSeekPosition(gopStartFrame),
0)
int64_t FfmpegRasterPager::calculateSeekPosition(unsigned int frame) const
{
double frameTime =frame *
mpFormatCtx->streams[mStreamId]->r_frame_rate.den /
static_cast<double>(mpFormatCtx->streams[mStreamId]->r_frame_rate.num);
AVRational bq = AV_TIME_BASE_Q;
return av_rescale_q(frameTime * AV_TIME_BASE, bq,
mpFormatCtx->streams[mStreamId]->time_base);
}
I occasionally get an assert when I randomly seek around the video. The
assert is in mpeg12.c in mpeg_decode_slice (called from mpeg_decode_frame
from avcodec_decode_video).
assert(s->field_select[dir][i]==0 || s->field_select[dir][i]==1);
I'm assuming this has something to do with attempting to decode a frame
without a previoud I frame.
I'm thinking I could store the seek time for the start of a GOP when I
decode it. Where can I get the proper seek time after I decode the first
frame in a GOP? Also, is there a better way to handle seeking to a frame
that has not been decoded yet? (i.e. I don't have an exact seek time)
Any help or suggestions are appreciated.
_______________________________________________
libav-user mailing list
[email protected]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user
