Hello, Working on video (no audio) playback library, I am only seeing frame rates of 15 fps or less. My code started based on the Dranger examples, but has since been updated to the latest APIs (such that I compile with no depreciated warnings.) I am using Visual Studio 2013 Community.
I have a 160x40 29.97 fps h264 clip with timecodes, and it plays back at exactly half speed, consuming very little cpu. An rtsp h264 stream from an IP camera plays back between 10 & 11 fps, while the latest full HD resolution Rogue One trailer plays back at 15 fps (both only consuming < 20% CPU.) My code does not use SDL, so I am using neosmart cross platform pevents (https://github.com/neosmart/pevents). I am using one thread to read packets, decode them to yuv & convert to RGB. OpenGL does the display in the main thread. Does achieving higher frame rates require the packet reading and packet decoding in separate threads? I thought the 160x40 file would decompress fast enough that I’d need to add necessary timing logic to delay the frames, but that is not the case. It plays Here’s the general logic of my playback setup, abbreviated for clarity, and the packet reading is below that: ------------------------------------------------------------------------------------------ avformat_open_input( &mp_format_context, [path to file or stream], NULL, NULL ); avformat_find_stream_info( mp_format_context, NULL ); m_video_stream = -1; for (ce_uint i = 0; i < (ce_uint)mp_format_context->nb_streams; i++) { if (mp_format_context->streams[i]->codecpar->codec_type==AVMEDIA_TYPE_VIDEO) { m_video_stream = i; break; } } if (m_video_stream == -1) return false; AVCodecParameters* codecPars = mp_format_context->streams[m_video_stream]->codecpar; AVCodec* pCodec = avcodec_find_decoder( codecPars->codec_id ); if (!pCodec) return false; mp_codec_context = avcodec_alloc_context3( pCodec ); if (avcodec_parameters_to_context( mp_codec_context, codecPars ) != 0) return false; mp_codec_context->thread_count = 0; // tried 4, 8… makes no difference AVDictionary* codec_options = NULL; av_dict_set( &codec_options, "threads", "auto", 0 ); // makes no difference, tried 4 & 8 too if (m_stream_type == 2) av_dict_set( &codec_options, "rtsp_transport", "tcp", 0 ); // needed for ffplay to work with my ip cam, makes no difference here... if (avcodec_open2( mp_codec_context, pCodec, &codec_options ) < 0) return false; mp_decompressed_frame = av_frame_alloc(); mp_display_frame = av_frame_alloc(); if (mp_decompressed_frame == NULL || mp_display_frame == NULL) return false; ce_int numBytes = av_image_get_buffer_size( AV_PIX_FMT_RGBA, mp_codec_context->width, mp_codec_context->height, 1 ); mp_display_buffer = (uint8_t *)av_malloc( numBytes*sizeof(uint8_t) ); av_image_fill_arrays( mp_display_frame->data, mp_display_frame->linesize, mp_display_buffer, AV_PIX_FMT_RGBA, mp_codec_context->width, mp_codec_context->height, 1 ); mp_sws_context = sws_getContext( mp_codec_context->width, mp_codec_context->height, mp_codec_context->pix_fmt, mp_codec_context->width, mp_codec_context->height, AV_PIX_FMT_RGBA, SWS_BILINEAR, NULL, NULL, NULL ); ————————————————————————————————————————— In my packet reading thread, the logic is simple: —————————————————————————————————— uint64_t milliseconds = 1; bool media_has_ended = false; bool camera_has_terminated = false; ce_uint packet_errors = 0; while (true) { // this basically allows the thread to idle (sleep) but it will // wake up if the events are signaled dwWaitRes = neosmart::WaitForEvent(m_stop_local_spin_event, milliseconds); if (0 == dwWaitRes) // stop event break; else if (m_is_playing && !m_paused) { // I’ve got 16 packet buffers so I can look at their data, but only use one at a time: ce_uint curr_packet_index = m_total_video_packets % CE_LIBAV_NUMVIDPKTS; AVPacket* curr_packet = &m_packet[ curr_packet_index ]; m_total_video_packets++; int stream_status = av_read_frame(mp_format_context, curr_packet); if (stream_status < 0) { if (m_stream_type == 0) { // end of media file // stream_types: 0=Media, 1=USB, 2=IP media_has_ended = true; break; } else { // camera stream has terminated unexpectedly camera_has_terminated = true; break; } } else { // stream delivered a packet fine if (curr_packet->stream_index == m_video_stream) { // Decode video frame(s): int ret = avcodec_send_packet( mp_codec_context, curr_packet ); if (ret < 0) { if (ret == AVERROR_EOF && m_stream_type == 0) { media_has_ended = true; break; } else packet_errors++; } else { while (!ret) { ret = avcodec_receive_frame( mp_codec_context, mp_decompressed_frame ); if (!ret) handle_new_frame( curr_packet, curr_packet_index ); } } av_packet_unref(curr_packet); } // end stream delivered a packet } // end if (m_is_playing && !m_paused) } // end while (true) if (media_has_ended) { if (mp_stream_ended_callback) (mp_stream_ended_callback)(m_frames_received, mp_stream_ended_object); if (camera_has_terminated) if (mp_term_callback) (mp_term_callback)(mp_term_object); —————————————————————————————————— Sincerely, -Blake Senftner Mad Computer Scientist p.s. My ffmpeg libav* 3.2 libs are, downloaded pre-built from Zeranoe early November, my versions are: libavutil 55. 34.100 / 55. 34.100 libavcodec 57. 64.100 / 57. 64.100 libavformat 57. 56.100 / 57. 56.100 libavdevice 57. 1.100 / 57. 1.100 libavfilter 6. 65.100 / 6. 65.100 libswscale 4. 2.100 / 4. 2.100 libswresample 2. 3.100 / 2. 3.100 libpostproc 54. 1.100 / 54. 1.100 _______________________________________________ Libav-user mailing list [email protected] http://ffmpeg.org/mailman/listinfo/libav-user
