Re: [FFmpeg-trac] #6975(ffplay:closed): Recent change to sync video output to screen refresh conflicts with -noframedrop

[FFmpeg]

#6975: Recent change to sync video output to screen refresh conflicts with
-noframedrop
+---
 Reporter:  Misaki  |Owner:
 Type:  defect  |   Status:  closed
 Priority:  normal  |Component:  ffplay
  Version:  git-master  |   Resolution:  invalid
 Keywords:  |   Blocked By:
 Blocking:  |  Reproduced by developer:  0
Analyzed by developer:  0   |
+---

Comment (by Misaki):

 Another update to this closed ticket that, like probably every other issue
 I've reported that wasn't already fixed (manual page was already fixed),
 won't get fixed!

 Setting vblank_mode=0 does not entirely fix everything. I still can't play
 1080p 60fps video in realtime, with -noframedrop.

 Why this won't get fixed:
 1) It only affects video playback at some border of performance. People
 with recent computers might be able to play 3840x2160 video at 60 fps
 without running into whatever issue I have.
 2) It only affects -noframedrop, which is not the default.

 A random, existing 1080p, 60fps h.264 video with High profile at 3200 kb/s
 performs slightly better than a completely black video at 28 kb/s.

 (
 The random high bitrate video took 11.15 seconds to play with option '-t
 10', ending with A-V at 0.53. User CPU of 9.7 seconds, system CPU 2.3.

 The black video took 11.50 seconds to run, ending with M-V at 1.0, user
 CPU 6.2, sys 2.3.

 When tested again with CPUs at performance setting = locked to highest
 frequency, the high bitrate video was still ahead at 10.88 vs 11.08
 seconds, with A-V at 0.285 vs 0.635 for the black video. Black video user
 CPU down to 4.95.
 )

 In this case, the CPU can't be the limiting factor. It doesn't even
 saturate one CPU core.

 Tested 3840x2160p video at 15 fps. This ran without the slowdown. The
 pixels per second is the same as the 1920x1080p 60fps video.

 3840x2160p at 30fps does have slowdown. With CPUs locked to high
 performance,
 {{{
 10.76 M-V:  0.813 fd=   0 aq=0KB vq=0KB sq=0B f=0/0

 real0m11.339s
 user0m10.104s
 sys 0m3.948s
 }}}

 Completely black VP9 video at 1920x1080p 60 fps decodes slightly easier,
 with user=4.4 and sys=2.1, but is still delayed ending at M-V=0.356 in
 10.8 seconds.

 All the above results are with vblank_mode=0.

 Tested mpeg1video, mpeg2video, ffv1, and mpeg4. All of them had slowdown.
 mpeg4 was the fastest with only 0.08 seconds late after 10 seconds. For
 those tested, not setting vblank_mode=0 mode the slowdown worse; for
 mpeg4, M-V only went up to 0.238. (Also, all of these codecs had bitrate
 of 700~1500 kbps for completely black video.)

 There does seem to be something significant about the framerate, with the
 difference in 2160p ('4K') at 15 fps vs 1080p at 60 fps.

 3840x4320 at 30 fps played back in 18~20 sec after a slow first trial,
 with user CPU around 19.5~20 and sys around 7.9. Idle CPU as seen in top
 around 15% for this, compared to 27% idle for 3840x2160 30 fps and 55~60%
 idle for 1920x1080 60fps.

 This seems like it could be consistent with some kind of ~~rendering~~
 process referenced by ffplay being single-threaded. Correction, it takes
 ffmpeg 14 userspace seconds to decode the 4Kx4K video with '-f null -'. If
 ffplay is being limited by a single-threaded action, it would have to be
 decoding, though I'm not sure if that makes any sense if the decoding is
 done by the ffmpeg or libav libraries, and we assume that it isn't a
 change in ffmpeg/ffplay that's responsible for this issue.

 Even if there is a single-thread limitation, it doesn't explain the
 slowdown with vblank_mode=0 and no CPUs saturated. In fact, having
 randomly noticed that you can make 'top' show CPUs separately, load is
 distributed evenly between them, but this might not rule out a single-
 threading cause.

 I tried to test how many frames were being dropped with -framedrop
 (default), but first attempt failed. Adding {{{-vf
 
"drawtext='fontcolor=white:fontsize=96:x=(W-tw)/2+4:y=(H-th)/2+4:text=%{n}:alpha=0.5',boxblur,drawtext='fontcolor=white:fontsize=96:x=(W-tw)/2:y=(H-th)/2:text=%{n}'"}}}
 seemed to be going well, with no obvious skipping and no M-V difference,
 but the numbers were actually going up much too slow and it was only
 around 190 when the video ended for VP9 video, instead of 600.

 So I encode VP9 again at 15 fps encoding speed, at fastest speed of
 '-speed 5 -quality realtime', and default bitrate. Expect VP9 to show more
 obvious frame dropping than H.264, unless maybe B-frames are disabled.

 Result: no obvious frame dropping from -framedrop. Without setting
 vblank_mode=0 but with -noframedrop, end delay is 0.972. With both set,
 end delay is 0.692. time is around 4.7 user, 2 sys, 

Re: [FFmpeg-trac] #6975(ffplay:closed): Recent change to sync video output to screen refresh conflicts with -noframedrop

[FFmpeg]

#6975: Recent change to sync video output to screen refresh conflicts with
-noframedrop
+---
 Reporter:  Misaki  |Owner:
 Type:  defect  |   Status:  closed
 Priority:  normal  |Component:  ffplay
  Version:  git-master  |   Resolution:  invalid
 Keywords:  |   Blocked By:
 Blocking:  |  Reproduced by developer:  0
Analyzed by developer:  0   |
+---

Comment (by Misaki):

 Another update to this closed ticket that, like probably every other issue
 I've reported that wasn't already fixed (manual page was already fixed),
 won't get fixed!

 Setting vblank_mode=0 does not entirely fix everything. I still can't play
 1080p 60fps video in realtime, with -noframedrop.

 Why this won't get fixed:
 1) It only affects video playback at some border of performance. People
 with recent computers might be able to play 3840x2160 video at 60 fps
 without running into whatever issue I have.
 2) It only affects -noframedrop, which is not the default.

 A random, existing 1080p, 60fps h.264 video with High profile at 3200 kb/s
 performs slightly better than a completely black video at 28 kb/s.

 (
 The random high bitrate video took 11.15 seconds to play with option '-t
 10', ending with A-V at 0.53. User CPU of 9.7 seconds, system CPU 2.3.

 The black video took 11.50 seconds to run, ending with M-V at 1.0, user
 CPU 6.2, sys 2.3.

 When tested again with CPUs at performance setting = locked to highest
 frequency, the high bitrate video was still ahead at 10.88 vs 11.08
 seconds, with A-V at 0.285 vs 0.635 for the black video. Black video user
 CPU down to 4.95.
 )

 In this case, the CPU can't be the limiting factor. It doesn't even
 saturate one CPU core.

 Tested 3840x2160p video at 15 fps. This ran without the slowdown. The
 pixels per second is the same as the 1920x1080p 60fps video.

 3840x2160p at 30fps does have slowdown. With CPUs locked to high
 performance,
 {{{
 10.76 M-V:  0.813 fd=   0 aq=0KB vq=0KB sq=0B f=0/0

 real0m11.339s
 user0m10.104s
 sys 0m3.948s
 }}}

 Completely black VP9 video at 1920x1080p 60 fps decodes slightly easier,
 with user=4.4 and sys=2.1, but is still delayed ending at M-V=0.356 in
 10.8 seconds.

 All the above results are with vblank_mode=0.

 Tested mpeg1video, mpeg2video, ffv1, and mpeg4. All of them had slowdown.
 mpeg4 was the fastest with only 0.08 seconds late after 10 seconds. For
 those tested, not setting vblank_mode=0 mode the slowdown worse; for
 mpeg4, M-V only went up to 0.238. (Also, all of these codecs had bitrate
 of 700~1500 kbps for completely black video.)

 There does seem to be something significant about the framerate, with the
 difference in 2160p ('4K') at 15 fps vs 1080p at 60 fps.

 3840x4320 at 30 fps played back in 18~20 sec after a slow first trial,
 with user CPU around 19.5~20 and sys around 7.9. Idle CPU as seen in top
 around 15% for this, compared to 27% idle for 3840x2160 30 fps and 55~60%
 idle for 1920x1080 60fps.

 This seems like it could be consistent with some kind of ~~rendering~~
 process referenced by ffplay being single-threaded. Correction, it takes
 ffmpeg 14 userspace seconds to decode the 4Kx4K video with '-f null -'. If
 ffplay is being limited by a single-threaded action, it would have to be
 decoding, though I'm not sure if that makes any sense if the decoding is
 done by the ffmpeg or libav libraries, and we assume that it isn't a
 change in ffmpeg/ffplay that's responsible for this issue.

 Even if there is a single-thread limitation, it doesn't explain the
 slowdown with vblank_mode=0 and no CPUs saturated. In fact, having
 randomly noticed that you can make 'top' show CPUs separately, load is
 distributed evenly between them, but this might not rule out a single-
 threading cause.

 I tried to test how many frames were being dropped with -framedrop
 (default), but first attempt failed. Adding {{{-vf
 
"drawtext='fontcolor=white:fontsize=96:x=(W-tw)/2+4:y=(H-th)/2+4:text=%{n}:alpha=0.5',boxblur,drawtext='fontcolor=white:fontsize=96:x=(W-tw)/2:y=(H-th)/2:text=%{n}'"}}}
 seemed to be going well, with no obvious skipping and no M-V difference,
 but the numbers were actually going up much too slow and it was only
 around 190 when the video ended for VP9 video, instead of 600.

 So I encode VP9 again at 15 fps encoding speed, at fastest speed of
 '-speed 5 -quality realtime', and default bitrate. Expect VP9 to show more
 obvious frame dropping than H.264, unless maybe B-frames are disabled.

 Result: no obvious frame dropping from -framedrop. Without setting
 vblank_mode=0 but with -noframedrop, end delay is 0.972. With both set,
 end delay is 0.692. time is around 4.7 user, 2 sys, 

Re: [FFmpeg-trac] #6975(ffplay:closed): Recent change to sync video output to screen refresh conflicts with -noframedrop

[FFmpeg]

#6975: Recent change to sync video output to screen refresh conflicts with
-noframedrop
+---
 Reporter:  Misaki  |Owner:
 Type:  defect  |   Status:  closed
 Priority:  normal  |Component:  ffplay
  Version:  git-master  |   Resolution:  invalid
 Keywords:  |   Blocked By:
 Blocking:  |  Reproduced by developer:  0
Analyzed by developer:  0   |
+---

Comment (by Misaki):

 I used to able to play 1080p 60fps video with ffplay -noframedrop. Then I
 upgraded from Ubuntu 16.10 to Ubuntu 17.10, and now ffplay will only play
 at about 80% speed with -noframedrop. It won't even play a 720p 60fps file
 without gradually falling behind. It isn't lack of CPU resources.

 Many things changed other than ffplay. Older version of ffplay acting the
 same way (I would have tested myself, but static packages of ffmpeg don't
 include ffplay) isolates the change to elsewhere.

 Maybe this always happens with OpenGL-based rendering, and my system
 changed so that ffplay previously didn't use OpenGL, but now it does, or
 how OpenGL works changed.

 -noframedrop can be helpful when this issue doesn't occur, because dropped
 frames can look worse than changes in video speed. VP9 is particularly
 prone to jumps in decoding complexity upon movement, leading to sections
 that will never display no matter how many times you try it, unless you
 use -noframedrop. H.264 can just drop B-frames, but VP9 falls behind and
 drops whole sections (1+ second) of video.

 It would be possible to prevent this issue from occurring. Ignoring the
 environment variable, or as it seems assuming a different value for the
 default, would be making the judgement call that "the user intends for
 this variable to prevent unseen frames in 3D rendering, and does not
 intend for it to prevent 60fps video from being played correctly in some
 cases."

--
Ticket URL: 
FFmpeg 
FFmpeg issue tracker
___
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-trac

Re: [FFmpeg-trac] #6975(ffplay:closed): Recent change to sync video output to screen refresh conflicts with -noframedrop

[FFmpeg]

#6975: Recent change to sync video output to screen refresh conflicts with
-noframedrop
+---
 Reporter:  Misaki  |Owner:
 Type:  defect  |   Status:  closed
 Priority:  normal  |Component:  ffplay
  Version:  git-master  |   Resolution:  invalid
 Keywords:  |   Blocked By:
 Blocking:  |  Reproduced by developer:  0
Analyzed by developer:  0   |
+---
Changes (by cehoyos):

 * status:  new => closed
 * resolution:   => invalid
 * component:  undetermined => ffplay


Comment:

 Replying to [ticket:6975 Misaki]:

 > {{{
 > $ ffmpeg -filter_complex color=black:r=120 -t 10 black.mp4
 > $ ffplay -noframedrop black.mp4
 > }}}
 >
 > This will show video falling behind as shown with M-V, if your refresh
 rate is less than 120.

 This is the expected behaviour afaict (what else would `noframedrop`
 mean?) that is also reproducible with old versions of FFplay, why do you
 believe there is a bug?
 {{{
 $ time ffplay -noframedrop black.mp4 -autoexit
 ffplay version 0.9, Copyright (c) 2003-2011 the FFmpeg developers
   built on Nov 22 2012 09:29:11 with gcc 4.7.1 20120723 [gcc-4_7-branch
 revision 189773]
   configuration: --enable-gpl
   libavutil51. 32. 0 / 51. 32. 0
   libavcodec   53. 42. 0 / 53. 42. 0
   libavformat  53. 24. 0 / 53. 24. 0
   libavdevice  53.  4. 0 / 53.  4. 0
   libavfilter   2. 53. 0 /  2. 53. 0
   libswscale2.  1. 0 /  2.  1. 0
   libpostproc  51.  2. 0 / 51.  2. 0
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'black.mp4':
   Metadata:
 major_brand : isom
 minor_version   : 512
 compatible_brands: isomiso2avc1mp41
 encoder : Lavf55.48.100
   Duration: 00:00:10.00, start: 0.00, bitrate: 27 kb/s
 Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
 320x240 [SAR 1:1 DAR 4:3], 15 kb/s, 120 fps, 120 tbr, 15360 tbn, 240 tbc
 Metadata:
   handler_name:
9.97 A-V:  0.000 fd=   0 aq=0KB vq=0KB sq=0B f=0/0   0/0

 real0m20.704s
 user0m0.549s
 sys 0m0.126s
 }}}

--
Ticket URL: 
FFmpeg 
FFmpeg issue tracker
___
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-trac