Greetings -
A tape I recently captured/encoded and further experiments have
provided some valuable insight that may be of interest and value
to others.
In a nutshell: the importance of processing the dark areas has GREATLY
been underestimated! Specifically 'desaturating the lows' (reducing
chroma information in dark areas) has a much larger impact on the
visual quality _and_ bitrate than previously thought.
The movie I worked on was 144 minutes and getting that on a single
DVD would have meant a maximum bitrate of about 4000Kb/s (or less) OR
using a dual layer disc (which I have and though would be necessary)
The movie was also quite dimly lit in most scenes which usually ends
up causing those ugly splotches/blocks.
My brother using (inferior IMHO) windoze tools encoded the DV
data (which I obtained via a Canopus ADVC300) and managed to squeeze
the movie onto a DVD with only 50MB or so free space on the media.
Since I had the data on the Mac I fired up FinalCutPro and 1) color
corrected the slight colorcast (using a calibrated Sony PVM-14L2
monitor), 2) applied a 50% desaturation below 10 (Y' of 16 is 0 on
the scopes, so 10 would be Y' of 26) and 3) added the 'noise killer'
filter (from http://www.cgm-online.com/eiperle/cgm_e.html) at a mild
threshold. Oh, the contrast/dynamicrange was also increased but the
first 2 adjustments are the important ones. Then the data was
exported back out to a DV file and run thru the pipeline:
----------------
#!/bin/sh
N=st001
smil2yuv -i 2 ~/$N.dv | \
y4mspatialfilter -L 5,0.85,5,0.85 -C 4,0.85,4,0.85 | \
y4mshift -n -4 -N 2 -b 12,4,696,472 | \
yuvmedianfilter -t 0 | \
y4mdenoise -p 0 -t 3 -z 2 | \
y4mscaler -v0 -O chromass=420_mpeg2 -O sar=src -O size=704x480 | \
yuvmedianfilter -f -r 1 -R 1 -w 12 | \
y4munsharp -L 3,0.4,4 | \
bfr -b 20m -T 70 | \
mpeg2enc -M 2 -f 8 -b 8500 -c -G 15 -D 10 -E -10 -q 3 -4 1 -2 1 -K tmpgenc
-o $N.m2v
------------------------
The resulting 144 minute .m2v file ended up being only
2716265216 7 Apr 18:35 st001.m2v
average bitrate was 2511Kb/s using -q 3! What's better is that the
dark areas look great. Since the bitrate came out so much lower than
expected I'm doing the encoding run again but without the two
y4mspatialfilter and yuvmedianfilter filters. The 2nd yuvmedianfilter
use did seem to soften the image a little more than I prefer. The
re-coding is about 60% done and the average bitrate (without the
spatial and median programs) crept up to 3040Kb/s so the movie will
easily fit.
So, that got me to thinking about how to show/demonstrate the
effect of "manipulating the lows" and then the thought hit me - there
was about a 1.5 minute black scene at the beginning of the movie
while the orchestral opening theme played.
I edited the 1m40s black section into two files "WithoutCorrection.dv"
(no filters at all) and "WithCorrection.dv" ("desaturate lows" filter
applied).
Then the 'y4mhist' (recently added to mjpegtools) program was used
to get a histogram of the clips. The simple script was used with
the appropriate input and output filenames substituted. The 'y4mshift'
command was used to get rid of the VCR trash in the edges of the
frame:
-------------------
#!/bin/sh
smil2yuv -i 2 WithCorrection.dv | \
y4mshift -n -4 -N 2 -b 12,4,696,472 | \
y4mhist
-------------------
First let's look at the "Without" (no special processing of the black
areas) histogram (the extremes have been edited out since they were
all 0) - see the Attachment labeled "Without Filter Histogram".
To the knowing reader several things stand out. FIRST is that the
black level is a bit high - it's fairly obvious that black is at 17 or
18 instead of 16 on this tape so lowering the black level slightly
would probably be a good idea. SECOND, there's a slight colorcast,
U is centered at 127 instead of 128 and V centered at 126 and 127.
THIRD, for 100 seconds of what looks black to the EYE there's
a LOT of color information there that the encoder WILL see (more on
that later - you'll see that the encoder does INDEED see the color
and waste MANY bits encoding it).
So one thing that would be done to the data is color correct the
blacks - that's the left color wheel in FCP's 3way color correction
filter but let's leave color correction for another day ;)
Next let's look at the "With" (desaturate lows below Y' of 26 by 50%)
histogram - see the attachment "With Filter Histogram".
The Y has been shifted down slightly and is a lot closer to 16 for
black. I'm not exactly sure why this happened, but it's a Good Thing.
The 17,18,19 values have basically been added in to the 16 value.
NOTE the U and V center points of 128 though and how much higher
they are than before! Most of the dark pixels had their U and V
values moved towards 128
The result is that blacks look black instead of dark grey (or dark
purplish or greenish). It also means that more pixels are identical
or closer in value. The latter you might expect to result in more
efficient encoding (lower bitrate) and you'd be correct. However, the
magnitude of the increase in efficiency will surprise you!
Now on to the encoding tests :)
To keep things simple and consistent the minimal script:
---------------
#!/bin/sh
smil2yuv -i 2 WithoutCorrection.dv | \
y4mshift -n -4 -N 2 -b 12,4,696,472 | \
y4mscaler -v0 -O chromass=420mpeg2 | \
mpeg2enc -f 8 -q 3 -o Without.m2v
---------------
was used (changing the input/output files names of course ;))
Encoding ~100 seconds of what LOOKS to be BLACK you would expect to
produce a very small file but appearances can be deceiving - to the
eye the data looks black but let's see what the encoder makes of
the data shall we?
First the Without (no filtering) mplex report:
INFO: [mplex] VIDEO_STATISTICS: e0
INFO: [mplex] Video Stream length: 27204973 bytes
INFO: [mplex] Sequence headers: 199
INFO: [mplex] Sequence ends : 1
INFO: [mplex] No. Pictures : 2976
INFO: [mplex] No. Groups : 199
INFO: [mplex] No. I Frames : 199 avg. size 10929 bytes
INFO: [mplex] No. P Frames : 2777 avg. size 9013 bytes
INFO: [mplex] No. B Frames : 0 avg. size 0 bytes
INFO: [mplex] Average bit-rate : 2191600 bits/sec
INFO: [mplex] Peak bit-rate : 2420400 bits/sec
NOTE the file length of ~27MB: 27204973 7 Apr 20:45 Without.m2v
Wow - average bitrate of 2191Kb/s for BLACK!!!
Ok, now on to the Withcorrection run:
INFO: [mplex] VIDEO_STATISTICS: e0
INFO: [mplex] Video Stream length: 3120492 bytes
INFO: [mplex] Sequence headers: 199
INFO: [mplex] Sequence ends : 1
INFO: [mplex] No. Pictures : 2976
INFO: [mplex] No. Groups : 199
INFO: [mplex] No. I Frames : 199 avg. size 7735 bytes
INFO: [mplex] No. P Frames : 2777 avg. size 569 bytes
INFO: [mplex] No. B Frames : 0 avg. size 0 bytes
INFO: [mplex] Average bit-rate : 251200 bits/sec
INFO: [mplex] Peak bit-rate : 461600 bits/sec
NOTE the file length: 3120492 7 Apr 20:32 With.m2v
27204973 7 Apr 20:45 Without.m2v
3120492 7 Apr 20:32 With.m2v
Did you expect a 9X difference by just a SIMPLE tweek to the 'lows'? :)
To the EYE there's no change in the visual quality of the 'black'
but there's a *huge* drop in the bitrate. Look at how the P frames
went from 9013 bytes to 569 bytes!
Other options to the encoder (-E) will of course change the numbers
(adding "-E -10" lowered the bitrate for both files but the ratio of
file sizes stayed the same!). Adding denoising and so on will also
change the results but the "cleaned up" version will still be lower.
Since the chroma is such a small portion of the total amount of data
(chroma being subsampled down to 4:1:1 or 4:2:0) the tendency has been
to think that noise in the chroma didn't affect the bitrate or quality
to any great extent.
I submit that this is not true and that 'chroma impurities/noise'
have a larger effect than previously believed and in the darker areas
the effect is VERY large both on bitrate and quality of final encoded
data.
The above were done with an extended "completely black scene" but the
results are applicable to movies as well: think about a video with
many dark scenes...
The End
Cheers,
Steven Schultz
Y 0 0
Y 1 0
Y 2 0
Y 3 0
Y 4 0
Y 5 0
Y 6 0
Y 7 0
Y 8 0
Y 9 0
Y 10 0
Y 11 0
Y 12 4
Y 13 50
Y 14 456
Y 15 9090
Y 16 119172195
Y 17 484955690
Y 18 416123632
Y 19 4474550
Y 20 3668921
Y 21 97583
Y 22 1340
Y 23 725
Y 24 496
Y 25 301
Y 26 177
Y 27 122
Y 28 79
Y 29 77
Y 30 66
Y 31 24
Y 32 5
Y 33 5
Y 34 3
Y 35 2
Y 36 3
Y 37 0
Y 38 2
Y 39 1
Y 40 0
U 120 0
U 121 0
U 122 0
U 123 0
U 124 68
U 125 47335
U 126 1885013
U 127 229322193
U 128 25810730
U 129 25631
U 130 12419
U 131 8334
U 132 6453
U 133 4058
U 134 2286
U 135 1103
U 136 525
U 137 187
U 138 43
U 139 18
U 140 2
U 141 1
U 142 0
U 143 1
U 144 0
V 120 0
V 121 0
V 122 0
V 123 8
V 124 3374
V 125 1382101
V 126 100605506
V 127 126352344
V 128 28538356
V 129 217297
V 130 26870
V 131 527
V 132 17
V 133 0
V 134 0
V 135 0
V 136 0
V 137 0
V 138 0
V 139 0
V 140 0
V 141 0
V 142 0
V 143 0
V 144 0
Y 0 0
Y 1 0
Y 2 0
Y 3 0
Y 4 0
Y 5 0
Y 6 0
Y 7 0
Y 8 0
Y 9 0
Y 10 0
Y 11 0
Y 12 8
Y 13 279
Y 14 2158
Y 15 38136
Y 16 1015028709
Y 17 11437147
Y 18 1990996
Y 19 3992
Y 20 1326
Y 21 822
Y 22 685
Y 23 466
Y 24 300
Y 25 185
Y 26 126
Y 27 89
Y 28 71
Y 29 53
Y 30 20
Y 31 15
Y 32 6
Y 33 3
Y 34 1
Y 35 2
Y 36 3
Y 37 1
Y 38 0
Y 39 1
Y 40 0
U 120 0
U 121 0
U 122 0
U 123 0
U 124 0
U 125 2
U 126 3522
U 127 3333409
U 128 253748054
U 129 26316
U 130 11853
U 131 2481
U 132 675
U 133 79
U 134 9
U 135 0
U 136 0
U 137 0
U 138 0
U 139 0
U 140 0
V 120 0
V 121 0
V 122 0
V 123 0
V 124 0
V 125 213
V 126 525890
V 127 2874201
V 128 253048788
V 129 674985
V 130 2250
V 131 73
V 132 0
V 133 0
V 134 0
V 135 0
V 136 0
V 137 0
V 138 0
V 139 0
V 140 0
