Hi Oskar, On Thu, Aug 25, 2011 at 2:59 AM, Oskar Arvidsson <[email protected]> wrote: > On Wednesday 24 August 2011 18.55.10 Ronald S. Bultje wrote: >> [...] >> >> Dither is all just because of ordered dither and I don't feel it's >> worth looking into. I've been planning to replace it with >> Atkinson-style dither (that's like f-s, which x264 uses) so let's just >> work towards that and be done with it. > > Still, ordered dither would actually work as intended if only using simple > shifting.
Right, but simple shifting changes range. > Also, I think the underlying problem would be the same with any type > of dither as what the current conversion method does is to increase the values > - thus posing larger errors in e.g. error diffusion if not the "byte > duplication" is reversed in one or another way. Correct. I think the correct solution is that for downscaling (e.g. 10-to-8-bit), we need to undo the byte duplication to correct the range again. So if for luma, 0x80 is upped to 0x201 in 8-to-10-bit conversion), then the reverse should be done also. I told you I intend hscale to use 1,0000,0000,0100,00b sum for upscaling to maintain range. The reverse could be done by using a sum of slightly less than 0x4000 (I don't know exactly how much right now, but basically 0x4000 * 0x400 / 0x401, just like the above sum is 0x4000 * 0x401 / 0x400). That would undo the byte duplication and change 0x201 back to 0x80 for luma. Does that sound acceptable? It's also easy to code up. >> > b) Should we distinguish between limited and full range YUV? >> >> Doesn't convertJpegRange already do that? Are you afraid that we >> become "out of range" when we scale between bitdepths with different >> ranges? (I admit I haven't looked much into that yet, i.e. don't know >> how to handle it yet.) > > I'm not too worried about becoming out of range, but the fact that to my > knowledge there's no distinction between limited and full range YUV in swscale > at the moment - having different methods for these would require more work and > would also mean we have to discuss user options etc. I think more and more that hscale coeff sum can help here, plus an additional hscale coeff base. Ronald _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
