On Mon, Mar 14, 2016 at 2:13 PM, Claudio Freire <klaussfre...@gmail.com> wrote:
> On Sun, Mar 13, 2016 at 10:30 PM, Ganesh Ajjanagadde <gajja...@gmail.com>
> wrote:
>> /**
>> * Calculate rate distortion cost for quantizing with given codebook
>> @@ -105,7 +106,7 @@ static av_always_inline float
>> quantize_and_encode_band_cost_template(
>> curbits += 21;
>> } else {
>> int c = av_clip_uintp2(quant(t, Q, ROUNDING), 13);
>> - quantized = c*cbrtf(c)*IQ;
>> + quantized = av_int2float(ff_cbrt_tab[c])*IQ;
>> curbits += av_log2(c)*2 - 4 + 1;
>> }
>> } else {
>
> Shouldn't that be c*av_int2float(ff_cbrt_tab[c])*IQ ?
cbrt_tab is a very unfortunate misnomer that stuck for a long time, it
is actually really x^(4/3). See e.g git checkout a9d986
libavcodec/cbrt_tablegen.h.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
