> Just to explain a little more what Segher is writing about:
> MP3 uses a nonlinear quantization of the MDCT coefficients.
> Let x = an MDCT coefficient after all scalefactors have been applied.
> MP3 needs to truncate x^(3/4) to an integer.
>
> As an example,
> x=1.80
> x^3/4 = 1.55
>
> two possible integers: i=1 or 2
>
> Take i=1: 1^(4/3) = 1 error: 1.75-1 = .80
> Take i=2: 2^(4/3) = 2.52 error: 2.52-1.75 = .72
>
> So i=2 is the better choice. The iso formula says to just take i =
> floor(x^3/4 + .4054) = 1, giving a sub optimal answer. I never knew
> where the .4054 came from, but it must be (as Segher mentions) optimal
> for a typical value of x.
>
> The midpoint (in terms of the error) is: .5 * (1 + 2.52) = 1.76.
> So if x<1.76, we should take i=1 and if x>1.76 we take i=2.
>
> Segher, shouldn't the formula be:
>
>> if (x > (0.5 * (pow(i, 4.0/3.0) + pow(i+1, 4.0/3.0))
>> i++;
>
>
The real quantization is not x^3/4 but x^3/4 * 2^(0,25*quantizer+sublock�).
So we do a bad choice if we are quantizing as you say it above and choosing
1 instead of 2, but the conversion to int is after the multiplication with
the power of 2.
The quantization may still be bad but i don't think we should correct it as
stated above.
Lionel
--
MP3 ENCODER mailing list ( http://geek.rcc.se/mp3encoder/ )
