> Okay, here it is, the quant formula I promised some time ago:
>
> We got a sample x, floating point, positive (put the sign back on later, if
> you like).
>
> Calculate
> i = floor(pow(x, 0.75))
> as usual. i is integer.
>
> Now:
>
> if (x > pow(0.5 * (pow(i, 8.0/3.0) + pow(i+1, 8.0/3.0)) , 0.375))
> i++;
>
> and i is the quantized value.
>
> Rationale: minimize the quantization noise per sample. The iso formula merely
>minimizes the average case, and not really good either.
>
> The 2nd step can be done per table lookup, of course :-)
>
> Hope you like it,
>
> Segher
>
Just to explain a little more what Segher is writing about:
MP3 uses a nonlinear quantization of the MDCT coefficients.
Let x = an MDCT coefficient after all scalefactors have been applied.
MP3 needs to truncate x^(3/4) to an integer.
As an example,
x=1.80
x^3/4 = 1.55
two possible integers: i=1 or 2
Take i=1: 1^(4/3) = 1 error: 1.75-1 = .80
Take i=2: 2^(4/3) = 2.52 error: 2.52-1.75 = .72
So i=2 is the better choice. The iso formula says to just take i =
floor(x^3/4 + .4054) = 1, giving a sub optimal answer. I never knew
where the .4054 came from, but it must be (as Segher mentions) optimal
for a typical value of x.
The midpoint (in terms of the error) is: .5 * (1 + 2.52) = 1.76.
So if x<1.76, we should take i=1 and if x>1.76 we take i=2.
Segher, shouldn't the formula be:
> if (x > (0.5 * (pow(i, 4.0/3.0) + pow(i+1, 4.0/3.0))
> i++;
Mark
--
MP3 ENCODER mailing list ( http://geek.rcc.se/mp3encoder/ )
