2012/2/28 Måns Rullgård <[email protected]>:
> Christophe Gisquet <[email protected]> writes:
>
>> 2012/2/28 Måns Rullgård <[email protected]>:
>>> Be careful. That function has a comment warning about it being
>>> numerically unstable. Such a transformation might cause it to
>>> blow up.
>>
>> I'm indeed only testing on only one sample, but this is the result I
>> observed:
>> stddev: 0.00 PSNR:142.70 MAXDIFF: 1 bytes: 65789952/ 65789952
>>
>> Indeed unstable, but not so dangerously.
>
> On that sample. I'd prefer if this were tested against the full
> conformance suite.
>
The particular issue occurs as the magnitude of alpha{0,1}[k]
approaches and passes 4. Rather than saturating at 4, they snap back
to zero (which is not some equivalent state to 4). Those dk values are
the denominator of alpha1[k].
I'd expect the SBR experts to probably say something not too useful
like well don't operate near magnitude 4.
It might be interesting to see what the largest magnitude across the
whole conformance suite is.
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
