On 7 May 2016 at 18:12, Rostislav Pehlivanov wrote:
> The problem is that with particularly complex images and especially at
> high bit depths and 5-level transforms the coefficients would overflow,
> causing huge artifacts to appear. This was discovered thanks to the fate
> tests, which will hav
2016-05-07 21:48 GMT+02:00 Rostislav Pehlivanov :
> The costliest part of the encoder right now is encoding the coefficients
> (~36%). Slightly less-costly is rate control (~31%), and after that is the
> transform (~12%). There really isn't anything else, other than 3 copies
> (input image convert
On Sat, May 07, 2016 at 06:12:43PM +0100, Rostislav Pehlivanov wrote:
> The problem is that with particularly complex images and especially at
> high bit depths and 5-level transforms the coefficients would overflow,
> causing huge artifacts to appear. This was discovered thanks to the fate
> tests
On 7 May 2016 at 18:55, Christophe Gisquet
wrote:
> 2016-05-07 19:12 GMT+02:00 Rostislav Pehlivanov :
> > There is a slight performance drop associated with this change, making
> > the encoder slower by 1.15 times, however this is necessary in order to
> > avoid undefined behavior and overflows
2016-05-07 19:12 GMT+02:00 Rostislav Pehlivanov :
> The problem is that with particularly complex images and especially at
> high bit depths and 5-level transforms the coefficients would overflow
I guess it also depends on the transform type, so that counts also for
the last comment.
> causing hu
The problem is that with particularly complex images and especially at
high bit depths and 5-level transforms the coefficients would overflow,
causing huge artifacts to appear. This was discovered thanks to the fate
tests, which will have to be redone as this fixes a multitude of
problems and incre
