Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Armin Hasitzka]

Thanks for your input! :)

I have something like that in mind but I don't think that a complicated 
algorithm is needed here; we're basically working with two alpha bitmaps of 
(roughly) the same size. I would simply compare the bitmaps on a pixel-by-pixel 
basis, weigh the results by the max. possible difference and get the average of 
all differences. That way it would also be simple to add a threshold - may it 
be absolute or relative. In the end, the result can also be applied to a 
logarithmic scale - something that I would play around with to find the optimal 
solution for the needs of FreeType.

On 19/03/2018 19:00:48, Nikolaus Waxweiler  wrote:
I'm not familiar with graphics testing, but maybe algorithms exist that look 
for a magnitude of a difference between two images (maybe something like this 
exists in video encoding land?). Then you could ignore changes less than 1% or 
something.___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Nikolaus Waxweiler]

I'm not familiar with graphics testing, but maybe algorithms exist that
look for a magnitude of a difference between two images (maybe something
like this exists in video encoding land?). Then you could ignore changes
less than 1% or something.
___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Armin Hasitzka]

Thanks a lot for clarifying that, I will look into that demo program! :)

Armin
On 19/03/2018 18:25:21, Werner LEMBERG  wrote:

> Further, if I understand correctly (I have to and will read up on
> this though), outlines do not take stem hints into account.

This is not correct. It's exactly the hinting that distorts glyph
outlines so that they are rendered better at lower resolutions. You
might play with the `ftgrid' demo program to see that in action.


Werner
___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Werner LEMBERG]

> Further, if I understand correctly (I have to and will read up on
> this though), outlines do not take stem hints into account.

This is not correct.  It's exactly the hinting that distorts glyph
outlines so that they are rendered better at lower resolutions.  You
might play with the `ftgrid' demo program to see that in action.


Werner

___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Armin Hasitzka]

Thanks for your insight! :)

I was just wondering in terms of a test framework that would detect and present 
rendering regressions: what should said framework compare, and what would it 
present? Have you had a chance to look at my GSoC draft proposal?

Judging from your text, comparing glyphs (that were rendered with different 
versions of FreeType) on a bitmap basis will almost certainly come up with many 
differences. The key is to list the glyphs according to their relative 
difference per pixel pair. In addition, a minimum threshold for the difference 
of considered pixel pairs could be useful to rule out off-by-one issues.

Further, if I understand correctly (I have to and will read up on this though), 
outlines do not take stem hints into account. My understanding is that hinting 
is only considered during rasterisation. This is part of the reason, why 
comparing outlines might not be the ultimate truth (compared to comparing 
bitmaps in combination with the metrics).

Please correct me where I am wrong and please also feel free to elaborate on 
what you think that a test framework (for glyph regression testing) SHOULD 
compare (bitmaps, outlines, metrics, ...) and what it should ultimately present.

Thanks,
Armin
 

On 19/03/2018 15:58:08, Alexei Podtelezhnikov  wrote:
> > Concretely: is the outline-to-bitmap algorithm definitive and
> > final or is it subject to (probably subtle) changes every now and
> > then?
>
> Most changes happen to the outlines. However, from time to time, both
> the B/W and graymap rasterizers get modified; for such cases it is
> necessary to run the whole test suite.

I doubt that coverage map calculations would fundamentally change. We
literally calculate percentages of square-grid pixels covered by
outlines. There are however rounding errors of two kinds. First, even
currently 64-bit and 32-bit algorithms are different and the resulting
maps are not exactly the same. These differences are not visible to
the naked eye, they are literally rounding errors of ±1 out of 255.
Such discrepancies should not be considered. Second, some rounding
happens when flattening Bézier curves. This can be tuned differently
and the discrepancies might be substantial and visible, The text
appearance perception is however dominated by vertical and horizontal
stems, which are not affected by flattening.

That being said. It is not trivial to judge the improvement of text
appearance from outlines alone without rasterization. I think the
balance between the contrast of images and the regularity in stem and
letter spacing is the key. This balance is highly subjective though.
___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Alexei Podtelezhnikov]

> > Concretely: is the outline-to-bitmap algorithm definitive and
> > final or is it subject to (probably subtle) changes every now and
> > then?
>
> Most changes happen to the outlines.  However, from time to time, both
> the B/W and graymap rasterizers get modified; for such cases it is
> necessary to run the whole test suite.

I doubt that coverage map calculations would fundamentally change. We
literally calculate percentages of square-grid pixels covered by
outlines. There are however rounding errors of two kinds. First, even
currently 64-bit and 32-bit algorithms are different and the resulting
maps are not exactly the same. These differences are not visible to
the naked eye, they are literally rounding errors of ±1 out of 255.
Such discrepancies should not be considered. Second, some rounding
happens when flattening Bézier curves. This can be tuned differently
and the discrepancies might be substantial and visible, The text
appearance perception is however dominated by vertical and horizontal
stems, which are not affected by flattening.

That being said. It is not trivial to judge the improvement of text
appearance from outlines alone without rasterization. I think the
balance between the contrast of images and the regularity in stem and
letter spacing is the key. This balance is highly subjective though.

___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Armin Hasitzka]

Thanks for the quick reply and clarification! :)

I was thinking the same: having the option to force the inspection of bitmaps 
in addition to checking outlines (which happens wherever possible).

On 18/03/2018 15:18:14, Werner LEMBERG  wrote:

> as suggested by Werner, I currently look into using outlines as a
> (considerably faster) means to detect rendering regressions. I do
> understand that the outlines of glyphs should definitely be checked.
> However, I wonder if it is enough to guarantee "no rendering
> regressions" by just comparing the outlines and without checking the
> rendered bitmaps?

It depends. If there are no changes to the rasterizer, it is
sufficient. So maybe two targets, `check_outlines' and
`check_rasterization' (or something like that) will do.

> To get a feeling for that matter, it would help me to understand
> where most changes of FreeType happen and where versions differ the
> most. Concretely: is the outline-to-bitmap algorithm definitive and
> final or is it subject to (probably subtle) changes every now and
> then?

Most changes happen to the outlines. However, from time to time, both
the B/W and graymap rasterizers get modified; for such cases it is
necessary to run the whole test suite.


Werner
___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

Re: [ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Werner LEMBERG]

> as suggested by Werner, I currently look into using outlines as a
> (considerably faster) means to detect rendering regressions.  I do
> understand that the outlines of glyphs should definitely be checked.
> However, I wonder if it is enough to guarantee "no rendering
> regressions" by just comparing the outlines and without checking the
> rendered bitmaps?

It depends.  If there are no changes to the rasterizer, it is
sufficient.  So maybe two targets, `check_outlines' and
`check_rasterization' (or something like that) will do.

> To get a feeling for that matter, it would help me to understand
> where most changes of FreeType happen and where versions differ the
> most.  Concretely: is the outline-to-bitmap algorithm definitive and
> final or is it subject to (probably subtle) changes every now and
> then?

Most changes happen to the outlines.  However, from time to time, both
the B/W and graymap rasterizers get modified; for such cases it is
necessary to run the whole test suite.


Werner

___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[ft-devel] [Rendering Regressions] Outlines vs. Bitmaps

[Armin Hasitzka]

Hi,

as suggested by Werner, I currently look into using outlines as a (considerably 
faster) means to detect rendering regressions. I do understand that the 
outlines of glyphs should definitely be checked. However, I wonder if it is 
enough to guarantee "no rendering regressions" by just comparing the outlines 
and without checking the rendered bitmaps?

To get a feeling for that matter, it would help me to understand where most 
changes of FreeType happen and where versions differ the most. Concretely: is 
the outline-to-bitmap algorithm definitive and final or is it subject to 
(probably subtle) changes every now and then?

Thanks for your input and have a wonderful weekend, everyone!

Armin___
Freetype-devel mailing list
Freetype-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-devel