Re: Odp: Re: The TrueType rendering accuracy nightmare
Hin-Tak, > There is a bilevel mode in ft2-demo, and you can also switch to v35 > to match Microsoft better in ft2-demo, from the default v40. I told > you so, many emails ago, in my first reply. > > If you feel that you know fontforge and freetype well enough to tell > which is fontforge and which is freetype, I have nothing more to say > on that. Sigh. if you ask FontForge for B/W rasterization it uses FreeType in v35 mode. In other words, the rasterization results returned by FontForge in this case are *identical* to what ft2-demo returns. Werner
Re: Build system considerations
That sounds very good. Personally I'd be very happy to see the header inclusion macros gone where possible, because they confuse both users and IDEs. I would also be very happy to see meson support, because it seems to be gaining usage nicely and I do find it nicer than CMake. I'd be a bit sad if CMake support would go away if there was no meson support, however with meson support I would not care that much about CMake. One of the features with both CMake and meson I find useful is ability to use other projects as subprojects, directly in source form, often as git(hub) submodules. This makes it easy to quickly navigate to the source of everything in the IDE. Another benefit is that you can use latest version of the library, perhaps your own fork with pull requests pending, without having to install anything on your system. On Sun, May 17, 2020, 20:45 David Turner wrote: > Continuing my own thread, after taking the time to experiment a little. > > First, I've refactored the build system so we turn the rules.mk into > declarative files (instead of Makefile fragments) and get rid of the > module.mk files. > This allowed me to write a Python script to parse modules.cfg and the > various rules.mk to generate a Makefile based on the configuration > expressed there. > The script is called meta-build.py, and could be easily extended to > generate something else if needed, but I haven't gone further than that. > However, this let me explore with the exact details on how we configure > our build, and the various options to build the library (the generated > Makefile allows you to build three variants of the library: a static > library and a shared library both compiled with -fPIC, and a static library > compiled without it). > > Second, I wrote support files for the Meson build for the library. It is > similar to the CMakeLists.txt version, except for the installation / > packaging part. It was a good way to get used to the tool and to compare > its system/header/library probing features. > In the end, and this is personal opinion, I find Meson cleaner than CMake, > sometimes much cleaner, though it has its own little warts that were a bit > surprising, as a new comer. Both systems are so much better than auto-tools > however. > > I believe that the ability for any developer to just download a release > archive and run "./configure && make" to get a *default build* of the > library (and "make install" to install) is really important. I insist on > the "default" here, because we have a sophisticated customization system, > which was designed in a different time. > In other words, before considering switching the build to something else, > I propose the following: > > - Whatever we do, ensuring that "./configure && make && make install" > continues to work on Posix systems without extra dependencies. This would > only be guaranteed to work on a default build of the library (i.e. without > module customization). Configure options would still be provided as usual > to handle external dependencies like zlib, libz2, etc. > > - I would like to drop the dynamic module instantiation / lookup code from > the library, to simplify certain code paths. The ability to inject at > runtime a new FT_Module was designed at a time where it was expected that > people would write custom modules for proprietary/custom font formats, and > tools like git which make it easy to rebase changes on top of an upstream > project were not available or still very unfamiliar. These days, anyone can > create its own fork of an open-source library to add custom features for > its own little embedded project. This means no more modules.cfg, and > ftmodule.h auto-generation, which are fortunately implementation details. > For ABI stability, we will have to keep functions like FT_Add_Module(), but > just change them to return an FT_Err_Unimplemented_Feature error code. > > - I would like to ensure src/base/ftsystem.c does the right thing for all > systems. These days, we can avoid probing for and and > assume a Posix system provides a valid mmap() / munmap() implementation. > This will remove one more reason to use auto-tools. > - Similarly, there are several versions of ftdebug.c which are essentially > similar except for two things: How messages are sent to "the log" (stderr > by default), and how to parse the FT2_DEBUG environment variable (when such > thing exists). These two things can be encapsulated in a separate file. > - Also, we don't need to support 8.3 Filepathnames (DOS and Windows 9x > development are long dead), so we can finally use longer filenames (e.g. > ftdebug_posix.cc, ftdebug_windows.cc, etc..) to better differentiate the > files under src/base/ instead of sprinkling them under builds// > - It also means we can probably drop some of the ugly macros for header > files we're using (still provide them for the API, but change all users > otherwise). Yeah! > - Regarding ftdebug.c and logging, we probably want the customizable >
Re: Build system considerations
Le dim. 17 mai 2020 à 21:47, Nikolaus Waxweiler a écrit : > First off: all of this sounds fantastic! I always love when stuff is > deleted :) > > > In the end, and this is personal opinion, I find Meson cleaner than > CMake, > > I'd vote for removing CMake support. Its inofficial status means that > people shouldn't rely on it anyway, even if there inevitably are > people that do. > > > - I would like to drop the dynamic module instantiation / lookup code > from > > the library, to simplify certain code paths. > > Has this ever been used by anyone? > > Yes, I did for a company I worked for a long time, then I rewrote an open-source version of it (the PFR font driver). But things are very different now, > > - There are many things I'd like to get rid of, but apparently [cut off] > > ... yes? :D > > > - Finally, the largest issue I've seen is the dependency on Harfbuzz. The > > situation where both libraries depend on each other is a little bit > > ridiculous. I wonder how much of Harfbuzz we need, and if it's worth > > re-implementing in FreeType2 itself. But something tells me implementing > > hb_shape() can be really subtle. Any informed opinions on this? > > This can probably only be answered by HarfBuzz people. I think Behdad > is lurking here somewhere... >
Re: The TrueType rendering accuracy nightmare
That's actually great feedback, thanks Piotr, we'll have a look. Le dim. 17 mai 2020 à 20:47, Alexei Podtelezhnikov a écrit : > >> From the image, it is evident that both FreeType and TD renderer (my > own renderer that relies on FreeType but uses my own rasterizer, > https://typedesign.netlify.app/tdrenderer.html) are extremely glitchy. It > is very important to take research into TrueType rendering to properly > simulate the Microsoft bilevel renderer because it would help render fonts > correctly. The immense complexity of FreeType's rasterizer is still not > enough. What a nightmare! > >> > > What are we supposed to answer to your post exactly? > > In Piotr's defense, he is much more constructive in his bug reporting. > I does look that he identified and helped with: > https://savannah.nongnu.org/bugs/index.php?58373 where the borderline > pixels on horizontal stems are not turned on > https://savannah.nongnu.org/bugs/index.php?58352 where rounding in > bytecode seems to be off by 1. > > Alexei >
Re: Build system considerations
First off: all of this sounds fantastic! I always love when stuff is deleted :) > In the end, and this is personal opinion, I find Meson cleaner than CMake, I'd vote for removing CMake support. Its inofficial status means that people shouldn't rely on it anyway, even if there inevitably are people that do. > - I would like to drop the dynamic module instantiation / lookup code from > the library, to simplify certain code paths. Has this ever been used by anyone? > - There are many things I'd like to get rid of, but apparently [cut off] ... yes? :D > - Finally, the largest issue I've seen is the dependency on Harfbuzz. The > situation where both libraries depend on each other is a little bit > ridiculous. I wonder how much of Harfbuzz we need, and if it's worth > re-implementing in FreeType2 itself. But something tells me implementing > hb_shape() can be really subtle. Any informed opinions on this? This can probably only be answered by HarfBuzz people. I think Behdad is lurking here somewhere...
Re: Odp: Re: The TrueType rendering accuracy nightmare (Re: Freetype-devel Digest, Vol 184, Issue 24)
On Monday, 18 May 2020, 02:38:24 GMT+8, piotrunio-2...@wp.pl wrote: > > You showed a few fontforge-generated screenshots, which shows good > > agreement between fontforge and Microsoft rendering. If you think they are > > not good enough, fontforge's github tracker is the place to report it. > It's by no means a good agreement because it's not pixel-for-pixel identical. > In fact, many pixels differ; this is called extremely glitchy and requires > several bugfixes to get it fixed. And FontForge has nothing to do with how > FreeType itself renders, nor does there exist a more accurate mode than the > already existing bilevel mode that's extremely glitchy. There is a bilevel mode in ft2-demo, and you can also switch to v35 to match Microsoft better in ft2-demo, from the default v40. I told you so, many emails ago, in my first reply. If you feel that you know fontforge and freetype well enough to tell which is fontforge and which is freetype, I have nothing more to say on that. Sigh.
Re: The TrueType rendering accuracy nightmare
>> From the image, it is evident that both FreeType and TD renderer (my own >> renderer that relies on FreeType but uses my own rasterizer, >> https://typedesign.netlify.app/tdrenderer.html) are extremely glitchy. It is >> very important to take research into TrueType rendering to properly simulate >> the Microsoft bilevel renderer because it would help render fonts correctly. >> The immense complexity of FreeType's rasterizer is still not enough. What a >> nightmare! >> > What are we supposed to answer to your post exactly? In Piotr's defense, he is much more constructive in his bug reporting. I does look that he identified and helped with: https://savannah.nongnu.org/bugs/index.php?58373 where the borderline pixels on horizontal stems are not turned on https://savannah.nongnu.org/bugs/index.php?58352 where rounding in bytecode seems to be off by 1. Alexei
Re: Odp: Re: The TrueType rendering accuracy nightmare (Re: Freetype-devel Digest, Vol 184, Issue 24)
You showed a few fontforge-generated screenshots, which shows good agreement between fontforge and Microsoft rendering. If you think they are not good enough, fontforges github tracker is the place to report it. Its by no means a good agreement because its not pixel-for-pixel identical. In fact, many pixels differ; this is called extremely glitchy and requires several bugfixes to get it fixed. And FontForge has nothing to do with how FreeType itself renders, nor does there exist a more accurate mode than the already existing bilevel mode thats extremely glitchy.
Re: Odp: Re: The TrueType rendering accuracy nightmare (Re: Freetype-devel Digest, Vol 184, Issue 24)
On Sunday, 17 May 2020, 14:34:17 GMT+8, piotrunio-2...@wp.pl wrote: > That's not an excuse not to fix the FreeType bugs. The FreeType guys should > not focus on my software and instead fix their own bugs. > I do not intend to ask for any help, I'm already working on fixing the TD > renderer bugs myself. But will the FreeType guys fix their own bugs? You showed a few fontforge-generated screenshots, which shows good agreement between fontforge and Microsoft rendering. If you think they are not good enough, fontforge's github tracker is the place to report it. Since we are not talking about your buggy piece of software, and you are not asking for help, you should stop referring to it. And if you want to talk about freetype, please keep-freetype-devel in the cc.
Re: The TrueType rendering accuracy nightmare
Le jeu. 14 mai 2020 à 20:35, piotrunio-2...@wp.pl a écrit : > The standard ppem of the test font is: 16 > The standard string of the test font is: !"#$% > > From the image, it is evident that both FreeType and TD renderer (my own > renderer that relies on FreeType but uses my own rasterizer, > https://typedesign.netlify.app/tdrenderer.html) are extremely glitchy. It > is very important to take research into TrueType rendering to properly > simulate the Microsoft bilevel renderer because it would help render fonts > correctly. The immense complexity of FreeType's rasterizer is still not > enough. What a nightmare! > > What are we supposed to answer to your post exactly? The bitmap you posted has little context that explains how it was generated (in particular how you configured the TrueType interpreter for FreeType), or what we're supposed to see as defects (hint: when it comes to vector graphics rendering: differences != defects). The web page you link to mentions that your code is buggy anyway. We spent a lot of time tuning the TrueType interpreter and the monochrome rasterizer, trying to match the bi-level rendering of Windows for popular / well-known fonts (e.g. Arial), but we do not guarantee that rendering results will be 100% identical for all fonts, since some of the geometric computations performed during bytecode execution have varying level of accuracy / rounding behaviour, depending on their exact implementation details (which have never been part of the spec). However, if you think we are wrong, we would be happy to see you research on the topic explaining how to improve things in the code base, so we can answer to your claims on a technical basis. I'm sorry if you thought that FreeType would match rendering 100% of the time, we never made this claim, even if we try very hard. Of course, you're free to help us improve the situation. - David
Re: Build system considerations
Continuing my own thread, after taking the time to experiment a little. First, I've refactored the build system so we turn the rules.mk into declarative files (instead of Makefile fragments) and get rid of the module.mk files. This allowed me to write a Python script to parse modules.cfg and the various rules.mk to generate a Makefile based on the configuration expressed there. The script is called meta-build.py, and could be easily extended to generate something else if needed, but I haven't gone further than that. However, this let me explore with the exact details on how we configure our build, and the various options to build the library (the generated Makefile allows you to build three variants of the library: a static library and a shared library both compiled with -fPIC, and a static library compiled without it). Second, I wrote support files for the Meson build for the library. It is similar to the CMakeLists.txt version, except for the installation / packaging part. It was a good way to get used to the tool and to compare its system/header/library probing features. In the end, and this is personal opinion, I find Meson cleaner than CMake, sometimes much cleaner, though it has its own little warts that were a bit surprising, as a new comer. Both systems are so much better than auto-tools however. I believe that the ability for any developer to just download a release archive and run "./configure && make" to get a *default build* of the library (and "make install" to install) is really important. I insist on the "default" here, because we have a sophisticated customization system, which was designed in a different time. In other words, before considering switching the build to something else, I propose the following: - Whatever we do, ensuring that "./configure && make && make install" continues to work on Posix systems without extra dependencies. This would only be guaranteed to work on a default build of the library (i.e. without module customization). Configure options would still be provided as usual to handle external dependencies like zlib, libz2, etc. - I would like to drop the dynamic module instantiation / lookup code from the library, to simplify certain code paths. The ability to inject at runtime a new FT_Module was designed at a time where it was expected that people would write custom modules for proprietary/custom font formats, and tools like git which make it easy to rebase changes on top of an upstream project were not available or still very unfamiliar. These days, anyone can create its own fork of an open-source library to add custom features for its own little embedded project. This means no more modules.cfg, and ftmodule.h auto-generation, which are fortunately implementation details. For ABI stability, we will have to keep functions like FT_Add_Module(), but just change them to return an FT_Err_Unimplemented_Feature error code. - I would like to ensure src/base/ftsystem.c does the right thing for all systems. These days, we can avoid probing for and and assume a Posix system provides a valid mmap() / munmap() implementation. This will remove one more reason to use auto-tools. - Similarly, there are several versions of ftdebug.c which are essentially similar except for two things: How messages are sent to "the log" (stderr by default), and how to parse the FT2_DEBUG environment variable (when such thing exists). These two things can be encapsulated in a separate file. - Also, we don't need to support 8.3 Filepathnames (DOS and Windows 9x development are long dead), so we can finally use longer filenames (e.g. ftdebug_posix.cc, ftdebug_windows.cc, etc..) to better differentiate the files under src/base/ instead of sprinkling them under builds// - It also means we can probably drop some of the ugly macros for header files we're using (still provide them for the API, but change all users otherwise). Yeah! - Regarding ftdebug.c and logging, we probably want the customizable "message" function to take several parameters, e.g.: void FT_Log_Message(int trace_component, int trace_level, const char* fmt, va_list args). It's possible to consider trace events (i.e. "start/stop") to generate flame graph that are easier to read. - There are different versions of ftconfig.h, with a ton of stuff shared between them, we should probably move these to common header files whenever possible. - Similarly ftoption.h contains both user-selectable options, and things that are more tweaks for the implementation that a typical developer will never have to care for. I suggest we move the latter to an internal header instead. - There are many things I'd like to get rid of, but apparently - Finally, the largest issue I've seen is the dependency on Harfbuzz. The situation where both libraries depend on each other is a little bit ridiculous. I wonder how much of Harfbuzz we need, and if it's worth re-implementing in FreeType2 itself. But something tells me implementing hb_shape() can be really
Re: Odp: Re: The TrueType rendering accuracy nightmare
> I fixed some bugs in a development version, which would make the 2.0 > version of TD renderer significantly more accurate than 1.0. > > https://i.imgur.com/oKDg7F1.png > > I also made sure to research how dropout control works and implement > much more accurate rules than in TD renderer 1.0. So the FreeType > guys should fix the bugs in their rasterizer as well. I don't have time right now (moving from one flat to another) – could you provide patches for FreeType? It would be at least interesting to find out where exactly the difference happens, which is caused by rounding, I guess. Werner
Re: Odp: Re: The TrueType rendering accuracy nightmare (Re: Freetype-devel Digest, Vol 184, Issue 24)
> I fixed some bugs in a development version, which would make the 2.0 version > of TD renderer significantly more accurate than 1.0. > https://i.imgur.com/oKDg7F1.png You might.have tried the recent FreeType but you didn’t have to. > I also made sure to research how dropout control works and implement much > more accurate rules than in TD renderer 1.0. So the FreeType guys should fix > the bugs in their rasterizer as well. We might but should we? This is not a business you know...
Re: Odp: Re: The TrueType rendering accuracy nightmare (Re: Freetype-devel Digest, Vol 184, Issue 24)
No, its evident that both are glitchy. You cant arbitrary say one renderer but not the other does it quite well when both have many errors. They are both extremely glitchy, and thats exactly the implementation nightmare. There is an international agreed ISO standard, ISO 14496-22, currently at its 4th edition, about fonts. It is quite possible to say one renderer is more compliant than another. So yours is broken, and more broken than others. Thats quite evident. I fixed some bugs in a development version, which would make the 2.0 version of TD renderer significantly more accurate than 1.0. i.imgur.com i.imgur.com I also made sure to research how dropout control works and implement much more accurate rules than in TD renderer 1.0. So the FreeType guys should fix the bugs in their rasterizer as well.
