Re: I'm back

[Behdad Esfahbod]

Hi David,

Thanks for the detailed response.

On Mon, Jun 29, 2020 at 5:07 PM David Turner  wrote:

>
>
> Le dim. 28 juin 2020 à 04:19, Behdad Esfahbod  a
> écrit :
>
>> Hi David,
>>
>> I've been meaning to reply to your request but the list of things I
>> wanted to communicate kept growing exponentially and so I have not been
>> able to say.
>>
>> But here, from a technical perspective this is my biggest issue, which
>> you can also see as a roadmap:
>>
>>   https://gitlab.gnome.org/GNOME/pango/-/issues/404#note_851881
>>
>>
> Thanks for the link Bhedad, this gives useful history about your woes and
> considerations. I'd like to address some of these here.
>
> First of all, the reason why FreeType has never been designed to be
> "thread-safe" (in the sense where it supports multiple threads operating on
> its objects concurrently [1]) is *entirely and very very intentional*.
>

I fully understand it was intentional.


> The library was designed, first and foremost to be as efficient as
> possible on embedded systems with limited memory and operating system
> primitives.
> Any kind of thread-safety scheme, even atomic refcounts, would have been
> detrimental to its portability, and most likely performance.
>

But that doesn't have to be the case. For example, HarfBuzz could always be
compiled with HB_NO_MT to remove all that overhead. See hb-atomic.hh We
even enable that in our HB_TINY configuration. See CONFIG.md and
hb-config.hh.


> Reducing memory usage means using in-place modification of existing
> objects and aggressive caching of state within various objects under a root
> FT_Library instance.
>

I'm not convinced that you lose *any* efficiency within the model that I
propose. I hold HarfBuzz to the same if not more extreme standards and we
have pretty much always been able to find a solution without compromising.
We continue to do that. In fact, last night I started a major overhaul of
the codebase.

That's why we have FT_Size and FT_GlyphSlot objects, very visible in the
> API, but there are also various levels of caching internally (most of them
> simple LRU lists that only change occasionally, but still a potential
> source of thread races).
>

Umm. I removed all of those a few years ago. Again, without any side effect.


> And as far as I know, this has always been pointed out explicitly in the
> FT_Library documentation.
>

Yeah I'm not claiming any of it is a surprise (there was one surprise I ran
into the other day that I'll write about separately).


> I understand that if you expected otherwise, you may have been
> disappointed (to keep it politely).
>

I came to this field with zero expectations. Everything I learned was from
how people in projects large and small needed to show text.


> However I do not consider the API to be "broken" regarding thread-safety,
> it's just not designed for this use case, at all. It solves a very
> different problem.
>

If you wish so. I'm not crazy about words. :-)


> Thus there is no way to "fix all thread-safety" issues in it, without
> changing the internals and the API tremendously.
>

Agree about the internals, but not the API. I think new API can be added
and old API kept. It will be a complete overhaul, I agree, but then again,
the project is way overdue for one. So I'm suggesting that instead of a
freetype3 that has incompatible ABI, just evolve freetype2 into freetype3,
and freetype4, ... and remove old cruft after ten years or twenty years or
never. That's what I'm leaving HarfBuzz with.

You may have made some modifications to the library to get rid of the most
> annoying thread-related issues you've encountered,
> but I assure you there are still by-design thread-races in many places.
> Also, some of the fixes seem a little dubious, i.e. the raster pool is now
> a fixed 16 kiB now, while it used to be resizable by the user.
>

In twenty years I never ran into *any* FreeType client who would get
anywhere like configuring anything like that. If they did, they'd just
modify the source.

Again, I understand everything I point out can be countered with a "some
might need". That's a difference in library design philosophy.


> This was useful when rendering large vector graphics (not fonts) with the
> FreeType rasterizers. Now the algorithm are essentially quadratic, which
> makes it painful or unusable for this use case.
>

Again, if there was a mission statement to FreeType, we could deduce
whether that should remain a goal.


> The root of your problem seems to be a vast impedance mismatch between the
> Cairo / Pango / Harfbuzz APIs which expect, or encourage, several threads
> (probably from clients, which would be worse) to use FT_Face objects
> liberally.
> This simply cannot work reliably.
>

We agree on the root cause indeed.


> My only immediate recommendation would be to use a different "face"
> abstraction object for these, that would either reimplement FreeType, or
> parts of it (the route Pango seems to have chosen), or wrap 

Re: I'm back

[David Turner]

Le dim. 28 juin 2020 à 04:19, Behdad Esfahbod  a écrit :

> Hi David,
>
> I've been meaning to reply to your request but the list of things I wanted
> to communicate kept growing exponentially and so I have not been able to
> say.
>
> But here, from a technical perspective this is my biggest issue, which you
> can also see as a roadmap:
>
>   https://gitlab.gnome.org/GNOME/pango/-/issues/404#note_851881
>
>
Thanks for the link Bhedad, this gives useful history about your woes and
considerations. I'd like to address some of these here.

First of all, the reason why FreeType has never been designed to be
"thread-safe" (in the sense where it supports multiple threads operating on
its objects concurrently [1]) is *entirely and very very intentional*.
The library was designed, first and foremost to be as efficient as possible
on embedded systems with limited memory and operating system primitives.
Any kind of thread-safety scheme, even atomic refcounts, would have been
detrimental to its portability, and most likely performance.
Reducing memory usage means using in-place modification of existing objects
and aggressive caching of state within various objects under a root
FT_Library instance.
That's why we have FT_Size and FT_GlyphSlot objects, very visible in the
API, but there are also various levels of caching internally (most of them
simple LRU lists that only change occasionally, but still a potential
source of thread races).
And as far as I know, this has always been pointed out explicitly in the
FT_Library documentation.

I understand that if you expected otherwise, you may have been disappointed
(to keep it politely). However I do not consider the API to be "broken"
regarding thread-safety, it's just not designed for this use case, at all.
It solves a very different problem.
Thus there is no way to "fix all thread-safety" issues in it, without
changing the internals and the API tremendously. You may have made some
modifications to the library to get rid of the most annoying thread-related
issues you've encountered,
but I assure you there are still by-design thread-races in many places.
Also, some of the fixes seem a little dubious, i.e. the raster pool is now
a fixed 16 kiB now, while it used to be resizable by the user.
This was useful when rendering large vector graphics (not fonts) with the
FreeType rasterizers. Now the algorithm are essentially quadratic, which
makes it painful or unusable for this use case.

The root of your problem seems to be a vast impedance mismatch between the
Cairo / Pango / Harfbuzz APIs which expect, or encourage, several threads
(probably from clients, which would be worse) to use FT_Face objects
liberally.
This simply cannot work reliably.

My only immediate recommendation would be to use a different "face"
abstraction object for these, that would either reimplement FreeType, or
parts of it (the route Pango seems to have chosen), or wrap FreeType
objects behind the right amount of
caching and thread-specific synchronization to make it work (like other
libraries like Skia do). I can't tell you which way is best. It would be
interesting to discuss which properties these "thread-safe font objects"
could have.
Maybe we can write a sane and small library on top of FreeType to provide
just that. I just don't think there is a way to do that within it.

Another alternative would be to refactor the FreeType library considerably
into a new version (with a different thread-safe API where it matters), but
that's a far harder problem, is likely to result in something that will be
larger, more complex internally, use more memory, and likely be slower.
At least we could rewrite the internals with a better language than C89 :)
But I'd rather have an extensive unit-test and regression test suite before
trying this.

- David

[1] Except if each thread has its own FT_Library instance (and
corresponding set of FT_Face/FT_Size/FT_GlyphSlot objects.

I will keep trying to comment on the two GSoC projects, and eventually get
> to write about the health of the freetype as a community project.
>
> Cheers,
> behdad
>
> On Fri, Apr 24, 2020 at 1:03 PM David Turner  wrote:
>
>> Hello freetype-devel@ list members,
>>
>> It's been a very very long time, but I have some free time in the coming
>> weeks to work on FreeType. Werner invited me to write a small announcement
>> here and I'm currently looking at the official bugs list.
>>
>> I'd like to know what are, in your opinion, the most pressing issues to
>> work on at that point?
>>
>> Apart from that, I had the following things in mind:
>>
>> - Improving / refactoring the build system a little. E.g. it should be
>> possible to simplify the rules.mk/module.mk files considerably, and
>> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single
>> source of truth (exact format to be defined), at least the parts that deal
>> with the headers / sources / configuration headers and the module
>> dependencies.
>>
>> - Improve testing (unit 

Re: I'm back

[Behdad Esfahbod]

Hi David,

I've been meaning to reply to your request but the list of things I wanted
to communicate kept growing exponentially and so I have not been able to
say.

But here, from a technical perspective this is my biggest issue, which you
can also see as a roadmap:

  https://gitlab.gnome.org/GNOME/pango/-/issues/404#note_851881

I will keep trying to comment on the two GSoC projects, and eventually get
to write about the health of the freetype as a community project.

Cheers,
behdad

On Fri, Apr 24, 2020 at 1:03 PM David Turner  wrote:

> Hello freetype-devel@ list members,
>
> It's been a very very long time, but I have some free time in the coming
> weeks to work on FreeType. Werner invited me to write a small announcement
> here and I'm currently looking at the official bugs list.
>
> I'd like to know what are, in your opinion, the most pressing issues to
> work on at that point?
>
> Apart from that, I had the following things in mind:
>
> - Improving / refactoring the build system a little. E.g. it should be
> possible to simplify the rules.mk/module.mk files considerably, and
> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single
> source of truth (exact format to be defined), at least the parts that deal
> with the headers / sources / configuration headers and the module
> dependencies.
>
> - Improve testing (unit and regression tests to be exact) There are lots
> of possibilities here, and it will probably better to do this in small
> incremental steps.
>
> Voila, I'd be happy to read your suggestions, Happy to be here.
>
> - David Turner
>


-- 
behdad
http://behdad.org/

Re: I'm back

[Ben Wagner]

Responding to all the scattered bits of this thread I know something
about...


> Skia: https://github.com/google/skia/tree/master/third_party/freetype2
> modules: no raster1, type1, pfr, bitmapped formats
> options: seems to be using the default, i.e. no subpixel rendering, no
> harfbuzz, no adobe glyph list


I wouldn't look too hard at the Skia one, it's set up that way for odd
testing reasons and isn't really part of a Skia release. You'd probably be
more interested in how FreeType is built and used by the Android Open
Source Project [0] (though always out of date) and by Chromium [1]. Note
that they both use their own build system.

[0]
https://android.googlesource.com/platform/external/freetype/+/refs/heads/master/Android.bp
[1]
https://source.chromium.org/chromium/chromium/src/+/master:third_party/freetype/BUILD.gn


> Well, we need a different corpus for rendering testing – I think
> we should use what Chromium currently uses.
>

Chromium itself currently uses checked in images and a bunch of custom
scripts to go with the builders to detect pixel changes. They're actually
looking to move to something a bit better. It's just difficult to change a
big thing like that in Chromium.


> BTW, Chromium does a 'live' testing of FreeType, this is, the project
> monitors FreeType's git HEAD and reports rendering differences to
> Chromium developers, which in turn report to us.  Of course, they
> would like to get rid of that eventually :-)


Note that Chromium doesn't just do live testing of FreeType, Chromium
actually stays at FreeType tip of tree. (Note that Chrome always ships with
a statically linked fairly recent FreeType; Chromium can be built with
FreeType either statically or dynamically linked; some distros build
Chromium to use the dynamically linked system libfreetype6.so.) It
generally takes ~4 hours from the time a FreeType commit is made to run all
the Chromium bots with the new FreeType and then a manual approval [2].
Chromium has enough in it's third_party directory to be a distro by itself
and tries to solve the distro type issues by just staying at tip of tree
(and since it's all internal doesn't need the API / ABI stability). My
earlier comments about releases and distros have a lot more to do with the
Chromium (not Chrome) builds the distros do, since the Chromium bug tracker
sometimes gets reports about an old FreeType in the distro misbehaving
[3][4]. These are examples of things that would have been nice to
cherry-pick in a FreeType branch and just let the distros pick up. Chromium
somewhat needs to be at FreeType tip of tree anyway for all the fuzzing, as
Chromium also runs the FreeType fuzzers and a few others in addition that
fuzz FreeType (independent of ossfuzz). Since keeping up to date is now
automated, far from trying to be rid of it, we actually like it.

[2] https://autoroll.skia.org/r/freetype-chromium
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932303
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=929982

There are probably several dimensions to explore from this,

from the simplest things to full goodies like Skia Gold
>  which

is really impressive, can be used for non-Skia projects,

but requires setting up / administering / paying for

Google-Cloud accounts / services.


If you're interested in Skia's Gold project I can put you in touch with the
maintainers. They are interested in making it more easily available and run
on more generic platforms.

Re: I'm back

[Nikolaus Waxweiler]

(oops, forgot to send to the list)

-- Forwarded message --
From: Nikolaus Waxweiler 
Date: Thu, 30 Apr 2020 09:28:34 +0100
Subject: Re: I'm back
To: David Turner 

> Can you clarify what that means exactly? I think I lack context and/or
> examples :-)

See https://www.freetype.org/freetype2/docs/text-rendering-general.html :)

>> Then Qt can go all in on stem
>> darkening and rendering fonts correctly. And hopefully everyone else.
>>
>> What do you mean by "going all in on stem darkening", especially for QT?
> Are you advocating for UIs with darker / bolder fonts or something like
> that?
> Just trying to understand.

To correctly render fonts, you need to linearly alpha blend FreeType's
output onto a surface and then gamma correct it. This necessarily
lightens text, so you stem-darken to counter the thinning. The Adobe
CFF renderer introduced that functionality and Qt 5.9+ will use linear
alpha blending and gamma correction plus stem darkening on CFF fonts.
I implemented stem darkening for the autohinter, but quality at
smaller sizes where it matters most does not match the CFF driver :(
Providing good stem darkening for all drivers would enable Qt (and
hopefully Skia and others) to use correct font rendering regardless of
the underlying font format.

>> Also, the build system generation thing sounds very meta. I'd prefer
>> settling on a single build system (either CMake or Meson and nothing
>> else) 
>>
>> I was thinking about something much simpler at first. For example, I have
> some local changes to considerably simplify the content of the rules.mk /
> module.mk.
> This uses a few custom GNU Make functions but that's a hidden /
> implementation detail. The point is to make these .mk files easily
> parseable by something else.
>
> The current build system was designed in a very different time, and there
> is definitely value in changing it more drastically. I'll start a different
> thread to discuss the idea though.

I suppose a simplification would be a good start in any case.

Maybe a single build system like Meson that also generates Makefiles
for other platforms while assembling a release would do the job?
Werner said he'd like a bare git checkout to just work with `make`,
but maybe it's good enough to provide that just in release tar balls?

> Is there a specific corpus of font files being used currently for
> regression testing? Should we try to create one or improve the existing
> one?

I don't know of any test fonts, outside of maybe Armin's fuzzing collection.

BTW, one idea for a regression test I had here would be to have some
fonts like the Noto fonts in OTF and TTF format compiled from the same
source and ensure that e.g. advance width and other metrics match
exactly between drivers (also slight autohinting that should not mess
with various metrics).

Re: I'm back

[Werner LEMBERG]

>> [...] I think it would be helpful if FreeType (and more open source
>> projects) tagged bugfix releases more often.

Tagging is the easy thing, but doing the release process is tedious,
for which I often lack time and, admittedly, energy.

> We should probably look at the patches applied on top of previous
> releases as well and see what important things should be moved into
> a release branch.  I'll file a bug for this with more details.  The
> main question will be how long do we support older releases, and
> what goes into fixes (I would suggest only security fixes as a
> starting point).

Having a helping hand in doing releases would be very, very helpful...


Werner

Re: I'm back

[Werner LEMBERG]

>> My favorite pet issue I never got around to fixing: make stem
>> darkening for the autohinter be as good as the CFF variant. Also,
>> at least basic stem darkening for bytecode'd TTFs.
> 
> Can you clarify what that means exactly? I think I lack context
> and/or examples :-)

I leave explanations to Nikolaus :-)

>> And even a basic testing framework sounds good.  One could, if so
>> inclined, even use https://github.com/rougier/freetype-py and use
>> pytest and hypothesis to hammer out a test suite.
>
> I'd prefer something that tests the FreeType API directly instead,
> so something based on C or C++ instead. GoogleTest seems a good
> candidate for this.

There are two things to test.

  * API and code coverage

  * Rendering

For the latter we have a potential GSoC project proposal this year, so
let's please wait until May 4th before continuing the discussion.

> Another possibility is to change the way regression testing is
> performed (generating MD5 directly in ftobjs.c is ... unelegant and
> limited).

What's the problem with MD5?  The idea is to find out whether two
FreeType versions render a glyph differently.  I'm not aware of
something simpler.  It's not intended for API testing, of course.

> Is there a specific corpus of font files being used currently for
> regression testing?

For regression testing with have Armin's project:

  https://github.com/freetype/freetype2-testing

> Should we try to create one or improve the existing one?

Well, we need a different corpus for rendering testing – I think
we should use what Chromium currently uses.

BTW, Chromium does a 'live' testing of FreeType, this is, the project
monitors FreeType's git HEAD and reports rendering differences to
Chromium developers, which in turn report to us.  Of course, they
would like to get rid of that eventually :-)


Werner

Re: I'm back

[Werner LEMBERG]

>> not a pressure issue, but i would like to have VF driver. A Summer
>> of Code task has begun, never finished, though.
>
> Are you talking about TeX Virtual Fonts, or OpenType Variable Fonts
> here?

He talks about the former.  Support for OpenType VFs is already part
of FreeType.

> For the former, that sounds like a lot of work, for a format that
> isn't used to create new fonts anymore.  And as far as I understand,
> most TeX fonts of interest have been transliterated to OpenType
> already.

Well, there are still people interested in such font formats,
obviously :-) I like a lot that FreeType supports many different font
formats, so why not the TeX ones?  It is very unfortunate that the
GSoC project some years ago didn't deliver anything useful – the job
was to integrate the excellent 'VFlib' library stuff into FreeType,
and the guy seemed to be overwhelmed with this job.

>> I would be interested in knowing whether anyone is using Jamfiles
>> these days?
> 
> So in other words, let's get rid of the Jamfiles if nobody is using
> them today.  If someone is opposed to that, please speak up!

Given that a special, unmaintained version of Jam is needed to build
FreeType, I think that removing Jam support is the right way to
proceed.

As a consequence I've just added a sentence to the CHANGES file that
Jam support will be removed in the release after the next one (i.e.,
not in 2.10.2 but in 2.10.3).


Werner

Re: I'm back

[David Turner]

Le lun. 27 avr. 2020 à 18:09, Ben Wagner  a écrit :

> One thing that's been on my mind is around releasing (and this is a
> fairly general observation not specific to FreeType). FreeType has
> generally tagged a VER-2-X-X release about once every six months and
> hasn't had maintained release branches. Distros (and other users)
> generally only update to tagged releases or branches and then try to
> maintain a patch pile of cherry-picks for bug fixes (essentially a
> release branch out of tree). In the present day any non-rolling distro
> model is slightly crazy since most open source projects work this way;
> the distro should be at more or less tip of tree or there will be
> known public exploitable bugs which the distro maintainer probably
> doesn't even know should be cherry-picked. However even rolling
> distros don't generally always stay at tip of tree to avoid incidental
> intermediate API/ABI breakage so tend to lean on the developers to tag
> commits to bless them. All of that to say, I think it would be helpful
> if FreeType (and more open source projects) tagged bugfix releases
> more often. For example if there are no API changes at all since the
> previous release tag a bugfix release as soon as reasonable.  It may
> seem a bit silly from the developer point of view, but it means that
> most users will get bugfixes much quicker. The versioned distros will
> still need to cherry-pick things back to their old versions, but users
> that keep up to date will benefit greatly.
>
> Using release branches (as they are described in Trunk Based Development
) for this makes total sense.
I have started looking at how various distributions / projects configure /
build FreeType [1], but only for the latest version (2.10.1).
For that one, the list of patches is rather minimal imho (most are about
changing configuration options).

We should probably look at the patches applied on top of previous releases
as well and see what important things should be moved into a release branch.
I'll file a bug for this with more details. The main question will be how
long do we support older releases, and what goes into fixes (I would
suggest only security fixes as a starting point).


>
> I was reminded of this sort of issue this morning when I experienced
> an extreme version of this issue with the 'ardour' package in debian
> which is currently code from two years ago and the package didn't ship
> in the bullseye release because the old code didn't work with newer
> libraries even though upstream had long since fixed the issue but
> there have been no new tags since.
>
>
>
[1] Nix:
https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/libraries/freetype
(subpixel-rendering + table-validation)

Debian testing: https://packages.debian.org/testing/libs/libfreetype6
(subpixel-rendering + table-validation + scale-phantom-points +
verbose-libtool)

Ubuntu focal: https://packages.ubuntu.com/focal/libs/libfreetype6 (same as
debian)

Fedora: https://src.fedoraproject.org/rpms/freetype/tree/
(subpixel-rendering + table validation + ABI fixes(?!?) + libtool fixes)

OpenSuse:
https://build.opensuse.org/package/show/openSUSE%3AFactory/freetype2
(subpixel-rendering, enable long family names, enable infinality,
libpng-not-required, cmex-font-workaround)

ArchLinux:
https://git.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/freetype2
(table validation, enable infinality, enable long pcf family names, phantom
points)

Gentoo:
https://gitweb.gentoo.org/repo/gentoo.git/tree/media-libs/freetype/freetype-2.10.1.ebuild
(TODO)

FreeBSD: https://www.freshports.org/print/freetype2/

NetBSD:
https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/graphics/freetype2/patches/
(TODO)

OpenBSD: only using freetype 1.3.1 !!

ReactOS: ??

Skia: https://github.com/google/skia/tree/master/third_party/freetype2
modules: no raster1, type1, pfr, bitmapped formats
options: seems to be using the default, i.e. no subpixel rendering, no
harfbuzz, no adobe glyph list

Le ven. 24 avr. 2020 à 16:02, David Turner  a écrit :
> >
> > Hello freetype-devel@ list members,
> >
> > It's been a very very long time, but I have some free time in the coming
> weeks to work on FreeType. Werner invited me to write a small announcement
> here and I'm currently looking at the official bugs list.
> >
> > I'd like to know what are, in your opinion, the most pressing issues to
> work on at that point?
> >
> > Apart from that, I had the following things in mind:
> >
> > - Improving / refactoring the build system a little. E.g. it should be
> possible to simplify the rules.mk/module.mk files considerably, and
> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single
> source of truth (exact format to be defined), at least the parts that deal
> with the headers / sources / configuration headers and the module
> dependencies.
> >
> > - Improve testing (unit and regression tests to be exact) There are lots
> of 

Re: I'm back

[David Turner]

Le ven. 24 avr. 2020 à 22:46, Nikolaus Waxweiler  a
écrit :

> Woah 
>
> My favorite pet issue I never got around to fixing: make stem darkening
> for the autohinter be as good as the CFF variant. Also, at least basic
> stem darkening for bytecode'd TTFs.


Can you clarify what that means exactly? I think I lack context and/or
examples :-)


> Then Qt can go all in on stem
> darkening and rendering fonts correctly. And hopefully everyone else.
>
> What do you mean by "going all in on stem darkening", especially for QT?
Are you advocating for UIs with darker / bolder fonts or something like
that?
Just trying to understand.


> Also, the build system generation thing sounds very meta. I'd prefer
> settling on a single build system (either CMake or Meson and nothing
> else) 
>
> I was thinking about something much simpler at first. For example, I have
some local changes to considerably simplify the content of the rules.mk /
module.mk.
This uses a few custom GNU Make functions but that's a hidden /
implementation detail. The point is to make these .mk files easily
parseable by something else.

The current build system was designed in a very different time, and there
is definitely value in changing it more drastically. I'll start a different
thread to discuss the idea though.

And even a basic testing framework sounds good. One could, if so
> inclined, even use https://github.com/rougier/freetype-py and use
> pytest and hypothesis to hammer out a test suite.
>
> I'd prefer something that tests the FreeType API directly instead, so
something based on C or C++ instead. GoogleTest seems a good candidate for
this.
Another possibility is to change the way regression testing is performed
(generating MD5 directly in ftobjs.c is ... unelegant and limited).
There are probably several dimensions to explore from this, from the
simplest things to full goodies like Skia Gold
 which is really impressive, can be
used for non-Skia projects, but requires setting up / administering /
paying for Google-Cloud accounts / services.

Is there a specific corpus of font files being used currently for
regression testing? Should we try to create one or improve the existing one?

Re: I'm back

[David Turner]

Le lun. 27 avr. 2020 à 18:21, Vincent Torri  a
écrit :

> Hello
>
> not a pressure issue, but i would like to have VF driver. A Summer of
> Code task has begun, never finished, though.
>
> Are you talking about TeX Virtual Fonts, or OpenType Variable Fonts here?

For the former, that sounds like a lot of work, for a format that isn't
used to create new fonts anymore.
And as far as I understand, most TeX fonts of interest have been
transliterated to OpenType already.

For the second, it looks like there are fonts available from
https://github.com/adobe-fonts/variable-font-collection-test
that could be a starting point, but we may have indeed other priorities for
now.


> about build system, i would say meson only for Windows, Linux, BSD*
> and Solaris, and no change for the other OS. That would simplify the
> build system. are Jamfiles really useful these days ? In addition,
> having a lot of build system is a pain to have them always up to date
>
> I would be interested in knowing whether anyone is using Jamfiles these
days?
I found them useful to me 10-15 years ago when I was hacking heavily on the
library because they built faster, and more correctly, and supported
rebuilding both the library and the demos at the same time.
However, all of this can be served by a better build system too, whatever
form it takes (let's keep that for a separate thread).

So in other words, let's get rid of the Jamfiles if nobody is using them
today. If someone is opposed to that, please speak up!

- David


> best regards
>
> Vincent Torri
>
> On Fri, Apr 24, 2020 at 10:02 PM David Turner  wrote:
> >
> > Hello freetype-devel@ list members,
> >
> > It's been a very very long time, but I have some free time in the coming
> weeks to work on FreeType. Werner invited me to write a small announcement
> here and I'm currently looking at the official bugs list.
> >
> > I'd like to know what are, in your opinion, the most pressing issues to
> work on at that point?
> >
> > Apart from that, I had the following things in mind:
> >
> > - Improving / refactoring the build system a little. E.g. it should be
> possible to simplify the rules.mk/module.mk files considerably, and
> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single
> source of truth (exact format to be defined), at least the parts that deal
> with the headers / sources / configuration headers and the module
> dependencies.
> >
> > - Improve testing (unit and regression tests to be exact) There are lots
> of possibilities here, and it will probably better to do this in small
> incremental steps.
> >
> > Voila, I'd be happy to read your suggestions, Happy to be here.
> >
> > - David Turner
>

Re: I'm back

[Alexei Podtelezhnikov]

On Fri, Apr 24, 2020 at 4:02 PM David Turner  wrote:
> Voila, I'd be happy to read your suggestions, Happy to be here.

.. and another project is on the table after we lost Moazin's GSoC
proposal on font-rs. I was actually advocating for a couple of
FT_Renderer modules for alternative rasterizers:
Raph Levien's font-rs for SIMD (or fontdue) and Pathfinder for GPU.
They claim to be the fastest rasterizer out there. Here is the links:

https://github.com/raphlinus/font-rs
https://github.com/mooman219/fontdue
https://github.com/servo/pathfinder

Alexei

Re: I'm back

[Alexei Podtelezhnikov]

On Fri, Apr 24, 2020 at 4:02 PM David Turner  wrote:
> Voila, I'd be happy to read your suggestions, Happy to be here.

Welcome back, David!

If you still enjoy coding the renderers... We need to implement a
color one and deal with FT_Glyph and FT_GlyphSlot insufficiencies for
that. I made some progress in the "color" branch but lost steam. This
is for modern COLR/CPAL emoji fonts. The goal is to make them work
seamlessly even with FTC caching.

The renderer itself has been implemented in the "color" branch. To
store the layer make-up and colors, I used FT_GlyphSlot->subglyphs and
expanded FT_OultineGlyph. Two things are still needed but I lack the
knowledge: (1) parsing of COLR/CPAL into subglyphs and (2) fitting FTC
for color bitmaps. I would love to recruit your help here.

Thank you,
Alexei

Re: I'm back

[Vincent Torri]

Hello

not a pressure issue, but i would like to have VF driver. A Summer of
Code task has begun, never finished, though.

about build system, i would say meson only for Windows, Linux, BSD*
and Solaris, and no change for the other OS. That would simplify the
build system. are Jamfiles really useful these days ? In addition,
having a lot of build system is a pain to have them always up to date

best regards

Vincent Torri

On Fri, Apr 24, 2020 at 10:02 PM David Turner  wrote:
>
> Hello freetype-devel@ list members,
>
> It's been a very very long time, but I have some free time in the coming 
> weeks to work on FreeType. Werner invited me to write a small announcement 
> here and I'm currently looking at the official bugs list.
>
> I'd like to know what are, in your opinion, the most pressing issues to work 
> on at that point?
>
> Apart from that, I had the following things in mind:
>
> - Improving / refactoring the build system a little. E.g. it should be 
> possible to simplify the rules.mk/module.mk files considerably, and 
> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single 
> source of truth (exact format to be defined), at least the parts that deal 
> with the headers / sources / configuration headers and the module 
> dependencies.
>
> - Improve testing (unit and regression tests to be exact) There are lots of 
> possibilities here, and it will probably better to do this in small 
> incremental steps.
>
> Voila, I'd be happy to read your suggestions, Happy to be here.
>
> - David Turner

Re: I'm back

[Ben Wagner]

One thing that's been on my mind is around releasing (and this is a
fairly general observation not specific to FreeType). FreeType has
generally tagged a VER-2-X-X release about once every six months and
hasn't had maintained release branches. Distros (and other users)
generally only update to tagged releases or branches and then try to
maintain a patch pile of cherry-picks for bug fixes (essentially a
release branch out of tree). In the present day any non-rolling distro
model is slightly crazy since most open source projects work this way;
the distro should be at more or less tip of tree or there will be
known public exploitable bugs which the distro maintainer probably
doesn't even know should be cherry-picked. However even rolling
distros don't generally always stay at tip of tree to avoid incidental
intermediate API/ABI breakage so tend to lean on the developers to tag
commits to bless them. All of that to say, I think it would be helpful
if FreeType (and more open source projects) tagged bugfix releases
more often. For example if there are no API changes at all since the
previous release tag a bugfix release as soon as reasonable.  It may
seem a bit silly from the developer point of view, but it means that
most users will get bugfixes much quicker. The versioned distros will
still need to cherry-pick things back to their old versions, but users
that keep up to date will benefit greatly.

I was reminded of this sort of issue this morning when I experienced
an extreme version of this issue with the 'ardour' package in debian
which is currently code from two years ago and the package didn't ship
in the bullseye release because the old code didn't work with newer
libraries even though upstream had long since fixed the issue but
there have been no new tags since.

Le ven. 24 avr. 2020 à 16:02, David Turner  a écrit :
>
> Hello freetype-devel@ list members,
>
> It's been a very very long time, but I have some free time in the coming 
> weeks to work on FreeType. Werner invited me to write a small announcement 
> here and I'm currently looking at the official bugs list.
>
> I'd like to know what are, in your opinion, the most pressing issues to work 
> on at that point?
>
> Apart from that, I had the following things in mind:
>
> - Improving / refactoring the build system a little. E.g. it should be 
> possible to simplify the rules.mk/module.mk files considerably, and 
> auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single 
> source of truth (exact format to be defined), at least the parts that deal 
> with the headers / sources / configuration headers and the module 
> dependencies.
>
> - Improve testing (unit and regression tests to be exact) There are lots of 
> possibilities here, and it will probably better to do this in small 
> incremental steps.
>
> Voila, I'd be happy to read your suggestions, Happy to be here.
>
> - David Turner

Re: I'm back

[Nikolaus Waxweiler]

Woah 

My favorite pet issue I never got around to fixing: make stem darkening 
for the autohinter be as good as the CFF variant. Also, at least basic 
stem darkening for bytecode'd TTFs. Then Qt can go all in on stem 
darkening and rendering fonts correctly. And hopefully everyone else.


Also, the build system generation thing sounds very meta. I'd prefer 
settling on a single build system (either CMake or Meson and nothing 
else) 


And even a basic testing framework sounds good. One could, if so 
inclined, even use https://github.com/rougier/freetype-py and use 
pytest and hypothesis to hammer out a test suite.

I'm back

[David Turner]

Hello freetype-devel@ list members,

It's been a very very long time, but I have some free time in the coming
weeks to work on FreeType. Werner invited me to write a small announcement
here and I'm currently looking at the official bugs list.

I'd like to know what are, in your opinion, the most pressing issues to
work on at that point?

Apart from that, I had the following things in mind:

- Improving / refactoring the build system a little. E.g. it should be
possible to simplify the rules.mk/module.mk files considerably, and
auto-generate most of the Makefiles / Jamfiles / CMakefiles from a single
source of truth (exact format to be defined), at least the parts that deal
with the headers / sources / configuration headers and the module
dependencies.

- Improve testing (unit and regression tests to be exact) There are lots of
possibilities here, and it will probably better to do this in small
incremental steps.

Voila, I'd be happy to read your suggestions, Happy to be here.

- David Turner