Re: GSOC 2024 - Proposal

[Werner LEMBERG]

Hello Mahaveer,


thanks for your proposal.  However, can you please tell us what
exactly you want to improve?  FreeType is a glyph rendering library;
usually, it takes a *glyph index* (and not a Unicode character) as an
input and returns a bitmap rendering of a glyph's outline.  Handling
Unicode-encoded input strings *must* be done in a higher-level library
like 'HarfBuzz' – we are definitely not going to add such
capabilities.

Only a very small part, namely the auto-hinter, processes a font's
Unicode character map table to gather more knowledge for hinting
glyphs.  Improving this part was a GSoC project of the last year (see
https://gitlab.freedesktop.org/freetype/freetype/-/merge_requests/316)


Werner

Re: regarding gsoc

[Werner LEMBERG]

Hello Akash,


> I am interested in the gsoc project titled "Integrate ftbench into
> FreeType's build structure".

Thanks for your interest – before you do anything, please read

  https://lists.nongnu.org/archive/html/freetype-devel/2024-03/msg4.html

since the description on https://freetype.org/gsoc.html is misleading.


Werner

Re: GSOC Proposal guidance

[Werner LEMBERG]

> My name is Mubin Ladhani and I want to contribute to your
> organization through GSOC so Can I get some guidance on making the
> proposal for GSOC for your organization so that my proposal gets
> shortlisted.

Have a look at

  https://freetype.org/gsoc.html


Werner

Re: FreeType website modernisation

[Werner LEMBERG]

> I’ve just created an account on Gitlab:
> Username: @coolmanlume

Thanks; I've invited you to be a developer of the 'freetype-web'
repository.


Werner

Re: FreeType website modernisation

[Werner LEMBERG]

Hello Manuel,


nice to hear from you!


> Lately, I stumbled upon the FreeType webpage.  I cannot help to
> think that its current design does not pay tribute or properly
> represent the quality, scale, and impact of this project anymore.

Well... :-)

> If you agree I would consider working on some of the low-hanging
> fruits that would re-align FreeType's web design with the current
> zeitgeist:
> 
> • Reform the Logo
> • Amend the webpage typography
> • Slightly tweak the webpage layout
> • Slightly modernize the CSS to make the content mobile-friendly
> 
> The first criterion would be to keep the footprint very small (In
> the ballpark of what is currently online).

This sounds great!

> If interested, I would need to fork the current website, somehow,
> so that I can work on a proposal.

It's a bit more convoluted to fork FreeType on freedesktop.org since
you have to request permissions.  I suggest instead that you register
on

  https://gitlab.freedesktop.org/users/sign_up

and I give you developer permissions for

  https://gitlab.freedesktop.org/freetype/freetype-web

so that you can create a branch directly in our repository.  To do
that, I need your registered user name.  Would that be OK for you?


Werner

Re: Integrate ftbench into FreeType's build structure(s)

[Werner LEMBERG]

Hello Peter,


thanks for your interest in FreeType.

> I'd like to work on this project: Integrate ftbench into FreeType's
> build structure(s).  I have implemented both approaches mentioned
> therein.
> 
> I modified ftbench to save its screen output to a file, which is
> then processed by my module to produce an HTML file. The code can be
> found here: pgfjr/ftbench-html: ftbench with HTML output
> (github.com).  Another
> version also modified ftbench to output an HTML file directly.  The
> code is here: pgfjr/ft-bench-html-v.2
> (github.com).
> 
> I have attached the output of the latter for your viewing. Both HTML
> outputs, of course, still need polishing, which I"m more than glad
> to do if I'm selected for this project.
> 
> I look forward to hearing from you soon.

Before doing any coding you should have contacted us.  The 'ftbench'
project was already tackled by a student last year, please look up the
many, many e-mails regarding this topic in our mailing list archive at

  https://lists.nongnu.org/archive/html/freetype-devel/

If you read the e-mails you will find out that the student (Ahmet
Goksu) has implemented almost exactly the same as you've shown to us.
In other words I'm sorry that you did a lot of work for something that
already exists.

  
https://gitlab.freedesktop.org/freetype/freetype-demos/-/tree/gsoc-2023-ahmet?ref_type=heads

The very problem, however, that couldn't be solved then, and which is
the topic of this year's 'ftbench' project, is to refine the results
so that the timing values become meaningful.  Right now, the timings
between identical runs vary so much that they are completely
meaningless, not helpful in any way.

We would be glad if you were interested in this topic.  Be warned,
however, that it is *not* trivial task.

[And yes, we should update the GSoC page with this information, sorry
for that.]


Werner

Re: GSOC Contribution

[Werner LEMBERG]

Hello Claire,


> I would like to become a contributor to your organization. How do I
> become a contributor?

FreeType is a library to render glyphs of a font, written in the
C programming language.  Becoming a 'contributor' essentially means
that you have experience with handling such a library, doing some
programming work.  You might have a look at

  https://freetype.org/gsoc.html

that lists some idea – if you have more suggestions, please write to
the 'freetype-devel' mailing list.


 Werner

Re: FT_Select_Size

[Werner LEMBERG]

>> `available_sizes` is mentioned in the documentation of
>> `FT_Select_Size` ??? it seems that you've missed that, so I wonder
>> whether you have a suggestion to improve the documentation...
> 
> Personally I would add a get-function to provide this information.
> I was also looking for a function-call and not something part of a
> structure.

We won't add such a function – FreeType is an old library with an old
API, and it doesn't make sense IMHO to blow up the interface with
'modern' getters for each and any variable in `FT_Face` (and all the
other, big structures).  I considuer such functions more suitable for
object-oriented languages like C++.

Anyway, you've not answered my question, so I ask again: Do you have a
suggestion how to improve the *documentation* (and not the API)?


Werner

Re: FT_Select_Size

[Werner LEMBERG]

> FT_Select_Size selects a size by an index.
> My question now is: how/where can I see what those sizes are?

Look at the `available_sizes` array in `FT_Face`; the field
`num_fixed_sizes` gives the number of available strikes.

`available_sizes` is mentioned in the documentation of
`FT_Select_Size` – it seems that you've missed that, so I wonder
whether you have a suggestion to improve the documentation...


Werner

Re: Question regarding gSoc 2024

[Werner LEMBERG]

Hello Nitish,


> As I was exploring the idea list, I noticed that the build system
> project was listed as an idea for GSoC 23. I'm interested in this
> project, but I'm uncertain if the ideas listed on
> https://freetype.org/gsoc.html are intended for GSoC 24. At the top
> of the page, it states, "The FreeType project participates in Google
> Summer of Code 2023. Here is our list of possible projects." If
> these ideas are not for GSoC 24, could you please direct me to where
> I can find the ideas for GSoC 24?

The ideas list hasn't been updated yet, sorry.

> Additionally, I'm curious if there are any communication channels
> like Discord or Slack available for FreeType, where contributors can
> discuss project ideas and collaborate effectively.

For the FreeType community we exclusively use the 'freetype-devel'
e-mail list.


Werner

merging Craig's GSoC 2023 auto-hinter code

[Werner LEMBERG]

Craig and others,


have a look at the Merge Request

  https://gitlab.freedesktop.org/freetype/freetype/-/merge_requests/316

This is a completely cleaned-up version of Craig's
'gsoc-craig-2023-final' branch, ready for being included into
'master'.

I've heavily formatted the code to be in sync with other FreeType
code, removed some unused stuff, cleaned up code here and there, and
re-generated the commits in smaller logical chunks from scratch to
make them more comprehensible.  Hopefully, I haven't screwed up the
code while doing so.

Please check and test!


Werner

Re: [PATCH] src/sfnt/ttcolr.c: fix build in C90 mode

[Werner LEMBERG]

> See attached patch

Applied, thanks!


Werner

Re: The current state of rendering and overlap

[Werner LEMBERG]

> I'm afraid I don't have an exact overlap-less example handy because
> the equivalent static font also hints a bit differently, hiding the
> issue. But I'm referring to the fact that the upper serifs cover the
> upper parts of the stems but the pixels above the stems are darker, so
> the overlapping area appears to be double-counted.

Thanks.  I'm not sure that this can be improved at such low
resolutions.

Alexei?


Werner

Re: The current state of rendering and overlap

[Werner LEMBERG]

> I've been looking into various FreeType issues related to the
> combination of CFF2 and hinting. As part of that I ran into the
> phenomenon represented in the attached capture of ftgrid. Which, to
> summarize, has a rendering artifact caused by overlap.

What exactly is the artifact?  I don't see it.  Do you have a
comparison image of the same outline without overlaps?


Werner

GSoC has ended

[Werner LEMBERG]

Folks,


GSoC 2023 is over; a few days ago the last project has finished – all
FreeType students have passed :-)

We had three projects:

* Ahmet's code to improve 'ftbench' so that it can emit benchmarking
  test reports:


https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-2023-ahmet-final

* Anurag's code to port 'font-rs', an alternative rendering engine, to
  FreeType and compare it to the built-in rasterizer; and to convert
  some parts of FreeType's documentation to Markdown


https://docs.google.com/document/d/14me6L4HEMnjBVFwOCjaBayRjQym-WK_RQuOJt7A3pCY/edit

https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-anurag-2023-final

https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-anurag-2023-docs-final

* Craig's auto-hinter code to implement a 'capabilities database' so
  that glyphs like the 'tilde' accent stay legible at smaller sizes:


https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-craig-2023-final

I want to say thank you to Ahmet, Anurag, and Craig – it's great to
have people like you who want to help improve FreeType – and wish you
all the best for your future careers!


Werner

Re: [Inquiry] Auto-fit on monochrome output

[Werner LEMBERG]

>> ahh it's on the very legacy display, which is monochrome at 72dpi
>> we have no choice but to find out what can be done to improve the
>> appearance
> 
> Hand-tuned bitmap fonts.

Yeah, and you should additionally use well-hinted Windows fonts that
contain support for monochrome rendering.  In most cases you are on
the safe side if you choose fonts that already existed in the XP era.
Of course, if you need support for more 'exotic' languages this route
will probably be unsuccessful, alas.

Check the 'gasp' table in the fonts, for example with `ttx -t -gasp`.

  https://learn.microsoft.com/en-us/typography/opentype/spec/gasp
  https://pypi.org/project/fonttools/

For example, Windows's `pala.ttf` font comes with monochrome hinting
support in the range 9 to 17 ppem, even today, AFAICS.


Werner

Re: [Inquiry] Auto-fit on monochrome output

[Werner LEMBERG]

>> Define 'improve hinting'.  It's not clear to me what you actually
>> want to achieve.
>
> Currently we are using FT_LOAD_RENDER | FT_LOAD_TARGET_MONO |
> FT_LOAD_MONOCHROME and still seeing very bad appearance.  So not
> sure if I missed anything there in order for the monochrome
> character to look nicer?

This is not what I mean.  I want to know *why* you need this very
special type of rendering.  What's the purpose?  Today, it's quite
unusual that you want monochrome rendering (at small sizes, I guess).


Werner

Re: [Inquiry] Auto-fit on monochrome output

[Werner LEMBERG]

>> No, (good) auto-hinting is not available for monochrome display.
> 
> Does this mean even if I load the glyph with FT_LOAD_FORCE_AUTOHINT,
> the autofit algorithm also will not perform any hinting?

It will perform hinting but expect very ugly results.

> I'm using the TrueType font, any way to improve the hinting other
> than FT_LOAD_TARGET_MONO?

Define 'improve hinting'.  It's not clear to me what you actually want
to achieve.

>> A segmentation fault shouldn't happen; I guess you are forgetting
>> to check a return value somewhere.  If the problem persists please
>> provide a small stand-alone code snippet for testing purposes that
>> I can compile and run on the command line.
> 
> bad case : Segmentation fault "potentially unexpected fatal signal 11"
>  -->tried with combination of
> FT_LOAD_RENDER | FT_LOAD_FORCE_AUTOHINT
> FT_LOAD_RENDER | FT_LOAD_TARGET_MONO | FT_LOAD_FORCE_AUTOHINT
> FT_LOAD_RENDER | FT_LOAD_MONOCHROME | FT_LOAD_FORCE_AUTOHINT

Thanks, but these images don't help in any way to identify a possible
bug in FreeType.  As I said, I need a small, compilable, stand-alone
snippet (written in C) that runs on the command line and exhibits the
problem.  This means some work on your side to reduce your code as
much as possible – you are done if removing anything from the example
no longer triggers the problem.


   Werner

Re: [Inquiry] Auto-fit on monochrome output

[Werner LEMBERG]

> I am exploring this FreeType font engine library, am wondering if the
> auto-fit(auto-hint) support for monochrome display as well?

No, (good) auto-hinting is not available for monochrome display.

> I get segmentation fault whenever I use this flag ->
> FT_LOAD_FORCE_AUTOHINT
> 
> FT_LOAD_RENDER | FT_LOAD_MONOCHROME | FT_LOAD_FORCE_AUTOHINT

A segmentation fault shouldn't happen; I guess you are forgetting to
check a return value somewhere.  If the problem persists please
provide a small stand-alone code snippet for testing purposes that I
can compile and run on the command line.


Werner

Re: Information Request for FreeType

[Werner LEMBERG]

Hello Jennifer,

>   1. Is there an organization which sponsors/publishes the project,
>  or a primary developer who audits the code for potential
>  vulnerabilities, errors, or malicious code? Y/N

There is no organization, but at least two people (Alexei und me)
continuously monitor changes in the code – which we usually commit by
ourselves – and take care of error reports, including potential
vulnerabilities.

>   2. We have identified contributors on GitHub from the following
>  locations: Austria, United States, Canada, United Kingdom,
>  Spain, India, Netherlands, Germany, France, Russia, Turkey,
>  Switzerland, Latvia, Argentina, Hong Kong, Japan, Australia,
>  and China.
>  https://github.com/freetype/freetype/graphs/contributors

If you add Singapore it looks correct.  However, I haven't checked
this in full detail.


Werner

Re: FT Cache Subsystem: Custom caches

[Werner LEMBERG]

> I'm trying to use FreeType cache subsystem to render glyphs. I tried
> using FTC_SBitCache but I need to modify the outline before
> rendering (translate and transform) so I could only use
> FTC_ImageCache, modify the outline and render it.
> 
> Is there a way to extend the cache mechanism to create my own cache
> that supports all these features?

Extend in which way?  It's not clear to me what you want to cache...


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> There was an intentional change to src/base/ftobjs.c.  I made the
> function that finds the best unicode charmap public so that I could
> use it from the autofitter.

OK, I see.  I was confused because the `FT_BASE_TAG` is missing for
this function in `ftobjs.c`.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> I have added and tested type 3 lookup handling, and also added the
> comments you asked for.

Thanks.  Minor nits that I've seen while skimming through:

  s/indicies/indices/
  s/flatenning/flattening/

> I will begin on the final branch if there's nothing else to be done.

OK.  Please rebase to master before you start.

And no accidental changes to `freetype/config/ftoption.h` and
`src/base/ftobjs,c`, please :-)


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> I need some help figuring out how to handle the type 3 lookups.
> I need to do 2 things:
> - Figure out which features contain type 3 lookups
> - Determine the number of variations the feature contains

Simply add all glyphs a type 3 lookup provides to the list of glyphs.
No further special handling is necessary.

> For the second one, this function seems relevant:
> https://harfbuzz.github.io/harfbuzz-hb-ot-layout.html#hb-ot-layout-feature-with-variations-get-lookups
> But this returns a list of lookups if you already know the variation
> index, when I want to know the range of possible variation indices.

This function is not relevant – it's about variation fonts, see

  
https://learn.microsoft.com/en-us/typography/opentype/spec/chapter2#device-and-variationindex-tables

> hb_ot_layout_lookup_get_glyph_alternates
> also looks useful and could partially solve the problem.

This is the right function, I think.

> With this function, I can handle the type 3 lookup cases in
> isolation, only finding the glyphs directly resulting from the
> feature and no further transformations.

Sounds sufficient to me.

> Also, can I have some advice on testing the code?  How should I make
> these changes bulletproof?

Alas, we don't have a testing framework for such issues.  However, as
soon as your code lands in 'master', the Chromium people and other
parties run their fuzzers on FreeType, which usually unveils memory
leaks or segfaults quite soon.  They also do intensive comparison of
graphic images; however, I don't know whether they use the auto-hinter
for that.


Werner

Re: [IDEA] Facilities to extract TT fonts from TTC font collections

[Werner LEMBERG]

> I recently had to extract a TrueType font from a TTC font collection
> as part of fixing bugs in the podofo library[1].

In general, FreeType is not the right tool for such operations.
FreeType is a *rendering* library that provides a thin API for
extracting SFNT tables, mainly the ones FreeType isn't parsing by
itself.  As Behdad mentioned, the `fonttools` package should help you
with your task.

> For example, a special tag TTAG_tdir
> could be defined to access the raw Table Directory data with
> FT_Load_Sfnt_Table, like the following:
> 
> FT_ULong size = 0;
> FT_Load_Sfnt_Table(face, TTAG_tdir, 0, NULL, );

Please file a feature request at

  https://gitlab.freedesktop.org/freetype/freetype/-/issues/


 Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

>> I simply noticed that it's possible for 2 characters to map to the
>> same glyph, which means that a glyph would map to 2 characters.  I
>> don't have any examples in mind for when this would actually
>> happen.  I was planning on either ignoring the situation to let it
>> be resolved arbitrarily, or removing both entries.
> 
> The situation is resolved arbitrarily for now.

OK.  Minor nit: Please avoid overlong git commit messages (i.e., not
longer than 78 characters).  And there should be an empty line after
the first line to help tools like `gitk` to properly display git
commits.  [Overlong lines should be avoided in the C code, too, both
for comments and code.]

> Also: what else needs to be done for the project to be complete and
> ready to become a part of freetype?  The remaining tasks I can think
> of are:
>
> - Fill in, or find someone to fill in the rest of the adjustment
>   database.

This is certainly helpful.  However, it doesn't need to be complete
right now, but it should cover a good share of languages that use the
Latin script.  BTW, please add a comment to the `adjustment_database`
array, explaining the format.

> - properly address the 'salt' table and similar cases in the glyph
>   alternative finding algorithm.
> - Test everything more thoroughly.

Sounds good, thanks.  I also request you to produce a 'final' GSoC
tree with cleaned-up commit messages, as mentioned in other e-mails to
this list to other GSoC participants.

> At this point, I know that the segment removal + vertical stretch is
> definitely the best approach, and the latest commit applies that to
> all the characters with tildes rather than a comparison of
> approaches.  I previously thought that it caused some regressions,
> but I now know that the examples I had were just preexisting quirks
> in either the font or the autohinter.

Excellent!  I think it would also be beneficial if you could mention
your findings in either a git comment or in the code itself, together
with some real-world examples of such quirks (i.e., font name, font
version, glyph name, reason why it fails, etc., etc.)


Werner

Re: Progress update on alternative rendering engines project

[Werner LEMBERG]

> I don't think this (caching flatten outlines) is a good idea.

Honestly, I don't have an opinion on this.

> Pretty much all serious implementations cache the rendered image.
> And pretty much all uses of FreeType don't render the entire glyph
> set.  And must implementations care a lot more about loading
> performance.  I know you are suggesting new API.  I just don't see
> how this is useful, unless I'm misunderstanding it.

The basic question is whether there is a *real* advantage in
comparison to the traditional way of calling FreeType that is so big
that the effort of having another API is worth it.

What use cases do you have in mind?  Where would your suggestion API
really shine?

>> This approach also does not support font hinting (as is the case
>> with fontdue) and I am unsure if/how it could be supported.

Assuming a new API this is not a big issue IMHO.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> > OK.  I think it is a bad side effect of the current auto-hinting
> > algorithm that there are different approaches.
> 
> I just want to clarify: you understood that the reason I used
> different approaches for each letter was to compare the approaches?
> My intent is to use one of those approaches as a universal algorithm
> for all characters with tildes.  So every character would just have
> a boolean flag for whether to apply tilde hinting or not.

Ah, I was confused, sorry – I thought that you get such varying
results for a single algorithm.  Sometimes it happens (not taking your
current work into account) that blue zones affect the hinting of
accents in a bad way, and I thought this were such cases.

> Also, did you see my question about a glyph mapping to multiple
> characters?

I missed it, sorry again.  You write:

> It's possible that 2 characters in the adjustment database could map
> to the same glyph, which will create 2 entries in the reverse
> character map with the same glyph as a key.  In this case, the
> character that glyph maps to is decided arbitrarily based on which
> one the binary search chooses and which order qsort puts them in.
> What should be done in these cases?

Perhaps the following?

(1) If glyph A is in the 'cmap' table, and glyph B is not, prefer
glyph A.

(2) If one glyph needs X lookups and another glyph needs Y, and X < Y,
prefer glyph X.

I'm not sure whether (2) makes sense, though.

Can you give one or more examples for such cases?


Werner

Re: LilyPond support for `festival`

[Werner LEMBERG]

> I wonder what we should do with the 'festival' and 'lilysong' files
> (and related stuff) in LilyPond.  Please have a look at
> 
>   https://gitlab.com/lilypond/lilypond/-/issues/1245
> 
> This stuff is completely undocumented...

Oops, I've sent this to the wrong e-mail list, sorry.


Werner


PS: And in case you are curious: The author recommends to delete the
stuff :-)

Re: Progress update on adjustment database

[Werner LEMBERG]

> > Thanks.  Do you have meanwhile found an explanation why o-tilde
> > looks so bad for Times New Roman at 16ppem?
>
> All 4 letters in each row have a different approach:
>
> õ: vertical stretch, no segment removal
> ñ: no vertical stretch, segment removal
> ã: vertical stretch and segment removal
> all other tildes: no changes applied

OK.  I think it is a bad side effect of the current auto-hinting
algorithm that there are different approaches.  However, using the
adjustment database I wonder whether the knowledge of the character
topology can help improve the situation.  In other words, do you see a
possibility to 'decouple' the (vertical) hinting of the tilde from the
base glyph hinting by checking a flag in the database?  For this
purpose, a 'tilde' could be defined as the contour that lies higher
than the ascender of small letters – this implies that you need
another flag or enumeration to refer to small letter, uppercase
letters, etc.

As an example, the database information for glyph 'o tilde' could be

  * lowercase character
  * hint contour(s) higher than the lowercase ascender hight
separately
  * stretch tilde vertically

> I implemented the algorithm for all glyph variants!  The version I
> used is different from what I wrote originally to fix some errors.

Looks good.  To help people understand the non-trivial algorithm I
suggest that you add a big comment that shows it working step by step
for an example font, using a reduced set of features and glyphs.

> I've only tried it on a pretty simple case so far, so I'll need to
> assemble a more complex test font or two.

A feature-rich (and freely available) font family is 'Libertinus', for
example.


Werner

Re: feedback

[Werner LEMBERG]

> I compiled splashutils-1.5.2 (sourceforge) on a Linux Lite 3.8
> 32-bit (frugal install) without any modifications successfully.  I
> hope, these informations help the developers.

Not really.  It's still not sufficient.  Actually, I think your
compiler installation or compilation environment is broken since we've
never received such a report.

Have a look into directory `builds/unix` and send the file
`config.log` as a compressed attachment.


Werner

LilyPond support for `festival`

[Werner LEMBERG]

Hello Milan,


I wonder what we should do with the 'festival' and 'lilysong' files
(and related stuff) in LilyPond.  Please have a look at

  https://gitlab.com/lilypond/lilypond/-/issues/1245

This stuff is completely undocumented...


Werner

Re: ftbench: last updates

[Werner LEMBERG]

> I hope my effort contribute to you. I wish to see you again.

:-) You are invited to continue your work even after GSoC has ended.

> Here is my last version of readme, [...]

Uh, oh, the formatting in the e-mail is completely broken...

> i will push submit my final submission regarding your feedback.

Ideally, you apply formatting and indenting changes to, say, the
fourth commit from the top (within your branch) by doing the
following:

```
git rebase -i HEAD   # interactive rebase


git add -p   # add diffs one by one
git commit --amend   # rewrite current commit
git rebase --continue# continue rebase
git push -f  # force-push
```

Currently, there is a bunch of whitespace commits on top of the
cleaned-up commits.  However, the idea of a cleaned-up tree is that
such formatting changes are *not* present – please bear in mind that
we don't squash the commits into a single one!

I would be glad if you could remove the whitespace commits and include
the whitespace fixes in the actual commits instead (don't worry about
today's GSoC deadline, this is cosmetics only).

> (also waiting for *-demos code review, i will push them too)

Thanks.


Werner

Re: ftbench: last updates

[Werner LEMBERG]

Hello Ahmet,


sorry for the late reply, I'm travelling right now.

> -Made documentation and comment line (will continue).

Very nice!  Some minor nits, in case you have still time today:

* Please ensure that lines in the documentation are not longer than 78
  characters (`src/tools/ftbench/README`).  Ideally, `README` comes in
  Markdown format, but this is not important.

* Use two spaces after a full dot in documentation (this simplifies
  the search for the beginning of a sentence with some editors like
  Emacs).

* Use `autopep8 -a -a -a` to format `tohtml.py` and do some manual
  editing to also shorten the remaining overlong lines to
  78 characters per line if possible and useful.

* There are some indentation errors in `ftbench.c` (functions
  `test_render`, `test_embolden`, etc.).

* Please synchronize `ftbench.c` with the git version in the
  `freetype-devel` repository (which is a trivial change only).

> In this version of the code, i just didnt splitted iterations into
> chunks, got same results with it.

OK.  Too bad that we couldn't find the real cause for the large time
differences, but hey, this is a GSoC project :-)


  Werner

Re: Facing very different type of issue after porting freetype

[Werner LEMBERG]

> I'm facing some issues after porting.  while I'm passing unsigned
> char Ubuntu[34816] which is Ubuntu-R.ttf file.  By using bin2c
> python script i converted into an C array.  while is passed this
> array to the function i'm getting all those error.  [...]

Please provide a *complete*, compilable, minimal working example
(MWE).  We can't help you otherwise.  Ideally, you should use a very
zsmall font for testing so that the example `.c` file stays small, too.


Werner

Re: Facing very different type of issue after porting freetype

[Werner LEMBERG]

> I'm facing some issues after porting.  while I'm passing unsigned
> char Ubuntu[34816] which is Ubuntu-R.ttf file.  By using bin2c
> python script i converted into an C array.  while is passed this
> array to the function i'm getting all those error.  [...]

Please provide a *complete*, compilable, minimal working example
(MWE).  We can't help you otherwise.  Ideally, you should use a very
zsmall font for testing so that the example `.c` file stays small, too.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> Is testing all these combinations really necessary?

I don't know :-) I just wanted to point out that feature combinations
have to be considered.

> [...] My intuition says very few of these combinations actually
> matter.

Yes, I agree.

> I wrote some pseudocode for a different approach that I believe
> accomplishes the same thing, while being more efficient and hopefully
> removing the need to constrain the set of features considered: [...]

Sounds good.  Unfortunately, I'm a bit short of time right now; I'll
think about your algorithm within the next few days.  However, please
proceed anyway!

> I attached some pictures of the tilde unflattening approaches.

Thanks.  Do you have meanwhile found an explanation why o-tilde looks
so bad for Times New Roman at 16ppem?

> I chose sizes that showcase the differences between the approaches,
> and also committed my current code if you would like to try it
> yourself.

Will try if I find some time.


Werner

Re: ftbench: last updates

[Werner LEMBERG]

Hello Ahmet,


> -I have changed the * and the sentence

Thanks.  Unfortunately, I was unclear that '*x*' in my e-mail is meant
as Markdown syntax and not to be taken verbatim.  In other words, 'x'
should be typeset in italics, similar to a mathematical variable.
Sorry for that, and please fix.

Something different: Please don't use `\r` in messages for newlines.
Use `\n` instead, which is standard (file `testing.mk`).

>> I already changed the working way of the timing. I only start the
>> benchmark at beginning and stop at the end.
>
> i mean, it times chunks, not single iteration.  timer starts at the
> beginning of the chunk and stop at the end (then divide the results
> size of a chunk). because of it does not time single iteration, it
> is already a bulk test.

Ok.

>> BTW, I suggest that you add another sentence, explaining *why*
>> there are two values at all.
>
> actually, i didnt get the reason well but it may differ even with
> same flags. i need help in this case.

Alexei, please have a look.  It seems that only(?) the 'Stroke' test
is affected...

> the code seems producing more accurate results after splitting the
> results into chunks. are results seem satisfactory in your machine?

Alas, they aren't.  Running your code on my GNU/Linux machine, I get
percental differences up to 14%.

BTW, I've also just compiled `valgrind` from its git repository (to be
sure that it exactly fits my 'libc' library version) and run

```
valgrind --trace-children=yes make baseline
valgrind --trace-children=yes make benchmark
```

expecting to get identical results, because `valgrind` uses a CPU
emulator.  However, to my great surprise, they are *not* identical,
and sometimes the values differ even enormously.  Has anybody an idea
why this happens?  AFAICS from valgrind's source files, the
`clock_gettime` function *is* caught.

Note that with valgrind the creation of the webpage fails; I get

```
Traceback (most recent call last):
  File "/home/wl/ahmet/tohtml.py", line 310, in 
main()
  File "/home/wl/ahmet/tohtml.py", line 67, in main
generate_total_results_table(html_file, BASELINE_DIR, BENCHMARK_DIR)
  File "/home/wl/ahmet/tohtml.py", line 192, in generate_total_results_table
diff = ((baseline - benchmark) / baseline) * 100
ZeroDivisionError: float division by zero
```

Attached are the two directories from valgrind runs.  Comparing this
with non-valgrind values I think there is a fundamental interaction
problem with the `bench` program: It seems that `bench` controls the
number of loops *in addition* to what you specify via the command
line.  Unfortunately, I don't have time right now to check this by
myself.

Too bad that you didn't try `valgrind` by yourself earlier, as I've
asked you repeatedly to do ...

> Also, While there are less than 10 days for final evaluation, there
> are points that are not completed:
>
> * meson
> * cmake
> * documentation
>
> because of our focus a bit changed, didnt worked on them
> much. Should I complete them all? Is there a priority?

Please forget meson and cmake, this is not important at this stage.
What I now ask is to create a new branch, with *all* of your changes
recreated as logical and (relatively small) incremental commits,
rebased to 'master'.  Ideally every commit is compilable.  If not
(which should be an exception, and there should be a good explanation
for that), please add a remark in the commit message.

* Try to adjust the formatting to be as near to the remaining code of
  FreeType.

* The commit messages should also be in FreeType style, which is much*
  *more verbose than what you currently do.

* Avoid trailing whitespace – whatever editor you use, *please* change
  the settings so that this gets automatically removed.  Right now, I
  see zillions of trailing whitespace in file `tohtml.py`.

* Hin-Tak is right: Documentation is of paramount importance so that
  you (or someone else) can continue the work.

Have a look at previous GSoC students' (remote) branches to see how
such a recreated branch should look like, for example
`GSoC-2019-nikhil` or `GSoC-2017-ewaldhew`.


Werner


valgrind.tar.gz
Description: Binary data

Re: ftbench: last updates

[Werner LEMBERG]

>> With 'bulk test' I mean that you don't time single iterations but
>> do timings for 10 iterations in a group, say, thus avoiding
>> issues with the granularity of the OS timing functions.
>
> I already changed the working way of the timing.  I only start the
> benchmark at beginning and stop at the end.

Sorry, but I don't understand your last sentence, please elaborate.

>> I wouldn't use '**' – this looks like a footnote, and people start
>> searching where it belongs to. Instead, please use a simple
>> itemization.

Well, in the latest incarnation of the HTML page there is now a single
line with a '*'.  An enumeration needs at least two items, otherwise
it again looks like a footnote.  In other words, the second sentence
need to start with '*', too.

>> * The sentence about '(X | Y') is cryptic.  [...]
>
> done!

Thanks, but the current text

  If two values are given in the 'Iterations' column, those are belong
  to baseline and benchmark in order.

is not correct English.  Maybe

  If a value in the 'Iterations' column is given as '*x* | *y*',
  values *x* and *y* give the number of iterations in the baseline and
  the benchmark test, respectively.

BTW, I suggest that you add another sentence, explaining *why* there
are two values at all.

>> The links for `Baseline` and `Benchmark` to the `.txt` files are
>> absolute and thus not portable.  They must be relative.
>
> I didnt get this clear.  Since objs dir and the source dir
> different, I needed to give abs path.

Why?  Simply give one directory relative to the other!  Assuming that
you have

  baseline_dir = /foo/bar/baseline/
  benchmark_dir = /foo/bar/benchmark/

and `benchmark.html` gets built in `benchmark_dir`, the two links to
file `Roboto_subset.txt` would be

  
  

What am I missing?  Maybe

  
https://stackoverflow.com/questions/29055511/how-to-find-relative-path-given-two-absolute-paths

helps.  The benchmark should abort if it is not possible to construct
a relative path.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> So, if my understanding is correct, hb_ot_shape_glyphs_closure will
> take an input character or characters and tell me all the glyphs
> that it gets transformed into, as well as the final form.

Yes.

> I'm not sure about this interpretation, because the documentation
> uses the term "Transitive closure", which I'm not familiar with.

Indeed, it's a bit unfortunate that the documentation is not more
verbose.

> As for iterating through auto-hinter styles, do you mean that I
> should get a list of features and try each one for the 'features'
> parameter?

Yes, you should try each one, and all combinations of them.  However,
the number of features that are of interest (at least for latin
scripts) is small, which means that the number of iterations doesn't
become very large; see macro `META_STYLE_LATIN` in file `afstyles.h`
for a list.

> Also, I wanted to share my progress in the tilde unflattening.
> [...]

This sounds very promising, thanks!

> The segment removal should be part of the solution, but the question
> is to what extent the vertical stretch should be part of the
> solution.

My gut feeling says that both is needed.  I hope that you find
constraints that work reliably for a large bunch of (common) fonts.

To try to answer this, I tested on a bunch of fonts.
> [...]

Please also post some images.


Werner

Re: to contribute to freetype

[Werner LEMBERG]

> Hey, I am a newbie to open source , I only know c , want to
> contribute to your organization . help me

This question has been asked many times.  Please visit and search the
e-mail archives at

  https://lists.nongnu.org/archive/html/freetype-devel


Werner

Re: ReactOS: stack vs heap

[Werner LEMBERG]

> Thanks. One thing I don't understand, is the "+ 2" in the code below:
>
>   AF_LatinBlue  blue_sorted[AF_BLUE_STRINGSET_MAX_LEN + 2];
>
>   for ( i = 0; i < axis->blue_count; i++ )
> blue_sorted[i] = >blues[i];
>
> If that + 2 is correct to be there, then we need a similar + 2 in
> aflatin.h for blues. But I cannot see why it's needed.

Good catch!  This is due to a synchronization error between FreeType
and ttfautohint: I've originally developed the code for ttfautohint,
which adds two artificial blue zones, then imported it back into
FreeType, where I forgot to adjust the array size accordingly.

Now fixed in git, thanks.


   Werner

Re: ReactOS: stack vs heap

[Werner LEMBERG]

> Upon further investigation, I think my proposed change is correct.

Thanks a lot, committed!


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> I discovered there was an issue when I tried using a test font I created
> with the following as one of its lookups (in ttx format):
> 
> 
> 
> 
> 
>   
>   
> 
>   
> 
> There is one lookup here, but 2 substitutions.  My program needs to
> iterate through each substitution individually, but the function
> hb_ot_layout_lookup_collect_glyphs only returns 2 unordered sets
> representing all the input and output characters for this lookup.
> How can I get one substitution at a time?

Indeed, you can't use use `hb_ot_layout_lookup_collect_glyphs` for
that.  However, given that you actually want to map input character
codes to output glyph indices, what about using
`hb_ot_shape_glyphs_closure` on single input characters, iterating
over the auto-hinter 'styles'?  If you get a single output glyph,
everything's fine.  If you get more than a single output glyph this
essentially means that two or more lookups have been applied in
succession, but you only have to take care of the glyph that is part
of the 'style'.

Note that this is untested on my side – I was just searching in the
HarfBuzz API.

Behdad, if you have a better idea, please chime in :-)


Werner

Re: ReactOS: stack vs heap

[Werner LEMBERG]

>> Perhaps a hybrid approach is the right one: Use the current
>> infrastructure up to a certain size, being as fast as possible because
>> dynamic allocation overhead can avoided, and resort to dynamic
>> allocation otherwise.
>
> FreeType is not shy about allocating buffers to load a glyph. This
> is just one more I highly doubt that it matters even at small
> sizes. We always allocate FT_Bitmap even for rendering too.  As a
> matter of fact FreeType loses to the dense renderers when rendering
> complex glyphs precisely because of multiple restarts to fit the
> small buffer.

Well, I thought of a solution similar to Behdad's approach in commits
6f16b10019d7 and 56ddafa01ce2...

However, I leave that to you, since you've worked much more
intensively on the rendering code than me.


   Werner

Re: ReactOS: stack vs heap

[Werner LEMBERG]

> I will try the dynamic heap allocations for the rendering
> buffer. This might be the largest of them, I think. In addition,
> this should help with the rendering speed when rendering complex
> shapes like
> https://fonts.google.com/specimen/Cabin+Sketch. Currently, FreeType
> makes several attempts until a sub-band can fit into a static stack
> buffer. We should be able to fit it into a dynamic buffer easily. I
> wonder if CabinSketch should be about as complex as we can tolerate
> and refuse anything much more complex than this. A lot of time-outs
> will be resolved...

Perhaps a hybrid approach is the right one: Use the current
infrastructure up to a certain size, being as fast as possible because
dynamic allocation overhead can avoided, and resort to dynamic
allocation otherwise.


Werner

Re: -warmup

[Werner LEMBERG]

> Something else: I think that the 'TOTAL' line doesn't make sense
> right now.  Please separate this line slightly from the rest of the
> table and print the *cumulated timing* (in 's', not 'µs') of all
> tests, something like
> 
>   Total duration for all tests: 25.3s
> 
> and
> 
>   Total duration of tests for 'Roboto_subset.ttf: 3.4s

I've just noticed other minor issues with the HTML page:

* The links for `Baseline` and `Benchmark` to the `.txt` files are
  absolute and thus not portable.  They must be relative.

* I wouldn't use '**' – this looks like a footnote, and people start
  searching where it belongs to.  Instead, please use a simple
  itemization.

* I suggest to rename column 'N' to 'Iterations' or '#Iterations' (or
  something similar).

* The sentence about '(X | Y') is cryptic.  Maybe simply start with

If two values are given in the 'N' column, ...

  and please add an explanation *why* there are two values at all.


Werner

Re: -warmup

[Werner LEMBERG]

>> I still think that for such cases the number of iterations of the
>> affected tests should be increased to get more precise values.
>
> the times are for single iteration. (chunk median/chunk size)

Yes, but the number of iterations is the same regardless whether a
test takes 10µs or 1000µs – I suspect that it is necessary to either
increase the number of iterations for the former case or to do bulk
tests.  With 'bulk test' I mean that you don't time single iterations
but do timings for 10 iterations in a group, say, thus avoiding issues
with the granularity of the OS timing functions.

I refuse to believe that we have to live with timing differences of
more than 50%.

BTW, have you checked whether replacing `CLOCK_REALTIME` with
`CLOCK_MONOTONIC` gives better results?

What platform do you actually develop on?  What #if clause in
function `get_time` of `ftbench.c` is run on your computer?


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> As I was testing my attempt at supporting GSUB lookups, I found that
> hb_ot_layout_lookup_collect_glyphs, is actually not what I need,
> because I assumed that each lookup contains exactly one
> substitution, when it actually may contain multiple.  What I really
> need is a way to get each individual substitution.  How do I do
> this?

It's not completely clear to me what you exactly need, please give an
example.

Behdad, any idea whether HarfBuzz can help here?  Otherwise it is
probably necessary to parse the GSUB table by ourselves, which is
something I would like to avoid...


Werner

Re: -warmup

[Werner LEMBERG]

> here is the results with chris’ suggestion. (thanks chris)

Much better, thanks!

> still a bit noise on only load and load_advances.  are results
> acceptable?

As far as I can see, the biggest differences occur if the 'Baseline'
and 'Benchmark' columns contain very small values.  I still think that
for such cases the number of iterations of the affected tests should
be increased to get more precise values.  Please try that.

Something else: I think that the 'TOTAL' line doesn't make sense right
now.  Please separate this line slightly from the rest of the table
and print the *cumulated timing* (in 's', not 'µs') of all tests,
something like

  Total duration for all tests: 25.3s

and

  Total duration of tests for 'Roboto_subset.ttf: 3.4s


 Werner

Re: -warmup

[Werner LEMBERG]

>> Should I proceed to detect outliers?  Since we do not get the same
>> error rate consistently, I think we will not find the target we
>> expected by outliers.
> 
> Why do you think so?  Please explain your reasoning.  Just remember
> that backup processes (like cleaning up the hard disk, running some
> cron jobs, etc.) can pop up anytime, thus influencing the result.
> Such spontaneous events have to be eliminated.
> 
> Have you actually tried something along the method I suggested?

BTW, here is another repository that provides a framework for
benchmarks:

  https://github.com/sharkdp/hyperfine

It looks quite nice, AFAICS – maybe you can check its code for
identifying outliers and the like.


Werner

Announcing FreeType 2.13.2

[Werner LEMBERG]

FreeType 2.13.2 has been released.

It is available from

https://savannah.nongnu.org/download/freetype/

or

https://sourceforge.net/projects/freetype/files/

The latter site also holds older versions of the FreeType library.

See below for the relevant snippet from the CHANGES file.

Enjoy!


   Werner


PS: Downloads from  savannah.nongnu.org  will redirect to your nearest
mirror site.   Files on  mirrors may  be subject to  a replication
delay   of   up   to   24   hours.   In   case   of  problems  use
https://download-mirror.savannah.gnu.org/releases/


--


https://www.freetype.org


FreeType 2  is a software  font engine that  is designed to  be small,
efficient,  highly   customizable,  and  portable   while  capable  of
producing high-quality output (glyph images) of most vector and bitmap
font formats.

Note that  FreeType 2 is  a font service  and doesn't provide  APIs to
perform higher-level features, like text layout or graphics processing
(e.g.,  colored  text  rendering,  'hollowing',  etc.).   However,  it
greatly simplifies these tasks by providing a simple, easy to use, and
uniform interface to access the content of font files.

FreeType  2  is  released  under  two open-source  licenses:  our  own
BSD-like FreeType  License and the  GPL.  It can  thus be used  by any
kind of projects, be they proprietary or not.


--


You  can use  a  `.sig` file  to verify  that  the corresponding  file
(without the  `.sig` suffix)  is intact.  First,  be sure  to download
both  the `.sig`  file and  the  corresponding archive.   Then, run  a
command like this:

  gpg --verify freetype-2.13.2.tar.gz.sig

If that command fails because you  don't have the required public key,
execute

  gpg --keyserver pgp.mit.edu --recv-keys BE6C3AAC63AD8E3F

to import it, then rerun the `gpg --verify` command.

SHA1 file checksums:

7a26f7f2174f257afbfd4c88ec874621a0a84ea9  freetype-2.13.2.tar.gz
2d8d5917a1983ebd04921f2993a88858d6f72dec  freetype-2.13.2.tar.xz
07d34040d9f98b0995a21ceb2421ed0ef8cffd1d  ft2132.zip
85284169bea0bbcffce0c4bc8a0be57288121e7f  ft2demos-2.13.2.tar.gz
655c82a431fae7f53a964bda8a7c0671531d05a4  ft2demos-2.13.2.tar.xz
b33cfa859563f1d269dc29273d972f4b65eb44e2  ftdmo2132.zip
cac82fb66405e53cbad6f93c07211befe5bbe3fd  freetype-doc-2.13.2.tar.gz
dbed086b3dba1d748e15b28103081ed30d24e3f3  freetype-doc-2.13.2.tar.xz
1cac5b0411773555470ca0b2b2a02c34362e178d  ftdoc2132.zip

SHA256 file checksums:

1ac27e16c134a7f2ccea177faba19801131116fd682efc1f5737037c5db224b5  
freetype-2.13.2.tar.gz
12991c4e55c506dd7f9b765933e62fd2be2e06d421505d7950a132e4f1bb484d  
freetype-2.13.2.tar.xz
b7e5b03d2e890c4a881e9fab5870463a37fc9cb934c886b9aab2f6fd637ae783  ft2132.zip
f4a2d44f19af121ff62421d989d03b558c374613c14fefc2b0db4eaf91fcafbf  
ft2demos-2.13.2.tar.gz
99ee2ed8b98bcfad17bc57c2d9699d764f20fe29ad304c69b8eb28834ca3b48e  
ft2demos-2.13.2.tar.xz
de326cd84513c8fa54aed2f570588adcc045a8176c079089e31d0d2650bbd468  ftdmo2132.zip
f5f18802e5fc64769c5918a21e44e7c15f7141fcf4e26d87368a22bcd145bd45  
freetype-doc-2.13.2.tar.gz
685c25e1035a5076e5097186b3143b9c06878f3f9087d0a81e4d8538d5d15424  
freetype-doc-2.13.2.tar.xz
6c0e33138688d171b3b445eb2a404c83c35dde836265f9dad47666a301388e06  ftdoc2132.zip


CHANGES BETWEEN 2.13.1 and 2.13.2 (2023-Aug-25)

  I. MISCELLANEOUS

  - Better support for CFF2 variation fonts.

  - TrueType interpreter  version 38 (also known  as 'Infinality') has
been removed.

  - Improved OpenVMS support.

Announcing FreeType 2.13.2

[Werner LEMBERG]

FreeType 2.13.2 has been released.

It is available from

https://savannah.nongnu.org/download/freetype/

or

https://sourceforge.net/projects/freetype/files/

The latter site also holds older versions of the FreeType library.

See below for the relevant snippet from the CHANGES file.

Enjoy!


   Werner


PS: Downloads from  savannah.nongnu.org  will redirect to your nearest
mirror site.   Files on  mirrors may  be subject to  a replication
delay   of   up   to   24   hours.   In   case   of  problems  use
https://download-mirror.savannah.gnu.org/releases/


--


https://www.freetype.org


FreeType 2  is a software  font engine that  is designed to  be small,
efficient,  highly   customizable,  and  portable   while  capable  of
producing high-quality output (glyph images) of most vector and bitmap
font formats.

Note that  FreeType 2 is  a font service  and doesn't provide  APIs to
perform higher-level features, like text layout or graphics processing
(e.g.,  colored  text  rendering,  'hollowing',  etc.).   However,  it
greatly simplifies these tasks by providing a simple, easy to use, and
uniform interface to access the content of font files.

FreeType  2  is  released  under  two open-source  licenses:  our  own
BSD-like FreeType  License and the  GPL.  It can  thus be used  by any
kind of projects, be they proprietary or not.


--


You  can use  a  `.sig` file  to verify  that  the corresponding  file
(without the  `.sig` suffix)  is intact.  First,  be sure  to download
both  the `.sig`  file and  the  corresponding archive.   Then, run  a
command like this:

  gpg --verify freetype-2.13.2.tar.gz.sig

If that command fails because you  don't have the required public key,
execute

  gpg --keyserver pgp.mit.edu --recv-keys BE6C3AAC63AD8E3F

to import it, then rerun the `gpg --verify` command.

SHA1 file checksums:

7a26f7f2174f257afbfd4c88ec874621a0a84ea9  freetype-2.13.2.tar.gz
2d8d5917a1983ebd04921f2993a88858d6f72dec  freetype-2.13.2.tar.xz
07d34040d9f98b0995a21ceb2421ed0ef8cffd1d  ft2132.zip
85284169bea0bbcffce0c4bc8a0be57288121e7f  ft2demos-2.13.2.tar.gz
655c82a431fae7f53a964bda8a7c0671531d05a4  ft2demos-2.13.2.tar.xz
b33cfa859563f1d269dc29273d972f4b65eb44e2  ftdmo2132.zip
cac82fb66405e53cbad6f93c07211befe5bbe3fd  freetype-doc-2.13.2.tar.gz
dbed086b3dba1d748e15b28103081ed30d24e3f3  freetype-doc-2.13.2.tar.xz
1cac5b0411773555470ca0b2b2a02c34362e178d  ftdoc2132.zip

SHA256 file checksums:

1ac27e16c134a7f2ccea177faba19801131116fd682efc1f5737037c5db224b5  
freetype-2.13.2.tar.gz
12991c4e55c506dd7f9b765933e62fd2be2e06d421505d7950a132e4f1bb484d  
freetype-2.13.2.tar.xz
b7e5b03d2e890c4a881e9fab5870463a37fc9cb934c886b9aab2f6fd637ae783  ft2132.zip
f4a2d44f19af121ff62421d989d03b558c374613c14fefc2b0db4eaf91fcafbf  
ft2demos-2.13.2.tar.gz
99ee2ed8b98bcfad17bc57c2d9699d764f20fe29ad304c69b8eb28834ca3b48e  
ft2demos-2.13.2.tar.xz
de326cd84513c8fa54aed2f570588adcc045a8176c079089e31d0d2650bbd468  ftdmo2132.zip
f5f18802e5fc64769c5918a21e44e7c15f7141fcf4e26d87368a22bcd145bd45  
freetype-doc-2.13.2.tar.gz
685c25e1035a5076e5097186b3143b9c06878f3f9087d0a81e4d8538d5d15424  
freetype-doc-2.13.2.tar.xz
6c0e33138688d171b3b445eb2a404c83c35dde836265f9dad47666a301388e06  ftdoc2132.zip


CHANGES BETWEEN 2.13.1 and 2.13.2 (2023-Aug-25)

  I. MISCELLANEOUS

  - Better support for CFF2 variation fonts.

  - TrueType interpreter  version 38 (also known  as 'Infinality') has
been removed.

  - Improved OpenVMS support.

[ft-announce] Announcing FreeType 2.13.2

[Werner LEMBERG]

FreeType 2.13.2 has been released.

It is available from

https://savannah.nongnu.org/download/freetype/

or

https://sourceforge.net/projects/freetype/files/

The latter site also holds older versions of the FreeType library.

See below for the relevant snippet from the CHANGES file.

Enjoy!


   Werner


PS: Downloads from  savannah.nongnu.org  will redirect to your nearest
mirror site.   Files on  mirrors may  be subject to  a replication
delay   of   up   to   24   hours.   In   case   of  problems  use
https://download-mirror.savannah.gnu.org/releases/


--


https://www.freetype.org


FreeType 2  is a software  font engine that  is designed to  be small,
efficient,  highly   customizable,  and  portable   while  capable  of
producing high-quality output (glyph images) of most vector and bitmap
font formats.

Note that  FreeType 2 is  a font service  and doesn't provide  APIs to
perform higher-level features, like text layout or graphics processing
(e.g.,  colored  text  rendering,  'hollowing',  etc.).   However,  it
greatly simplifies these tasks by providing a simple, easy to use, and
uniform interface to access the content of font files.

FreeType  2  is  released  under  two open-source  licenses:  our  own
BSD-like FreeType  License and the  GPL.  It can  thus be used  by any
kind of projects, be they proprietary or not.


--


You  can use  a  `.sig` file  to verify  that  the corresponding  file
(without the  `.sig` suffix)  is intact.  First,  be sure  to download
both  the `.sig`  file and  the  corresponding archive.   Then, run  a
command like this:

  gpg --verify freetype-2.13.2.tar.gz.sig

If that command fails because you  don't have the required public key,
execute

  gpg --keyserver pgp.mit.edu --recv-keys BE6C3AAC63AD8E3F

to import it, then rerun the `gpg --verify` command.

SHA1 file checksums:

7a26f7f2174f257afbfd4c88ec874621a0a84ea9  freetype-2.13.2.tar.gz
2d8d5917a1983ebd04921f2993a88858d6f72dec  freetype-2.13.2.tar.xz
07d34040d9f98b0995a21ceb2421ed0ef8cffd1d  ft2132.zip
85284169bea0bbcffce0c4bc8a0be57288121e7f  ft2demos-2.13.2.tar.gz
655c82a431fae7f53a964bda8a7c0671531d05a4  ft2demos-2.13.2.tar.xz
b33cfa859563f1d269dc29273d972f4b65eb44e2  ftdmo2132.zip
cac82fb66405e53cbad6f93c07211befe5bbe3fd  freetype-doc-2.13.2.tar.gz
dbed086b3dba1d748e15b28103081ed30d24e3f3  freetype-doc-2.13.2.tar.xz
1cac5b0411773555470ca0b2b2a02c34362e178d  ftdoc2132.zip

SHA256 file checksums:

1ac27e16c134a7f2ccea177faba19801131116fd682efc1f5737037c5db224b5  
freetype-2.13.2.tar.gz
12991c4e55c506dd7f9b765933e62fd2be2e06d421505d7950a132e4f1bb484d  
freetype-2.13.2.tar.xz
b7e5b03d2e890c4a881e9fab5870463a37fc9cb934c886b9aab2f6fd637ae783  ft2132.zip
f4a2d44f19af121ff62421d989d03b558c374613c14fefc2b0db4eaf91fcafbf  
ft2demos-2.13.2.tar.gz
99ee2ed8b98bcfad17bc57c2d9699d764f20fe29ad304c69b8eb28834ca3b48e  
ft2demos-2.13.2.tar.xz
de326cd84513c8fa54aed2f570588adcc045a8176c079089e31d0d2650bbd468  ftdmo2132.zip
f5f18802e5fc64769c5918a21e44e7c15f7141fcf4e26d87368a22bcd145bd45  
freetype-doc-2.13.2.tar.gz
685c25e1035a5076e5097186b3143b9c06878f3f9087d0a81e4d8538d5d15424  
freetype-doc-2.13.2.tar.xz
6c0e33138688d171b3b445eb2a404c83c35dde836265f9dad47666a301388e06  ftdoc2132.zip


CHANGES BETWEEN 2.13.1 and 2.13.2 (2023-Aug-25)

  I. MISCELLANEOUS

  - Better support for CFF2 variation fonts.

  - TrueType interpreter  version 38 (also known  as 'Infinality') has
been removed.

  - Improved OpenVMS support.

___
Freetype-announce mailing list
Freetype-announce@nongnu.org
https://lists.nongnu.org/mailman/listinfo/freetype-announce

Re: Need Help!!Font Family Name is not coming as needed.

[Werner LEMBERG]

 
> Looking more closely at code I think freetype is returning preferred
> Family name of font as font family if that is present inside font.
> Is there anyway to know if the font name is picked from preferred
> Family name or Family only.

No, there isn't.  Why do you need that?

You can either use `FT_PARAM_TAG_IGNORE_TYPOGRAPHIC_FAMILY` and/or
`FT_PARAM_TAG_IGNORE_TYPOGRAPHIC_SUBFAMILY` to enforce the old
four-faces-per-family scheme, or you can parse the 'name' table by
yourself, see `FT_Get_Sfnt_Name`.


   Werner

New FreeType release within a few days

[Werner LEMBERG]

Folks,


I plan to do a new, minor FreeType release until the end of August,
mainly to fix a problem with variable CFF2 fonts.  Is there anything
urgent that should be tackled, too?


Werner

Re: Need Help!!Font Family Name is not coming as needed.

[Werner LEMBERG]

> I am using FT_FaceRec_ object to get the font family name in my
> code. But not getting the correct (entire font name as family name)
> for few fonts.  For e.g. Arial Black, Freetype is returning font
> name as Arial and Style as Black.  On further looking at code I
> found it goes into the check for TYPOGRAPHIC_FAMILY and return the
> valid string as Arial and didn't execute the condition with
> FONT_FAMILY.  Is there is any way I can know if the font family name
> has TYPOGRAPHIC_FAMILY or FONT_FAMILY.

You have to follow the rules given in the OpenType specification.
Have a look at 

  https://learn.microsoft.com/en-us/typography/opentype/spec/name
  https://learn.microsoft.com/en-us/typography/opentype/spec/recom#name


Werner

Re: -warmup

[Werner LEMBERG]

Ahmet,


> I have edited the code aligning with the Hin-Tak’s suggestion. Here
> is the two results pages, also pushed on gitlab.

Thanks.  It seems to me we are getting nearer.  However, there are
still large differences.

* Chris mentioned a potential problem with `clock_gettime` in the
  code of `ftbench.c`.  Please have a look.

* As mentioned a few times already in previous e-mails I think we need
  some code to increase the run time for individual tests.  For
  example, the line

  ```
  Load_Advances (Fast)   47500   202   284   -40.6
  ```

  indicates that 47500 iterations only took 202µs vs. 284µs – due to
  the 'CPU noise' I think this interval is far too short to be
  meaningful.

  For example, if the cumulative time for test X is less than a
  certain threshold, increase N so that the cumulative time is very
  near to the threshold.  This value, if performed by the 'baseline',
  should be stored in a configuration file so that the 'benchmark'
  stage can extract this information to use exactly the same N value
  for test X.

  Please work on that.

* It would be great if you could use a statistics program of your
  choice and prepare some diagrams of the most problematic cases that
  show the actual timing distributions graphically.  Right now, we
  only see the final cumulative value; however, it would be most
  interesting to see more details of the timing slots.

  For example, I can imagine that you add some `printf` calls
  (printing to a buffer, which gets dumped to stdout or whatever
  *after* the tests); then GNU plot or something similar can generate
  diagrams.


Werner

Re: -warmup

[Werner LEMBERG]

>> To summarize: Benchmark comparisons only work if there is a sound
>> mathematical foundation to reduce the noise.
> 
> I am probably not qualified, but I am following the discussion for
> some time.  And I think there is a problem with the benchmarking
> itself.  If I understand correctly the nice tables show the same
> code on the same machine so 40% difference or so is not ok.
> 
> I had a quick look at ftbench.c and I have the impression that the
> timer ist using by clock_gettime for every single iteration twice.
> I had expected to do N iterations with a single clock_gettime before
> and after N iterations.  If the benchmarked code is short this will
> accumulate errors that cannot be removed afterwards.  But I may be
> wrong...

Alexei?


Werner

Re: ttfautohint's functionality from the removal of the infinality patch

[Werner LEMBERG]

>>> And dropping support, and/or compatibility/ awareness of intended
>>> usages of outcome in other *cough* microsoft *cough* hinting
>>> models?
>>
>> Well, the Infinality stuff was basically unmaintained; it
>> essentially consisted of a large bunch of exceptions for some
>> special fonts.  Alexei and I agree that we no longer need this.
>> There was not a single voice objecting to its removal, by the
>> way...
> 
> Great!
> 
> Given that freetype still somehow has the role of a free software
> reference interpretation of the loose MS instructed gridfit
> description, it can only be welcomed if freetype-specifics get out
> of that.

What exactly do you mean with 'freetype-specifics'?  The Infinality
patch aimed to implement v38 of the MS TrueType interpreter as close
as possible, and for the supported fonts it worked quite well.


Werner

Re: ttfautohint's functionality from the removal of the infinality patch

[Werner LEMBERG]

>> It's not clear what you are actually asking.  ttfautohint is an
>> incarnation of FreeType's auto-hinter, translated into TrueType
>> bytecode, more or less.  This is not related to interpreting
>> bytecode.
>
> But does it contain hidden assumptions about how bytecodes are
> interpreted (by specific renderer, and specific version of
> now-simplified renderer )?

No hidden assumptions – everything is documented :-)

https://freetype.org/ttfautohint/doc/ttfautohint.html#stem-width-and-positioning-mode


Werner

Re: ttfautohint's functionality from the removal of the infinality patch

[Werner LEMBERG]

> Yesterday's opentype font meeting touched upon hinting and
> ttfautohint briefly.  I see the infinality patch is already gone
> (next release, 2.13.2 I guess - bits of it was removed in 2.13.1
> already).  Question is, does its removal impact the functionality of
> ttfautohint?

No.

> Are we getting into a situation where ttfautohint is hinting for the
> (limited) "good enough" light hinting model of recent freetype?

It's not clear what you are actually asking.  ttfautohint is an
incarnation of FreeType's auto-hinter, translated into TrueType
bytecode, more or less.  This is not related to interpreting bytecode.

> And dropping support, and/or compatibility/ awareness of intended
> usages of outcome in other *cough* microsoft *cough* hinting models?

Well, the Infinality stuff was basically unmaintained; it essentially
consisted of a large bunch of exceptions for some special fonts.
Alexei and I agree that we no longer need this.  There was not a
single voice objecting to its removal, by the way...


Werner

Re: -warmup

[Werner LEMBERG]

>> What happens if you use, say, `-c 10', just running the
>> `Get_Char_Index` test? Are the percental timing differences then
>> still that large?
> Actually Get_Char_Index, on the three pages I have sent in the
> prev. mail, is higher than 6% only 4 times out of 15 total. (which is
> seem on other tests as well).

Well, the thing is that IMHO the difference should be *much* smaller –
your HTML pages show the execution of identical code on an identical
machine, right?

> about outliers, i splitted every tests into chuncks that is sized
> 100.  Made IQR calculations and calculated average time on valid
> chunks.  you can find the result in the attachment also pushed to
> gitlab.

Thanks.  Hin-Tak gave additional suggestions how to possibly improve
the removal of outliers.

> also, since statistics and benchmarking are a sciences their self, i
> am a bit struggling while approaching the problem as well as feels
> like out of the gsoc project scope.

Indeed, the focus lately shifted from a representational aspect to a
more thorough approach how to handle benchmarks in general.  You are
done with the first part, more or less, and it looks fine.  The
latter, however, is definitely part of the GSoC project, too, and I'm
surprised that you think this might not be so: What are benchmark
timings good for if the returned values are completely meaningless?

In most cases, a small performance optimization in FreeType might
yield, say, an improvement of 1%.  Right now, such a change would not
be detectable at all if using the framework you are working on – it
would be completely hidden by noise.

To summarize: Benchmark comparisons only work if there is a sound
mathematical foundation to reduce the noise.  I don't ask you to
reinvent the wheel, but please do some more internet research and
check existing code how to tackle such problems.  I'm 100% sure that
such code already exists (for example, the Google benchmark stuff
mentioned in a previous e-mail, scientific papers on arXiv, etc.,
etc.) and can be easily used, adapted, and simplified for our
purposes.


Werner

Re: -warmup

[Werner LEMBERG]

> Just remember that backup processes (like cleaning up the hard disk,
> running some cron jobs, etc.) can pop up anytime, thus influencing
> the result.

s/backup/background/

Re: -warmup

[Werner LEMBERG]

> I have added the total table that you suggested.

Thanks.

> I think Get_Char_Index is not the problem, the results varies all
> the time.

As far as I can see, there is a direct relationship between the total
cumulated time of a test and the timing variation: The smaller the
cumulated time, the larger the variation.

What happens if you use, say, `-c 10', just running the
`Get_Char_Index` test?  Are the percental timing differences then
still that large?

> Should I proceed to detect outliers?  Since we do not get the same
> error rate consistently, I think we will not find the target we
> expected by outliers.

Why do you think so?  Please explain your reasoning.  Just remember
that backup processes (like cleaning up the hard disk, running some
cron jobs, etc.) can pop up anytime, thus influencing the result.
Such spontaneous events have to be eliminated.

Have you actually tried something along the method I suggested?


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> It is one of those things I never remember - winding rule of
> truetype and postscript are different, one is even-odd, the other is
> non-zero.  [...]

You are on the completely wrong track here.  What I'm talking about is
strictly local and not (directly) related to having two different
contours.  For example, to recognize stems and serifs, the contour
must be the same.  In case you are not aware of how the auto-hinter
collects 'segments' and 'edges', please read

  https://www.tug.org/TUGboat/tb24-3/lemberg.pdf

which still gives a good overview (inspite of its age and some changed
details here and there).


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> I'm still missing something.  Why would the direction of the contour
> matter if, in either case, it's the same set of points?

It doesn't directly matter for your code.  However, FreeType's
auto-hinter handles these cases differently (namely, to detect stems),
which might in turn influence the adjustments you have to manage.
It's just a heads-up.


Werner

Re: Starting as a FreeType Contributer

[Werner LEMBERG]

[Please always answer to the list, not to me personally.]


> I can do work based on C, I wanna contribute to the UI and UX of
> FreeType as well.

UI?  UX?  What are you talking about?  FreeType is a low-level
*library*!  Its API is more or less unchanged for the last 20 years.

Note that 'I wanna contribute' doesn't get you started.  The normal
way is to work with the library, get used to it, identify problems or
bugs, then trying to fix them.


Werner

Re: Starting as a FreeType Contributer

[Werner LEMBERG]

Hello Harsh,


> it's Harsh from India, i'm a student and I believe in Contributing
> while Learning Approach.  I have gone through your documentation and
> I think I can contribute to the Efficiency and portablability in the
> program.  I wanna do it with the C language.

welcome to FreeType!  What exactly do you want to do?  Your statement
is far too broad to be answered in any sensible way...


   Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> You said that for an i - like shape:
>> Both contours have the same direction.
> 
> What kind of problems does this rule protect against?

Sorry, this was sloppily formulated.  It's about the *local* direction
of contours, that is, whether a horizontal contour segment goes from
left to right or from right to left.  For the 'i' stem and the 'i'
dot, both contours must have the same direction globally, but locally,
at the dividing space, the corresponding lower and upper segments must
have the opposite directions.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> If you're testing this yourself, keep in mind that the adjustment
> will only be applied to n with tilde.  Any other letter with a tilde
> can be used as a control group.

Right now I'm abroad (actually, in New York :-) – it will take some
time until I can do such tests, sorry.


Werner

Re: -warmup

[Werner LEMBERG]

>> What exactly means 'Baseline (ms)'? Is the shown number the time
>>  for one loop? For all loops together? Please clarify and mention
>>  this on the HTML page.
>
> Clarified that the times are milliseconds for the cumulative time
> for all iterations.

Thanks.  The sentence is not easily comprehensible.  Perhaps change it
to something like

```
Cumulative time for all iterations.  Smaller values means better.
```

BTW, in column 'N' I see stuff like '68160 | 65880'.  What does this
mean?  Please add an explanatory comment to the HTML page.

Another thing: Please mention on the HTML page the completion time for
each test, and the total execution time of all tests together.

>> Looking at the 'Load_Advances (Unscaled)' row, I think that 100%
>>  difference between 0.001 and 0.002 doesn't make any sense. How do
>>  you compute the percentage? Is this based on the cumulative time
>> of  all loops? If so, and you really get such small numbers, there
>> must  be some fine-tuning for high-speed tests (for example,
>> increasing N  for this particular test by a factor of 10, say) to
>> get meaningful  timing values.
>
> it was cumulative time in milliseconds but converted it microseconds
> as how it was and it seem got better.

We are getting nearer, again :-)

What worries me, though, is that we still have such enormous
differences.  For `Get_Char_Index` I think it's lack of precision.
Please try to fix this – if the ratio

   cumulative_time / N

is smaller than a given threshold, N must be increased a lot.  In
other words, for `Roboto_subset.ttf`, N should be set to, say, 10*N.

For the other large differences I think we need some statistical
analysis to get better results – simple cumulation is not good enough.
In particular, outliers should be removed (at least this is my
hypothesis).  Maybe you can look up the internet to find some simple
code to handle them.

An idea to identify outliers could be to split the cumulation time
into, say, 100 smaller intervals.  You can the discard the too-large
values and compute the mean of the remaining data.  My reasoning is
that other CPU activity happens in parallel, but only for short
amounts of time.

Have you actually done a statistical analysis of, say, 'Load_Advances
(Normal)' for `Arial_subset.ttf`?  For example, printing all timings
of the datapoints as histograms for runs A and B?  *Are* there
outliers?  Maybe there is another statistical mean value that gives
more meaningful results.


Werner

Re: Progress update on alternative rendering engines project

[Werner LEMBERG]

Hello Anurag,


> From what I understood, the main performance improvement that
> fontdue claims to have is that it stores the font outlines
> differently, i.e. it seems to perform the curve flattening at
> initial load and stores the outline in the form of "v_lines" and
> "m_lines", presumably indicating "vertical lines" and lines spanning
> "multiple pixels horizontally".
> 
> While trying to implement and benchmark it, I realized I had made a
> mistake in my previous benchmarks : I did not disable hinting.
> 
> Fontdue does not support hinting, while the default mode for
> FreeType seems to have hinting enabled, leading to a decrease in
> performance.  This was also an issue with the benchmark program used
> by fontdue.
> 
> I created a small benchmark program and reran the benchmarks with
> hinting disabled, and as expected freetype was much faster with
> hinting disabled.  So much so that on arm64 where there is no SIMD
> in fontdue, FreeType is faster in every single case.

With 'FreeType' you mean the standard rasterizer in `src/smooth`,
right?

> Its only with SIMD optimizations that fontdue is faster in a small
> range of sizes.

Very interesting!  Raph, can you comment, please?  You can find
Anurag's comparison images at

  https://lists.nongnu.org/archive/html/freetype-devel/2023-08/msg8.html

> I have also discovered some sort of regression with the `smooth`
> rasterizer using the font "Cascadia Code".  Its performance degrades
> rapidly when rendering that font.  A preliminary check indicates
> something to do with "overlap_spans", I will send the `perf` results
> as well ASAP.

Thanks in advance.

> While I am trying to investigate more thoroughly, the benchmarks at
> first glance seem to indicate that only the SIMD improvement is
> worth considering and the other claims of performance don't seem to
> work well.

What exactly did you compare time-wise with FreeType's 'smooth'
render?  The original Rust version of 'fontdue', or your conversion of
'fontdue' into a FreeType driver module written in C?

  https://gitlab.freedesktop.org/freetype/freetype/-/tree/gsoc-anurag-2023


Werner

Re: -warmup

[Werner LEMBERG]

> It is warming up as the given number of seconds with -w flag before
> every benchmark test.
>
> There are still differences like 100%..  Also, 1 sec warmup means
> (test count)*(font count) 70 secs for the results.

Mhmm, I'm not sure whether a warmup *time span* makes sense.  I would
rather have thought that every test would get a certain number of
warmup *loops*.  For example, '--warmup 100' means that for a value of
N=5, the first 100 loops of each test are not taken into account
for timing so that effects of the various processor and memory caches,
the operating system's memory page swapping, etc., etc., doesn't have
too much influence.  This should be just a very small fraction of
time, not 70s.

> I am thinking of what else can be done and waiting for your test.

Just looking at your most recent HTML page I see some peculiarities.

* What exactly means 'Baseline (ms)'?  Is the shown number the time
  for one loop?  For all loops together?  Please clarify and mention
  this on the HTML page.

* There seems to be a fundamental math problem in calculating the
  percentage numbers.  For example, looking at the 'TOTAL' field, the
  percental difference between 2.788 and 2.740 is not -6.1% but -1.7%!
  What am I missing?

* Looking at the 'Load_Advances (Unscaled)' row, I think that 100%
  difference between 0.001 and 0.002 doesn't make any sense.  How do
  you compute the percentage?  Is this based on the cumulative time of
  all loops?  If so, and you really get such small numbers, there must
  be some fine-tuning for high-speed tests (for example, increasing N
  for this particular test by a factor of 10, say) to get meaningful
  timing values.


 Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> [...] I have confirmed by commenting/uncommenting steps of the
> hinting process that af_glyph_hints_align_edge_points is the
> function that snaps the tilde back to flat, so if my understanding
> is correct, the points at the top and bottom of the tilde are
> forming edges and are being rounded towards each other, causing the
> tilde to remain flat.

Yes, it seems so.  In case you haven't read it already, this article
gives a nice overview, but please be aware that it is 20(!) years old
and thus partially outdated with respect to some details:

  https://www.tug.org/TUGboat/tb24-3/lemberg.pdf

> How should I proceed?

Perhaps you can try to remove the affected segments from the
corresponding edges.  If that fails there is still the possibility to
apply the vertical distortion afterwards...


Werner

Re: ftbench update: make integrated

[Werner LEMBERG]

> I have done the changes you want.

Thanks!

>> 36.5% run difference is bd. AFAICS, you haven't yet worked on
>> omitting 'warmup' iterations, right?
>
> I am planning to increase the iteration count by 10% and ignore the
> results for them.

You mean you are going to ignore the first 10% of the iterations?
This might be a good default.  However, I still think that a
`--warmup` command-line option makes sense to control this.

> Trying to figure out the benchmarking program but actually drowning
> in 1500 lines of code.

:-)  I hope you can eventually find what you need.


Werner

Re: Progress update on alternative rendering engines project

[Werner LEMBERG]

> I am currently working on implementing optimizations from fontdue in
> the `dense` module.

Any progress code-wise?  Two weeks have passed, and I only see minor
adjustments to the MarkDown document conversions – and no report from
you :-)


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> I have an algorithm I'm testing for the tilde unflattening.  I went
> with doing it before all steps, because it worked better with my
> idea, but the function af_glyph_hints_align_weak_points is undoing
> my changes.

Hmm.  Have you checked the OpenType specification how the IUP
instruction works?

  
https://learn.microsoft.com/en-us/typography/opentype/spec/tt_instructions#interpolate-untouched-points-through-the-outline

Have you 'touched' the points you move (i.e., setting
`AF_FLAG_TOUCH_Y`) so that `af_glyph_hints_align_weak_points` doesn't
move them again?


Werner

Re: ftbench update: make integrated

[Werner LEMBERG]

Here is my new bunch of remarks.  In total, we are getting nearer to
meaningful results :-)

* The build process seems to be ok now, very good!

* After doing `make baseline` (and `make benchmark`) I see as the last
  line

  ```
  Processing 100%...\nBaseline created.
  ```

  This `\n' looks strange.

* It would be nice if `make benchmark` shows as the last line
  something like

  ```
  Benchmark results created in file `/foo/bar/benchmark.html`.
  ```

* Looking at `benchmark.html`, I'm quite happy now with the layout,
  thanks!  However, running baseline and benchmark tests for the same
  commit on my computer I'm still 'greeted' in the very first test
  line with

  ```
  Load  47500  0.167  0.228  -36.5
  ```

  36.5% run difference is bd.  AFAICS, you haven't yet worked on
  omitting 'warmup' iterations, right?

* In `benchmark.html`, I see

  ```
  *Smaller values mean faster operation
  ```

  please insert a space after '*'.  Additionally, you should mark the
  columns that are affected by this comment with '*', too, so that the
  reader knows what you are referring to.


 Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> > Just take the extrema of *all* points – for fonts this should be
> > good enough, because it is standard to have points at the curve
> > extrema, thus making the bounding box of the curve identical to
> > the bounding box of the points (both on and off points).
> 
> Ok, thanks.  This is exactly what I needed.  I was already trying
> this, but I thought it was wrong because of off points.

For your information, you might also look at FreeType's function
`FT_Glyph_Get_CBox`, which handles the generic case.

> What I mean is: my code allows the adjustment for i to be applied to
> glyphs with more than 2 contours to also adjust accented o and a, so
> the rules you suggested would reject around half of the characters
> that are currently being adjusted this way.  I think your rules can
> still be enforced by treating the contour to be moved as "A" and all
> other contours collectively as "B".

Yeah, I was especially referring to glyph 'i', since here the problem
is most visible.

>> [...] I ask you to have this in mind to find a solution that can be
>> easily extended to cover this situation, too (for example, by using
>> an extendable structure instead of a plain variable).
> 
> In this case, do you mean that instead of making a codepoint a key
> for the database directly, I should wrap it in a struct so that
> other kind of keys can be added?

Something like this, yes.  Just try to be generic, and think of
possible extensions.


   Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

Hello Craig,


again sorry for the late reply.

> During this time, I realized that knowing the bounding box of the
> tilde contour would help a lot.  In fact, the logic for the other
> vertical separation adjustments assumes that it can get the bounding
> box by taking the minimum/maximum coordinates of all the points, but
> this doesn't work because of off-points, which I didn't consider at
> the time.  How do you find this bounding box?

What exactly do you mean with 'find'?  The algorithm?  Just take the
extrema of *all* points – for fonts this should be good enough,
because it is standard to have points at the curve extrema, thus
making the bounding box of the curve identical to the bounding box of
the points (both on and off points).

> I should note that, in your example, check #3 is too restrictive.
> The logic allows for the bottom shape that needs to be separated to
> be made up of any number of contours, which allows it to work for
> characters with more complex shapes.

What exact rule are you referring to?  My rule #3 was

  (3) All points of A are lower than all points of B (or vice versa).

which doesn't seem to fit what you are talking about...

> I want to clarify: are you adding glyph names in the database as a
> requirement for the project?

No, I'm not, but I ask you to have this in mind to find a solution
that can be easily extended to cover this situation, too (for example,
by using an extendable structure instead of a plain variable).


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

>> Many-to-one: accent marks, e.g. umlauts
>> One-glyph-to-many-unicode-characters: ligatures, e.g. "ff", "fi".
> 
> I don’t see how these two cases are different. An accented glyph like
> ⟨ɑ̃⟩ is made up of two Unicode characters, ɑ+◌̃ (U+0251 U+0303);
> similarly a ligated glyph like ⟨fi⟩ is made up of the two Unicode
> characters f+i (U+0066 U+0069).

The first case is usually handled in the GPOS table; the auto-hinter
can only ignore this because the rendering of the components doesn't
happen together but in succession.  See my other mail that gives both
a many-to-one and a one-to many example that might be part of GSUB.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

>> A mapping from input Unicode characters to glyph indices based on
>> the GSUB + cmap tables and not on cmap alone.
>
> Right now, the only cases where the GSUB table is helpful that I am
> aware of, for the purposes of this project, are for handling glyph
> alternates and combining characters.  Those would be one-to-one
> mappings and many-to-one mappings, respectively.

Yes.

> Would this general solution involve other kinds of GSUB mappings?

Yes, but it won't affect your database format, which always contains
single Unicode input characters (or glyph names, see below).

> If so, it opens up edge cases such as: if a glyph on the "many" side
> of a many-to-one mapping needs a vertical separation adjustment,
> does the resulting glyph need it too?  This could be answered
> quickly by looking at the specific characters involved, but how
> would I answer this question in general?
>
> Even sticking to just many-to-one and one-to-one mappings, the
> adjustment database must make assumptions specific to the characters
> it's dealing with.

Yes, I think some kind of topological tests are needed.  For example,
the following assumptions should be checked for an 'i'-like shape:

(1) There are two contours A and B.
(2) Both contours have the same direction.
(3) All points of A are lower than all points of B (or vice versa).
(4) There is some horizontal overlap between A and B.

> In the case of combining characters, a separate database table is
> required because the existing table is a list of unicode characters
> and the actions that should be applied to them, while a glyph
> resulting from a combining character might not be a unicode
> character.

I don't think you need another database.  Let's discuss an 'fi'
ligature example.

* Analyzing the GSUB table shows that it is the result of two input
  characters, 'f' and i'.
* 'f' doesn't occur in your database and can be thus ignored if it is
  about to be queried.
* 'i' appears in your database, so check whether glyph 'fi' satisfies
  the constraints for an 'i'-like shape (as described above): No, it
  doesn't, since it either is a single contour, failing condition (1),
  or if there are two contours, failing condition (3).

It would be beneficial if your database could accept also glyph names
in addition to Unicode input character codes – glyph names, if
present, follow the Adobe Glyph List conventions today, which means
that they are standardized to a certain extent.  For example, there
might be an entry for glyph name 'fi' that ensures that on the right
side of the glyph there is vertical separation between the bottom and
top shape (i.e., the merged 'i').

Another example, this time a one-to-many mapping: Let's assume that
glyph U+2173, SMALL ROMAN NUMERAL FOUR, gets mapped in the GSUB table
to two glyphs, 'v.numeral' and 'i.numeral' (but 'i' is not mapped to
'i.numeral' or vice versa because of incompatible horizontal metrics).
Let's further assume that glyph 'i.numeral' is about to be queried.

* According to the GSUB table, 'i.numeral' is part of U+2173.
* U+2173 is in your database, and the constraints for considering it
  are the same as for the 'i' entry, which are met here.

> As for the tilde correction, I'll try doing it after the grid
> fitting like you recommended.

OK.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> Right now, I don't understand the needs well enough to know what to
> generalize or to test whether the general solution works well
> enough.  I'd rather start with specific use cases rather than a
> general solution with unclear goals.

Well, there are no 'unclear goals': the general solution is *exactly*
what I was talking about all the time, and what you need for any entry
in the adjustment database: A mapping from input Unicode characters to
glyph indices based on the GSUB + cmap tables and not on cmap alone.

>> Right now, I favor the latter: It should be a last-minute action,
>> similar to TrueType's `DELTAP[123]` bytecode instructions.
> 
> I disagree with doing the adjustment after grid fitting because in
> this case, grid fitting is a destructive action.  Doing it after
> would require taking a flat line and adding the wiggle back in,

Yes, but you have all the available data because you can access the
original glyph shape.  In other words, you can exactly control which
points to move.

> possibly in a way that doesn't match the font.

At the resolutions we are talking about this absolutely doesn't
matter, I think.  Essentially you have to make

  

appear as

   x x
  x x

> It sounds easier to prevent that from happening in the first place.

OK, give it a try.  Rounding information is available also, so this
might work as well.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> Since hinting glyphs that are descendants of combining characters
> will help few fonts, what other ways does the database need to use
> the GSUB table?  The only other use case I'm aware of are one to one
> substitutions providing alternate forms of a glyph.

Probably yes, but who knows.  It would be nice to have a generic
solution that completely covers the whole situation, and we never have
to think about it again.

> As for the tilde un-flattening, the approach I'm thinking of is to
> force the tilde to be at least 2 pixels tall before grid fitting
> begins.  Would this ever cause the tilde to be 3 pixels because of
> rounding?

This leads to the basic question: Shall the correction be applied
before or after the grid fitting?  Right now, I favor the latter: It
should be a last-minute action, similar to TrueType's `DELTAP[123]`
bytecode instructions.

  
https://learn.microsoft.com/en-us/typography/opentype/spec/tt_instructions#managing-exceptions

In other words, if a tilde character's wiggle (not the whole tilde's
vertical size!) is detected to be only 1px high, the shape should be
aggressively distorted vertically to make the wiggle span two pixels.
To do this, some code has to be written to detect the the inflection
and extremum points of the upper and lower wiggle of the outline; only
the extrema are then to be moved vertically.


Werner

Re: The COLRv1 hook code

[Werner LEMBERG]

> I'd like to not call "FT_Glyph_To_Bitmap()" but just do
> 'FT_New_Glyph()' on my own, but that always crashes.  Why?

What does the debugger say?  This might give a hint.

Are you sure that the memory allocation routine in your code is the
one used by FreeType?  `FT_MEM_ALLOC` is an internal function...


Werner

Re: ftbench update: make integrated

[Werner LEMBERG]

> about percentages, i runned the bench with -c 200 to have instant
> results for development process.  here in the benchmark file
> attached, it made more acceptable result when increased the -c flag
> to 2000.

This is much better, thanks!  However, there are still tests which
have a difference of over 10% for the same commit, inspite of having a
very large number of runs.

Increasing the number of iterations is actually a brute-force method –
what are the timings for the new defaults?  The tests must not be too
slow, overwise I could run everything in a virtual machine like
'valgrind' and use just a single iteration...

Apropos timings: Please add some info to the HTML page that tells how
long it takes to test a given font (or perhaps even more detailed
information to tell how long it takes to perform a certain test).

I suggest that you have a look at other statistical tools that do such
sampling, for example google's 'benchmark' project.  In particular,
have a look at its user manual:

  https://github.com/google/benchmark/blob/main/docs/user_guide.md

What caught my attention especially was the warmup-time option: Maybe
it helps if you add an option `--warmup=N` to the benchmark program to
make it ignore the first N iterations before starting the timing.

Maybe there are other things in the user manual (and/or source code)
that you could use to improve the statistical quality of the FreeType
tests.

Other useful information to reduce the variance can be found here;
please do some research of what might be applicable!

  https://github.com/google/benchmark/blob/main/docs/reducing_variance.md

> I changed compiling and the linking process as the demo programs. i
> would like to continue to another build system if it seem ok.

Will test soon.


Werner

Re: Progress update on adjustment database

[Werner LEMBERG]

> The next thing I'm doing for the adjustment database is making
> combining characters work.  Currently, only precomposed characters
> will be adjusted.  If my understanding is correct, this would mean
> finding any lookups that map a character + combining character onto
> a glyph, then apply the appropriate adjustments to that glyph.

Yes.  I suggest that you use the `ttx` decompiler from fonttools and
analyse the contents of a GSUB table of your favourite font.

  https://pypi.org/project/fonttools/

At the same time, use the `ftview` FreeType demo program with an
appropriate `FT2_DEBUG` setting so that you can see what the current
HarfBuzz code does for the given font.  Examples:

```
ttx -t GSUB arial.ttf
FT2_DEBUG="afshaper:7 afglobal:7 -v" \
  ftview -l 2 -kq arial.ttf &> arial.log
```

Option `-l 2` selects 'light' hinting (i.e., auto-hinting), `-kq`
emulates the 'q' keypress (i.e., quitting immediately).  See appended
files for `arial.ttf` version 7.00.

In `arial.log`, the information coming from the 'afshaper' component
tells you the affected GSUB lookups; this helps poking around in the
XML data as produced by `ttx`.  The 'afglobal' information tells you
the glyph indices covering a given script and feature (start with
'latn_dflt').

You might also try a font editor of your choice (for example,
FontForge, menu entry 'View->Show ATT') to further analyze how the
GSUB data is constructed, and to get some visual feeling on what's
going on.

> Right now, I'm trying to figure out what features I need to look
> inside to find these lookups.  Should I just search all features?

Yes, I think so.  Since the auto-hinter is agnostic to the script and
the used language, you have to have all information in advance.

> After that, I'm going to tackle the tilde-flattening issue, and any
> other similar marks that are getting flattened.

Note that in most fonts you won't find any GSUB data for common
combinations like 'a' + 'acute' -> 'aacute'.  Usually, such stuff gets
handled by the GPOS table, i.e., instead of mapping two glyphs to a
another single one, the accent gets moved to a better position.  In
this case, the glyphs are rendered separately, *outside of FreeType's
scope*.  This means that we can't do anything on the auto-hinter side
to optimize the distance between the base and the accent glyph (see
also the comment in file `afshaper.c` starting at line 308, and this
nice article
https://learn.microsoft.com/en-us/typography/develop/processing-part1).

It thus probably makes sense to do the tilde stuff first.


Werner


arial-7.00.ttx.xz
Description: Binary data


arial-7.00.log.xz
Description: Binary data

Re: Font Rendering: Multiple FT_Faces and baselines

[Werner LEMBERG]

> I'm aware that when drawing text, all glyphs should be aligned on a
> single baseline.  The example I've given does not do this; it aligns
> the text on different font's baseline, causing the inconsistent
> look.

This is what I told you in the very beginning: Your logic to position
the glyphs vertically is flawed; you actually don't align them
correctly – somewhere, you are not using the correct vertical offsets.
I suggest that you check the FreeType demo programs for some guidance.

  https://gitlab.freedesktop.org/freetype/freetype-demos

> When I'm talking about "aligning the baselines", I'm referring to
> choosing a suitable baseline from different fonts. I use the term
> "baseline" loosely as when rendering text from top to bottom, the
> ascent value is used.

Why not position the baseline for all fonts at position y=0?

> Initially, I plan to calculate a global baseline from all the fonts
> by averaging their respective baselines. Another way is to pick the
> "lowest" baseline (as in the highest ascent value) as the global
> baseline. I'm unsure of what strategy to use, so I'm curious about
> what approach do other applications use to solve this problem.

If I understand you correctly, we have an XY problem here.  The real
issue seems not to be the alignment on a baseline but rather how to
compute the global bounding box of a text string that comprises
different fonts.  The answer to that was given repeatedly on the
mailing list: If you need a bounding box correct to the pixel and you
do hinting, you have to make FreeType compute all glyphs, store them
somewhere, *then* walk over all individual bounding boxes and take the
maximum dimensions to get the global one.  Finally, you can render the
glyphs with the correct offsets.


Werner

Re: Font Rendering: Multiple FT_Faces and baselines

[Werner LEMBERG]

[It would be nice if you could use a decent e-mail writer that creates
 readable plain-text messages (which we prefer on this mailing list),
 without completely garbling quoted material.]


> > This looks like a logic error in your program.  If I say `ftdump
> > cjk.otf` I get
> >
> > ```
> > ascender: 1160
> > descender: -288
> > ```
> >
> > among other data.
> >
> > The output of `ftdump en.ttf` gives
> >
> > ```
> > ascender: 935
> > descender: -265
> > ```
> >
> > This is perfectly fine.  [...]
>
> Thank you for pointing it out.  I'm unsure about the strategy to
> align the baselines.  The simplest would be aligning to the lowest /
> highest baselines, but that may break some faces.

Uh, oh, please update your knowledge on how fonts are actually working
within the frame of typography.  There is no 'lowest' or 'highest'
baseline – there is only *the* baseline (assuming horizontal
typesetting).  It is the very idea of a baseline that *all* glyphs of
*all* fonts are by default aligned on that!  There exist additional
concepts like superscript or subscript, but this is something else and
not relevant here, as far as I can see.

Maybe you want to achieve something different, I don't know.  Please
educate yourself in advance to use the right technical terms so that
we can actually communicate without misunderstandings.

You might have a look at

  https://freetype.org/freetype2/docs/glyphs/glyphs-3.html

as a starter.


Werner

Re: Strange hinting of precomposed character

[Werner LEMBERG]

> Yep! https://github.com/wenzhuman/fonttools

Thanks.  Unfortunately, this lacks top-level documentation, AFAICS,
describing the changes in comparison to upstream 'fonttools'.
Additionally, the git clone uses the 'master' branch for changes,
which is problematic.

It would be great if someone could clean this up.  BTW, have you ever
contacted the 'fonttools' maintainers and asked them whether they are
interested in the changes?


Werner

Re: Strange hinting of precomposed character

[Werner LEMBERG]

> A long time ago I had some grad students write a compiler
> infrastructure for TrueType bytecode.

Ah, I faintly remember :-)

> Maybe it's mostly bit-rotted, but if someone were interested in
> working on this, I could try to help.

Thanks.  Is this code available in a public repository?


Werner

Re: COLRv1 to gray/alpha question (& color-blindness question)

[Werner LEMBERG]

[leaving 'freetype-devel' in the CC]

>> Let's assume that you can't discern colors A and B, where both map to
>> exactly the same gray value C (or to almost identical values).  If a
>> glyph uses those two colours exclusively, you will have problems with
>> both a colour and a gray-level version of it.
> 
> Oh, I see. Yes, I would indeed have trouble using such a font. But
> in that case I’m tempted to blame the designer: if they can’t be
> bothered to account for colourblindness, FreeType can’t do very much
> to compensate for that!
> 
> (Unless it can? Maybe there’s scope to allow the user to remap
> colours, or to switch between the two greyscale modes Hin-Tak
> mentioned, in much the same way that they can, say, select hinting
> modes. But I don’t know how feasible this would be.)

Different colour schemes are supported; the question is about
defaults.  For example, let's assume that a font contains color
schemes A and B, the latter suitable for (most) color-blind people.
Let's further assume that scheme A doesn't render well on a grayscale
device because of identical grayscale values.  Does COLRv1 contain any
information to quickly decide which color scheme should be used for
grayscale rendering?


Werner

Re: FT_Bitmap and FT_BitmapGlyph life cycles

[Werner LEMBERG]

> How to throw away the bitmap resulting from running
> FT_Glyph_To_Bitmap(), and replacing it with a new one that may be
> one pixel large or smaller in width and/or height?

It's not exactly clear to me what you want to do, but what about using
`FT_New_Glyph` and/or `FT_Glyph_Copy`?


Werner

Re: COLRv1 to gray/alpha question (& color-blindness question)

[Werner LEMBERG]

> >  A somewhat related question - colour fonts are used beyond
> >  emoji's. While there are 5 kinds of emoji fonts now, and most
> >  people are using one of 4... but if you check Google Fonts, there
> >  are 10 colour fonts, one is emoji, but 6 are Arabic (useful for
> >  annotating the Quran...) and 3 are Latin. So there are intentions
> >  for text fonts. A few percents of western male population is
> >  color-blind. Colour-blindness is one of the most common eye
> >  problems, after short-sightedness :-).
> 
> I’m colour-blind, but not sure I understand what you’re asking
> here. None of the colour fonts on Google Fonts seem obviously
> difficult to read for me.

Let's assume that you can't discern colors A and B, where both map to
exactly the same gray value C (or to almost identical values).  If a
glyph uses those two colours exclusively, you will have problems with
both a colour and a gray-level version of it.


Werner

Re: Bug in rsvg+cairo hook with Nabla?

[Werner LEMBERG]

>> ... please post a link to the report if you do so :-)
> https://gitlab.gnome.org/GNOME/librsvg/-/issues/997

By the way, is there any progress whether Adobe's compact SVG native
viewer library can be used as a FreeType hook?  It comes with a Cairo
backend, among others...

  https://github.com/adobe/svg-native-viewer


Werner

Re: Strange hinting of precomposed character

[Werner LEMBERG]

>> If Craig's modifications to the auto-hinter code become mature it
>> would be very helpful if they got added to `ttfautohint`, too
>> (however, this wouldn't be a trivial thing to do, unfortunately).
> 
> Hmm, what would make it so hard?

The 'new rules' for the auto-hinter must be converted into TrueType
bytecode functions.  Have a look into

  https://repo.or.cz/ttfautohint.git/blob/HEAD:/lib/tafpgm.c

to get some taste...


Werner

Re: Bug in rsvg+cairo hook with Nabla? (Re: another small patch for ft2-demos, and other stuff.

[Werner LEMBERG]

>> ... please post a link to the report if you do so :-)
> https://gitlab.gnome.org/GNOME/librsvg/-/issues/997

Thanks!


Werner

Re: QT6 build of ftinspect bit-rotten?

[Werner LEMBERG]

> I'll look into this after I resolved all backlog issues of ftinspect
> (MM/GX default and PSName issue, UI bug in #26, and colored SVG
> fonts).

Great, and thanks in advance!


Werner

Re: Strange hinting of precomposed character

[Werner LEMBERG]

>> I would expect that, if the diacritic is to be hinted as a
>> horizontal bar, that would occur only at the smallest sizes, and
>> would equally affect the combining diacritic too.
> 
> Somebody more knowledgeable can correct me, but I think combining
> diacritics happens with:
> 
> Outline1 -> hinted bitmap1
> Outline2 -> hinted bitmap2
> 
> Bitmap output = hinted bitmap1 + hinted bitmap2 positioned at
> grid-fitted (ie another "hinting") offsets
> 
> (Actually, I think it happens in the opposite order: the grid-fitted
> offsets are calculated first)
> 
> compared to a single glyph with two sets of outlines, where the
> outlines are hinted together.

Maybe this section from the ttfautohint manual helps.

  https://freetype.org/ttfautohint/doc/ttfautohint.html#composite-glyphs


Werner