Hi folks, I have prepared a final report for the project.
You can get it here: https://moazin.bitbucket.io/gsoc-2019/ It's also available from repository: https://bitbucket.org/moazin/freetype-docs/src/master/ot-svg-report.md I'm also attaching a copy of the final version. Moazin
# GSoC 2019: Adding support for OpenType-SVG fonts to FreeType ## A Project Report ### Project Information | Field | Value | | --------------- | ------------------------------------------------------------ | | Project Link | https://summerofcode.withgoogle.com/projects/#6096036961976320 | | Organization | [The FreeType Project](https://www.freetype.org/) | | Mentors | Werner Lemberg and Suzuki Toshiya | | Languages/Tools | `ANSI C`, `Unix Build Tools` | | My Email | `[email protected]` | ### Project Discussion In this project I successfully added OpenType-SVG font support to FreeType. This involved: 1. Writing code to read the SVG table and load SVG documents. 2. Researching on SVG rendering libraries and using them to render the documents. This involved: * Closely observing the rendering behavior of popular SVG rendering libraries ([`librsvg`](https://github.com/GNOME/librsvg/), [`resvg`](https://github.com/RazrFalcon/resvg/) and [`svgnative`](https://github.com/adobe/svg-native-viewer)) with SVG documents. * Studying the specification to learn about glyph placement in documents and getting them rendered properly. 3. Writing the necessary code to connect FreeType to the external library and render the glyphs. This was done by creating a rendering module named `ot-svg`. 4. Modifying the FreeType demo tools and the FreeType cache system to work with OpenType-SVG glyphs. 5. Modifying the build system of FreeType to incorporate OT-SVG support. All of the goals have been achieved. The only thing remaining is to decide on a default SVG rendering library. There is no hurry for that. We need features that are either absent or under development in most libraries. Thus this decision will be taken quite late. ### Final Work Product 1. [*Development code in FreeType repository*](https://git.savannah.gnu.org/cgit/freetype/freetype2.git/log/?h=GSoC-2019-moazin): All of my code lives in the branch `GSoC-2019-moazin`. Since it's a development branch, the commit history doesn't look clean. I made many mistakes and later fixed them. The work done within the GSoC period is from commit [0729a6516](https://git.savannah.gnu.org/cgit/freetype/freetype2.git/commit/?h=GSoC-2019-moazin&id=0729a65165dd27445bc2ebd1df41df288c85acdd) to [f943af649](https://git.savannah.gnu.org/cgit/freetype/freetype2.git/commit/?h=GSoC-2019-moazin&id=f943af64905378bde29eaf7c4404f1a0fb971830). See the *diff* [here](https://git.savannah.gnu.org/cgit/freetype/freetype2.git/diff/?id=f943af64905378bde29eaf7c4404f1a0fb971830&id2=0729a65165dd27445bc2ebd1df41df288c85acdd). 3. *Writing rendering ports*: FreeType doesn't link directly with SVG rendering libraries. Instead, a callback hook mechanism was created to allow users to plug-in their own libraries. Later on, once we decide on a default rendering library, we will set default hooks so that FreeType supports SVG fonts out-of-the-box. Currently, `librsvg` has been set as the default one. This is just temporary. I call these hooks **rendering ports**. I have written three ports for three popular SVG rendering libraries: * [`librsvg port`](https://bitbucket.org/moazin/librsvg-port-freetype-otsvg): Last commit within GSoC period is [9743ce8cf](https://bitbucket.org/moazin/librsvg-port-freetype-otsvg/commits/9743ce8cf4a8b5ee2e729a3a8cac2d2b9b6674f9) * [`resvg port`](https://bitbucket.org/moazin/resvg-port-freetype-otsvg/src) * [`svgnative port`](https://bitbucket.org/moazin/svgnative-port-freetype-otsvg) Out of these, `librsvg port` is the most up-to-date and I plan to update the other ones soon. ### Outline This report is a large document. Please jump to the relevant section depending on what you seek. * [Usage](#usage): This section should help you immediately run the code I have written and get OT-SVG glyphs rendered. * [A brief summary of how OT-SVG fonts work](#a-brief-summary-of-how-ot-svg-fonts-work): This section introduces how OT-SVG fonts really work. This is almost a brief summary of the [OT-SVG specification](https://docs.microsoft.com/en-us/typography/opentype/spec/svg). * [OT-SVG project in clean steps](#ot-svg-project-in-clean-steps): This section is for people who are already familiar with FreeType and want to know the technical details of the project's implementation in a simple series of steps. * [Design choices and challenges faced](#design-choices-and-challenges-i-faced): This section discusses the challenges I faced in the project and the design choices I made to fix them. ### Usage The project code hasn't been merged to master yet. Thus, it only lives in the branch `GSoC-2019-moazin`. OT-SVG feature can be compiled with or without default hooks. #### Procedure to test OT-SVG fonts without default hooks Firstly, I should mention that this has been only tested on Ubuntu 19.04. To be able to use this project, you must have the latest in-development version of `librsvg` compiled and installed in your system. At the time of writing this, I am using tag [`2.45.91`](https://github.com/GNOME/librsvg/tree/2.45.91). Please follow their instructions to compile and install the library. Once you've done that, follow these steps: 1. Clone the FreeType repository and checkout `GSoC-2019-moazin`. ```bash git clone https://git.savannah.gnu.org/git/freetype/freetype2.git cd freetype2 git checkout GSoC-2019-moazin git checkout f943af6490537 # GSoC-2019-moazin might break due to future commits. cd .. ``` 2. Clone the port repository and checkout to the current commit. ```bash git clone https://[email protected]/moazin/librsvg-port-freetype-otsvg.git cd librsvg-port-freetype-otsvg git checkout 9743ce8cf4a8b5 # again, future commits might change things cd .. ``` 3. Copy the `rsvg_port.c` and `rsvg_port.h` files to the FreeType source. ```bash cp ./librsvg-port-freetype-otsvg/port/rsvg_port.c ./freetype2/src/svg/ cp ./librsvg-port-freetype-otsvg/port/rsvg_port.h ./freetype2/src/svg/ ``` 4. Build FreeType. ```bash cd freetype2 sh autogen.sh ./configure --with-svg=no-default make cd .. ``` 5. Build the port and run the test program. ```bash cd librsvg-port-freetype-otsvg/port mkdir build cd ../tester mkdir build sh compile_port.sh # ./build/main ./path/to/font-file.otf c <character you want to print> # or # ./build/main ./path/to/font-file.otf i <glyph id you want to print> ./build/main ./path/to/font-file.otf c g # Just an example ./build/main ./path/to/font-file.otf i 120 # Just an example ``` In case you're getting some errors, please make sure you have all the dependencies installed. Check the `Makefile` in `librsvg-port-freetype-otsvg/tester` to see the dependencies. In case you need help, feel free to reach me out at `[email protected]`. ### A brief summary of how OT-SVG fonts work [SVG](https://en.wikipedia.org/wiki/Scalable_Vector_Graphics) is a popular vector graphics format which is being extensively used in web pages these days. The [`SVG`](https://docs.microsoft.com/en-us/typography/opentype/spec/svg) table is a feature addition to OpenType that allows SVG documents to be used to describe glyphs. The table contains a list of SVG documents where each SVG document can contain glyph descriptions for one or a whole range of glyph IDs. If an SVG document contains the glyph description of only one glyph ID, the whole document is supposed to be rendered. If on the other hand, the document contains multiple glyph descriptions for a range of glyph IDs, only the element with the ID `'glyph<ID>'` is to be rendered. Read the section *[Glyph Identifiers](https://docs.microsoft.com/en-us/typography/opentype/spec/svg#glyph-identifiers)* to read more about this. #### SVG coordinate system and glyph placement SVG, being a vector graphics format, ultimately relies on Bézier curves. These curves consist of points which exist at certain coordinates in a coordinate system that we shall call the `SVG` coordinate system. This coordinate system should be analogous to the coordinate system of TTF/CFF outlines. However, unlike the TTF/CFF coordinate system, this one has an inverted `y` axis. So, positive `x` is to the right while positive `y` points down. Just like the TTF/CFF coordinate system has an EM square which font designers use, same is true with the SVG coordinate system, though, unlike TTF/CFF outlines, the EM square's size can vary. The `viewBox.width` and `viewBox.height` or `width/height` attribute on the root `SVG` node can indicate the size of the EM square. If such attributes are missing, it should be assumed that the EM square is of the same number of units as the corresponding TTF/CFF outlines' EM square. The `viewBox` attribute describes a rectangle. Imagine that the whole SVG coordinate system is covered by an opaque sheet with one transparent window and that transparent window is the `viewBox`. Any SVG rendering library will output what can be seen through this window. We shall define a new coordinate system `SVG'` as the coordinate system formed by translating the `SVG` coordinate system to the point `viewBox.x` and `viewBox.y`. In case the `viewBox` attribute is missing, `SVG'` coordinate system is totally identical to the `SVG` coordinate system. The glyph is placed in such a way that the baseline is at `y = 0` of the `SVG'` coordinate system. The `advance.x` is the same as that for the corresponding TTF/CFF glyph. For properly rendering the glyph, the origin of `SVG'` must be placed on top of the origin of the current pen point. If that's done properly, the glyph will just land up right where it's supposed to. ### OT-SVG project in clean steps In FreeType, there are two basic steps involved in rendering a glyph that has a particular ID. 1. **Loading the glyph**: The glyph outlines (or the vector data) is loaded into the `glyphslot`. 2. **Rendering the glyph**: The glyph outlines that have been loaded are rendered into a pixmap. For OpenType SVG glyphs, we devise a similar plan. In the **loading** part, the appropriate SVG document will be fetched and stored in the glyph-slot. In the **rendering** part, the outlines will be sent to an external SVG rendering library, which will return a pixmap. #### Making OT-SVG an optional feature `FT_CONFIG_OPTION_SVG` is added to allow toggling OT-SVG support. This was done so that smaller builds of FreeType can disable OT-SVG support to reduce build size. #### Adding code to load the SVG table In FreeType, for fonts that have an `sfnt` header, loaded tables are usually stored in `TT_FaceRec`. The manner in which they are stored differs between tables. For `SVG` a strategy similar to `CPAL` and `SVG` has been adopted. An internal module specific structure `Svg` is created to hold basic information extracted from the `SVG` table. `TT_FaceRec` gets a new field of type `void *` named `svg` which would hold a pointer to this structure. A flag `FT_FACE_FLAG_SVG` is added, this will be set whenever an `SVG` table is present and has been loaded. `sfnt` related functions are exposed by the `sfnt` module. Thus, `SFNT_Interface` was modified to add two more fields `load_svg` and `free_svg`. The actual functions to fill these fields were created with the names `tt_face_load_svg` and `tt_face_free_svg`. The function `sfnt_load_face` loads all the available tables of interest and `sfnt_free_face` frees all the allocations performed by `sfnt_load_face`. Both of these were modified to *load* and *free* the `Svg` structure respectively. #### Adding code to *load* SVG glyphs To store the SVG document in the `glyphslot`, a structure named `FT_SVG_DocumentRec` is created which contains the SVG document along with other necessary details to accompany it. Please see the in-file documentation of `otsvg.h` for more details of this structure. This document is referenced by `other` field of `FT_GlyphSlotRec`. `ft_glyphslot_init`, `ft_glyphslot_clear` and `ft_glyphslot_done` were modified to take care of allocating and freeing memory for this structure. To fetch the correct SVG document, one more function is added to `SFNT_Interface` named `load_svg_doc`. The actual function that populates this place is `tt_face_load_svg_doc`. Ultimately, `cff_slot_load` and `TT_Load_Glyph` were modified to add special code that'll call `load_svg_doc` and fetch `advance` information (and set it) if an OT-SVG glyph is found, otherwise the control flow remains totally the same. #### Adding code to *render* SVG glyphs To render OT-SVG glyphs, an external library is to be used. For this purpose, a new *Renderer Module* is created named `ot-svg`. However, instead of directly linking an external library to FreeType, hooks are used. Thus, the client application can plug in any SVG rendering library of its choice by writing the hook functions. Later, we will decide on a default rendering library and have it linked by default. Thus, FreeType will support OT-SVG glyphs out-of-box and will also give users the ability to plug in other SVG rendering libraries if they want to. We could just create a renderer module like the other ones out there but we need to store the hooks somewhere. For this reason, we create a new type `SVG_RendererRec` that inherits from `FT_RendererRec` and adds some more fields ```c++ typedef struct SVG_RendererRec_ { FT_RendererRec root; /* This inherits FT_RendererRec */ FT_Bool loaded; FT_Bool hooks_set; SVG_RendererHooks hooks; /* Holds out hooks to the outside library */ } SVG_RendererRec; ``` 1. `loaded` is a flag that allows us to lazily load the SVG rendering library at the point when the first SVG glyph is rendered. 2. `hooks_set` is a flag that simply indicates whether hooks have been set or not. In case they haven't been set, the module isn't really functional. 3. `hooks` is a structure that contains the hooks. There are a total of *four* hooks. ##### 1. Init Hook ```c++ typedef FT_Error (*SVG_Lib_Init_Func)( FT_Library library ); ``` This hook is called at the time when the first OT-SVG glyph is to be rendered. Once it has been called `loaded` will be set to `TRUE`. The job of this hook is to let the SVG rendering library perform any sort of initializations that it wants to. We would want the SVG rendering library to have a *state* or *persistence* between different function calls. The library can create a *state* structure of its own, allocate it upon initialization and set `svg_renderer_state` of `library` to reference it. ##### 2. Free Hook ```c++ typedef void (*SVG_Lib_Free_Func)( FT_Library library ); ``` To let the SVG rendering library perform any cleanups. For example, the library should free the *state* structure if it created one. ##### 3. Preset Hook ```c++ typedef FT_Error (*SVG_Lib_Preset_Slot_Func)( FT_GlyphSlot slot, FT_Bool cache); ``` The preset hook does the job of *presetting* the bitmap slot. However, for an OT-SVG glyph, *presetting* the slot needs many calculations which are useful at the rendering stage too. The preset hook is called from two places only. 1. At the **loading** stage from `ft_glyphslot_preset_bitmap`. 2. Right before rendering the glyph in `ft_svg_render`. When it is the latter, we want the function to store those useful calculations in the *state* of the library so that they can be later used by the `render` hook. The calls are right next to each other so we can be sure that the state will remain intact. However, when it's the former, we do not want the function to modify state in anyway. That's the reason we have the `cache` argument. When it is `TRUE` the calculations can saved in *state*, when it's `FALSE` the calculations can't be stored in *state*. The reason why the preset hook is called again in `ft_svg_render` is because we want to make sure that the `glyphslot` has been preset, so we can use its `width/height` to allocate enough memory for the pixmap buffer. ##### 4. Render Hook ```c++ typedef FT_Error (*SVG_Lib_Render_Func)( FT_GlyphSlot slot ); ``` The render hook simply renders the loaded SVG document and places the resultant pixmap in `slot->bitmap.buffer`. Memory for the buffer is already allocated so it shouldn't be allocated from inside. The following values must be set: ```c++ slot->bitmap.pixel_mode = FT_PIXEL_MODE_BGRA; slot->bitmap.num_grays = 256; slot->format = FT_GLYPH_FORMAT_BITMAP; ``` #### Adding support of OT-SVG glyphs in Glyph Management API A crucial part of FreeType is the **Glyph Management API**. Thus, it's necessary to add OT-SVG support to it. This is done by creating new handle `FT_SvgGlyph`, its structure `FT_SvgGlyphRec` and creating a new glyph class, named `ft_svg_glyph_class`. We also need to modify some functions to appropriately handle OT-SVG glyphs but the changes are small. #### Adding support for transforms to OT-SVG glyphs For traditional outlines, transformations are directly applied on the actual outlines itself. For OT-SVG glyphs, FreeType cannot reach any of the actual graphics data, since, only the SVG document string is held. Thus, we can't apply any transforms ourselves. The only thing we can do is store the transformation matrix and later have the external library apply it for us. For this purpose, we add two fields, `transform` and `delta` to `FT_SVG_Document` and `FT_SvgGlyphRec`. If multiple transformations are applied one after another, some maths is done to compute an equivalent transformation matrix. The external library should pick up these fields and apply the transforms while presetting and rendering the document. So far, this approach has worked pretty well. ### Design choices and challenges I faced For any project, I think one of the hardest part is making choices. Same was the case with this project. I list the major challenges I faced below: 1. *Deciding where to store the SVG document and other relevant information*: I needed to store the SVG document itself and some information about it in the `glyphslot` but couldn't really figure out where to store it. Firstly, I tried adding new fields to `TT_GlyphSlotRec` and `CFF_GlyphSlotRec`. The approach worked but soon I realized there were some problems with this approach. [See the thread: [msg00092](https://lists.nongnu.org/archive/html/freetype-devel/2019-06/msg00092.html)] So, upon the suggestion of FreeType developers, I created a new structure `FT_SVG_DocumentRec` and referenced it in the `other` field of `glyphslot`. `FT_SVG_DocumentRec` contains all the fields needed. This approach has turned out to work pretty well. 2. *Nature of the SVG rendering module*: We wanted to give client users the ability to plug in different SVG rendering libraries if they want to. Thus, it was decided not to link the external library directly. Instead, we created a callback hook mechanism. The users can set their own hooks to plug-in another library. There was a contrary idea under consideration too, which was to create different SVG rendering modules for different libraries. The matter was under discussion in the mailing list but ultimately it was decided that the *callback hooks* idea is preferable. [See these threads: [msg00042](https://lists.nongnu.org/archive/html/freetype-devel/2019-07/msg00042.html), [msg00074](https://lists.nongnu.org/archive/html/freetype-devel/2019-07/msg00074.html) and [msg00051](https://lists.nongnu.org/archive/html/freetype-devel/2019-08/msg00051.html)] 3. *Setting the hooks*: How do we get the hooks from the external user? At first, we were using an API function, `FT_Set_Svg_Hooks`, but later we dropped it. Instead, we use *module properties* to set the hooks. [See these threads: [msg00077](https://lists.nongnu.org/archive/html/freetype-devel/2019-07/msg00077.html), [msg00098](https://lists.nongnu.org/archive/html/freetype-devel/2019-07/msg00098.html) and [msg00051](https://lists.nongnu.org/archive/html/freetype-devel/2019-08/msg00051.html)] 4. *Storing the hooks*: There was a confusion about where to store the function pointers for the callback hooks. The solution I came up with, was, instead of creating an `FT_RendererRec` as the rendering module, I created a subclass of it named `SVG_RendererRec` and I added a hooks field to this structure as well as some flags. This solution has worked well and looks fine from a design point of view. 5. *Rendering the SVG glyphs*: Read the section [SVG coordinate system and glyph placement](#svg-coordinate-system-and-glyph-placement) to read about how SVG glyphs are placed within the SVG coordinate system. The behavior of popular SVG rendering libraries is that they just render whatever is in the `viewBox` to a pixmap. To properly render a glyph its bounding box must be known so that we can shift the coordinate system to get a proper rendering. How to get this bounding box is a challenging problem. Different SVG rendering libraries do provide ways of getting the bounding box but they are not always accurate (can be bigger than the actual bounding box). Firstly, we decided to use the bounding box of `TTF/CFF` outlines and from that predict the bounding box in SVG coordinates. [See the thread: [msg00050](https://lists.nongnu.org/archive/html/freetype-devel/2019-06/msg00050.html)] This worked perfectly for many fonts but failed for one. I brought up the issue in the [OpenType-SVG list](https://lists.w3.org/Archives/Public/public-svgopentype/) and ultimately it was realized that the bounding boxes don't have to be the same at all. [See the thread: [0000](https://lists.w3.org/Archives/Public/public-svgopentype/2019Jul/0000.html)] Thus, we resort to using the the SVG rendering library specific methods to get the bounding boxes. [See the thread: [msg00010](https://lists.nongnu.org/archive/html/freetype-devel/2019-08/msg00010.html)] The bounding box is almost always accurate and when it's not, `bitmap_left` and `bitmap_top` make sure that the glyph is positioned correctly after the rendering. Everything works great! :-) 6. *Transformation support for OT-SVG glyphs*: For traditional glyphs, FreeType can manipulate the outlines by itself, and thus it can apply the transformations directly to the outlines. For SVG glyphs, that isn't true, all we store is just the document string. To support transforms, we store a transformation matrix and a translate vector in `FT_SVG_DocumentRec` and `FT_SvgGlyphRec` and later the SVG rendering library picks it up and applies the transformation. There is a problem though. FreeType users input a transformation matrix that has been designed for the TTF/CFF coordinate system and it turns out that the `SVG` coordinate system is quite different. Thus, directly applying the transformation should give unexpected results. The solution I came up with is, we let the user pretend that the coordinate system is just a traditional one, then we convert the transformation to an equivalent one in SVG coordinates and then apply it. Another problem is, how do we support cascaded transformations? The solution I figured out was to use do some maths to compute an equivalent transformation matrix that has the effect of all the transformations applied. This has worked pretty well and the results are good. [See the thread: [msg00037](https://lists.nongnu.org/archive/html/freetype-devel/2019-08/msg00037.html)]
_______________________________________________ Freetype-devel mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/freetype-devel
