[HarfBuzz] Adding tracing

Dear All, I've added tracing to harfbuzz-ng. This is to help font developers rather than programmers, figure out what is going on with their fonts. The cost is a simple if() for each lookup. If that is too high, we can probably make tracing optional. Code available from

Re: [HarfBuzz] harfbuzz coordinate system

Dear Behdad, I think I'm going to make that change, especially since everytime I hooked up HarfBuzz to another system, it worked the first try except for a missing negation sign for y_offset. I don't think it matters too much which way you jump on this one so long as: 1. You clearly state

Re: [HarfBuzz] A few HarfBuzz-ng questions

Dear Behdad, 2. SHAPERS: In void hb_shape(...) in hb-shape.cc, I see this: The current Graphite shaper is disabled because it was crashing on me all the time. Deep in the libgraphite code... That's a major concern. Most probably I never enable graphite or any other non-included backend

Re: [HarfBuzz] Quick show of hands needed

Dear Behdad, I'm wondering: should harfbuzz try compatibility composition/decomposition (NFKC/NFKD) if a font doesn't support a character? No, I don't think this belongs in a shaping/rendering engine. I agree. It shouldn't go near compatibility characters. But it should do canonical

[HarfBuzz] cd build ../configure

Dear Behdad, The enclosed patch fixes this not building ./autogensh make distclean mkdir build cd build ../configure make diff --git a/test/Makefile.am b/test/Makefile.am index b1a9b87..adf1ec8 100644 --- a/test/Makefile.am +++ b/test/Makefile.am @@ -7,7 +7,7 @@ DISTCLEANFILES =

[HarfBuzz] graphite 1.0.1 released

Dear All, Graphite2 v1.0.1 has been released as the first major release of the new graphite engine. It is available from: http://sourceforge.net/projects/silgraphite/files/graphite2/graphite2-1.0.1.tgz/download

Re: [HarfBuzz] graphite 1.0.1 released

Dear Shriramana, On Sat, Aug 6, 2011 at 5:08 AM, Behdad Esfahbod beh...@behdad.org wrote: Thanks Martin.  Now that we have multi-shaper infrastructure in place, I'll go ahead and do that. Sorry to sound ignorant, but HB is OpenType technology for rendering, Graphite is a different

Re: [HarfBuzz] graphite 1.0.1 released

Dear Shriramana, Perhaps I wasn't sufficiently clear. I was interested in knowing whether Graphite rendering via Harfbuzz is slower than OT rendering via Harfbuzz for the same text/script. Somehow I get the perception that OT is inbuilt into HB whereas Graphite is an *external* library

Re: [HarfBuzz] graphite 1.0.1 released

Dear Shriramana, Without any offence to anybody, and with no desire to bash OT but only to make things clear (?) it appears (from this) that the basic principle upon which OT lookup tables were designed aren't very effective. Is that so? No that is far from the case. OpenType is generally

Re: [HarfBuzz] Fwd: Where to report a bug with Tamil text rendering in LibreOffice?

Dear Shriramana, hb-view only produces images. The problem is with the cursor placement. Unless I configure a tiny text box to use hb-ng for CTL, I can't test whether the problem exists in hb-ng myself. Can you please tell me how to do that? Or is cursor placement entirely out of the

Re: [HarfBuzz] harfbuzz-ng: Changes to 'graphite2'

Dear Behdad, You mentioned that you intend to simplify the whole userdata approach for both uniscribe and graphite, which is a great idea. One thing that is worth pointing out is that graphite is somewhat unique in needing a difference between undef (not yet tried to create a gr_face) and NULL

[HarfBuzz] what not to indic shape

Dear Behdad, Here is a list of scripts that I think shouldn't be using the indic shaper. Justification of simple means that there is no reordering or conjuncts involved and that there is probably no actual shaping (so just generic shaping will be sufficient). BATAK: ? Simple BRAHMI:

Re: [HarfBuzz] what not to indic shape

Dear Behdad, The questionable ones are rare scripts and I would take the risk of treating them as simple given there is no reordering involved. We can always move them with a bug report. One key aspect is that there is no reordering involved, and that's a key issue. The way I

Re: [HarfBuzz] hb-view assert with graphite version of Scheherazade

Dear Behdad, Humm. As it is the logic in hb-graphite.cc only handles LTR runs. I'm not fluent enough in Graphite to try to fix that. I'll wait to see what Martin has to offer. OK. Rather than bend my brain around all the possible configurations of clustering in RTL text, let me ask the

Re: [HarfBuzz] hb-view assert with graphite version of Scheherazade

Dear Behdad, Humm. As it is the logic in hb-graphite.cc only handles LTR runs. I'm not fluent enough in Graphite to try to fix that. I'll wait to see what Martin has to offer. OK. Rather than bend my brain around all the possible configurations of clustering in RTL text, let me ask the

[HarfBuzz] applying the graphite patch

Dear Behdad, I notice that you have made various fixes to the hb-graphite code, but you seem not to have applied the patch I sent you that both re-enables graphite and fixes the rtl problem. At least that's my understanding from the git repo pull I have here. TIA, GB, Martin

Re: [HarfBuzz] Tai Tham / Lanna (iso15924=lana) shaping question

Dear Behdad, Good to know. I'll give HB a run on my Myanmar corpus and see if I can fix a few high-impact issues. I would commend UTN#11 as worth reading at least the first half, on this. It'll give you a good feel for what's involved. In the case of Tai Tham, we took the Myanmar model as

Re: [HarfBuzz] HarfBuzz.old backend in new HarfBuzz

Dear Behdad, Pretty much like the Uniscribe and CoreText backends, this new backend is primarily for testing, and may be removed in the future (after I have convinced everyone to move to the real HarfBuzz). Is now the right time to ask for the reenabling of the graphite backend? Yours,

[HarfBuzz] non-portable linkages

Dear Behdad, I would just like to point out that linking harfbuzz or anything against ICU, glib, freetype makes the library just as non-portable as linking against graphite2. Yours, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org

[HarfBuzz] Adding font debugging

Dear Behdad, It would be great if we could add a little telemetry to harfbuzz to make life easier for font developers so they can find out why their font behaves other than how they are expecting it to behave. In the Graphite project we've been working on a gui based debugger that does after

[HarfBuzz] building debug

Dear All, Probably a dumb question, but how do I build harfbuzz with debug turned on? I was hoping for a --enable-debug option to configure or some such. TIA, Yours, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org

Re: [HarfBuzz] pangocairo on Mac: transformation matrix and fontsize

Dear Behdad, The next hold-up is some graphite2 build problems and test failures on PowerPC that I'm dealing with that developer about. Please package without graphite2. That's my recommendation anyway. And my recommendation is the opposite (of course). Please do package with graphite

Re: [HarfBuzz] pangocairo on Mac: transformation matrix and fontsize

Dear Khaled, The warning comes from libtool, I also get it when linking XeTeX with system harfbuzz and graphite2. I don’t have the slightest idea what it means, but it seems to be caused by this line in libgraphit2.la: # Should we warn about portability when linking against -modules?

[HarfBuzz] patch to add getters for gr_font and gr_face

Dear Behdad, Khaled says he needs this for xetex. I enclose a patch to add getter functions to hb-graphite2 for gr_font and gr_face. No idea if the work though ;) Yours, Martin diff --git a/src/hb-graphite2.cc b/src/hb-graphite2.cc index 6c890d4..06eca24 100644 --- a/src/hb-graphite2.cc +++

Re: [HarfBuzz] Tai Tham NGA, SAKOT is not Kinzi

Dear Richard and Thep, In a Lanna tutorial [1], it's stated in page 12 that MAI KANG LAI is placed on the second consonant only. But the position is actually in the middle. [1] http://dl.dropbox.com/u/12266813/TaiTham/lanna-tutorial.pdf I think this is a wrong analysis. On p23 of the same

Re: [HarfBuzz] Change in HarfBuzz after version 0.90 ?

Dear Ed, When shaping text, the correct ordering for your typical consonant-vowel-consonant syllable is: BASE_CONSONANT + (i)VOWEL_MARK + (ii) TONE MARK +(iv) U+1A60 SAKOT + (iii) SUBJOINED CONSONANT With Harfbuzz 0.90, I get the shaping that I expect as shown in the first image. But with

Re: [HarfBuzz] Question regarding the use of HB_SCRIPT_KATAKANA for regular Japanese

Dear All, https://github.com/arielm/Unicode/blob/master/Projects/ScriptDetector This is awesome! Thank you. As I work with minority languages, automatic language detectors make me shudder and cry. Please do not assume that because something is in, say Thai script, that it is in Thai

Re: [HarfBuzz] Thai below-base normalization

Dear All, It's not clear to me, then, why uniscribe treats this in the way it does. (Perhaps there was no good reason, and it was merely an arbitrary choice of ordering in the absence of any clear requirement?) In my data, I have no examples of a U+0E3A occurring after/below U+0E38/9. So I

[HarfBuzz] Adding debug support to hb

Dear Behdad, As we approach a v1.0 I would like to encourage us to add debug support for font developers. Looking at the rather nice hb_auto_trace, it would really help if the trace routine could be passed the buffer as well as the lookup being traced. That way a font debugger could show the

[HarfBuzz] Fixing Khmer U+17DD

Dear Behdad, Khmer has a rare character U+17DD used by linguists and minorities and it isn't shaping correctly in that there is a dotted circle inserted if it occurs before U+17C8. The fix to this is to change the test for vowel to include U+17DD. In hb-ot-shape-complex-indic.cc, near the

[HarfBuzz] bug in graphite integration

Dear Behdad, I came across this bug in the graphite integration code. The cluster component of the info structure needs to point back into the original string. Currently the code is returning a character offset that is zero based. The fix is to use the cluster attributes passed in. I don't

Re: [HarfBuzz] behavior of mark-width-zeroing

Dear Jonathan, Our current mark-zeroing code, in zero_mark_widths_by_gdef() and zero_mark_widths_by_unicode(), modifies only the advance of the glyphs, so that they no longer take up any space on the line. I'm wondering whether we should also adjust the offset, by subtracting the

Re: [HarfBuzz] Inter-glyph spacing

Dear Simon, What also confuses me is that the result is very font-specific. SIL fonts are squashed. Times and Optima render perfectly: Pango and Harfbuzz equivalent. Adobe Garamond Pro and Caslon Pro are horrible, with some very strange inter-glyph spacing; in particular there is

Re: [HarfBuzz] Inter-glyph spacing

Dear Simon, if (FT_Set_Char_Size(uds-ft_face,f-pointSize * 64.0, 0, 0, 0)) return 0; What also confuses me is that the result is very font-specific. SIL fonts are squashed. Times and Optima render perfectly: Pango and Harfbuzz equivalent. Adobe Garamond Pro and

Re: [HarfBuzz] feature execution order

Dear Bob, I'm copy/pasting some stuff I wrote a while back in another thread. In general, look for add_gsub_pause in harfbuzz source code and you get to the place you need. Which, from reading the code for arabic results in the following feature order. Merged features are listed on the same

Re: [HarfBuzz] feature execution order

Dear Bob, Further to the script specific list. The general features list plan is as follows, again, all features on the same line are merged into one pass, except the script specific features which follow the plan laid down for that script. rtla, rtlm, frac, numr, dnom script specific features

Re: [HarfBuzz] Thai text render issue in Android L

Dear Roozbeh, My mistake. This is intentional. Basically, the grapheme cluster would go too deep for Android, so Noto Sans Thai UI pushes the SARA UU to the left so it can show something instead of making SARA UU disappear. Moving the sara uu to the side is certainly a novel solution to

Re: [HarfBuzz] Unsure how to use HB output to combine characters

Dear Simon, When I try to shape the diglyph སྐ (TIBETAN LETTER SA U+0F66, TIBETAN SUBJOINED LETTER KA U+0F90) in the Kokonor font I get back two glyph values, 118 and 160: 1: (སྐ) 1: U+0F66,U+0F90 1: [118=0+1539|160=0+0] Is it that the Kokonor font has the diacritics as overstriking,

[HarfBuzz] Unsure how to use HB output to combine characters

Dear Simon, When I try to shape the diglyph སྐ (TIBETAN LETTER SA U+0F66, TIBETAN SUBJOINED LETTER KA U+0F90) in the Kokonor font I get back two glyph values, 118 and 160: 1: (སྐ) 1: U+0F66,U+0F90 1: [118=0+1539|160=0+0] Is it that the Kokonor font has the diacritics as overstriking,

[HarfBuzz] using HB_DEBUG

Dear Behdad, How does one use HB_DEBUG? Are there magic values? Is there a configure option? How do you typically turn on HB_DEBUG? TIA, Yours, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org

[HarfBuzz] towards an OT debugger

Dear Behdad, Thanks for your help in getting the debug output from hb. I've been playing and have some thoughts about how to use the existing debug framework to help with font development. 1. It would be great if the HB_DEBUG variable controlled which categories of debug are output, rather

[HarfBuzz] is opentype crazy

Dear All, It seems that if I want to insert a glyph (say a dotted circle) between two diacritics, then I can. I can sub diac1' diac2 by diac1 dottedcircle or something akin to that. But when I do, because diac1 is a mark and not a ligature, harfbuzz resets the advance of my dottedcircle to 0,

Re: [HarfBuzz] Choosing dev2 vs deva OT script for shaping

Dear Behdad, I can extend the BCP 47 extension to also choose the script system if it's available. Eg, a language setting of x-hbotdeva will choose deva whereas x-hbotdev2 will choose dev2. This works for script tags that have four letters (ie, not 'lao ', 'yi ', 'nko ', and 'vai '). We

[HarfBuzz] Loading Graphite dynamically

Dear All, A number of people have asked me for a mechanism by which graphite may be dynamically loaded only when a Graphite font is used. I've struggled with the notion of this, but I think I understand it now. I hope that this can help everyone to have what they want for minimal cost. I've

Re: [HarfBuzz] Loading Graphite dynamically

users to ship with the Graphite library, unless they want to. HTH, Yours, Martin Thanks, behdad On 15-05-18 05:50 AM, Martin Hosken wrote: Dear All, A number of people have asked me for a mechanism by which graphite may be dynamically loaded only when a Graphite font is used

Re: [HarfBuzz] Loading Graphite dynamically

Dear Jonathan, AFAICS, the patch will leak the graphite2_funcs_t record that's attached to the face, as it fails to free it in _hb_graphite2_shaper_face_data_destroy. Thanks for flagging that. I've restructured grfuncs to be part of the shaper_data so there is no m/calloc to free now. It

Re: [HarfBuzz] Loading Graphite dynamically

the framework on which the application is based. Yours, Martin Thanks, behdad On 15-05-18 05:50 AM, Martin Hosken wrote: Dear All, A number of people have asked me for a mechanism by which graphite may be dynamically loaded only when a Graphite font is used. I've struggled

Re: [HarfBuzz] Loading Graphite dynamically [2]

Dear Behdad, 3. Why stop at Graphite? Why not use this for ICU, FreeType, glib, gobject, fontconfig, as well as others? That way I can have one libharfbuzz.so with all the bits and pieces without pulling in 100MB worth of libraries; and we can fold libharfbuzz-icu.so back into

Re: [HarfBuzz] Loading Graphite dynamically

Dear Behdad, 1. Can distro people please chime in with their preferences? Debian, Ubuntu and Fedora and derivatives (AFAIK) make their harfbuzz packages dependent on the libgraphite package. Thus they all enable Graphite at the system level. libgraphite is too small not to ship. The only

Re: [HarfBuzz] Streamlining hb_font_t some more

Dear Simon, > In InDesign, both (1) and (2) get 12 x 1.2 = 14.4pt interline space. > This means that the descender of the "p" in "ipsum" will bump into > letters on the next line. That's clearly wrong. > > In the CSS model, both (1) and (2) get half leading added to the top and > the bottom of

[HarfBuzz] USE and zeroing marks

Dear Behdad, I see from the code that the USE shaper doesn't zero marks. But that the USE spec implies that they are: the width of the base character must be added back using the feature. This is necessary because OT processing cancels the width associated with a mark. It is necessary to

[HarfBuzz] Spacing Mark Proposal

Dear All, Here is an initial draft of a proposal to change OpenType to support spacing marks. # Spacing Marks This is a proposal to extend the OpenType standard to support spacing marks. Spacing marks are marks considered to have extent. When attached to a base or another mark, such marks

[HarfBuzz] Move Lookup Proposal

Dear All, This is a proposal to add a move lookup to GSUB: # Move Lookup This is a proposal to add a GSUB lookup to support glyph movement. The purpose of this lookup is to move a glyph relative to its current position in the glyph string. The lookup also supports swapping two glyphs. Most

[HarfBuzz] hb-graphite2 patch progress chase

Dear Behdad, Any progress on accepting that core of the patch I sent you? I notice that it fixes a bug in Sile, for example. Yours, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Kerning, glyph width, and x advance

Dear Simon, > So I'm using Harfbuzz to shape stuff and put it out to PDF. When you > output a string in PDF, you are expected to kern it manually, or else > each glyph will be placed one after the other with no kerning: > > No kerning: Td (VAVAVOOM) Tj > Kerning: Td[(V) 153 (A) 122 (V) ...

Re: [HarfBuzz] harfbuzz: Branch 'master'

Dear Behdad, > > So, I would add an enum to the debug message to give a debug message event > > type. > > My current thinking is that everything is transferred as a text API in > one-line messages. The client can transform that to an enum if desired. That works only if the messages are

Re: [HarfBuzz] harfbuzz: Branch 'master'

Dear Behdad, > buf = hb.buffer_create () > +class Debugger(object): > + def message (self, buf, font, msg, data, _x_what_is_this): > + print(msg) > + return True > +debugger = Debugger() > +hb.buffer_set_message_func (buf, debugger.message, 1, 0) > hb.buffer_add_utf8

Re: [HarfBuzz] harfbuzz: Branch 'master'

Dear Behdad, > buf = hb.buffer_create () > +class Debugger(object): > + def message (self, buf, font, msg, data, _x_what_is_this): > + print(msg) > + return True > +debugger = Debugger() > +hb.buffer_set_message_func (buf, debugger.message, 1, 0) > hb.buffer_add_utf8

[HarfBuzz] uniscribe confusion

Dear Behdad, This is default shaping. The u1037 is attached to the u102F and we get the following results: * uniscribe: [u1000=0+1002|u103D=1@-55,0+0|u102F=2+147|u1037=3@217,0+0] * harfbuzz: [u1000=0+1002|u103D=0@-55,0+0|u102F=0+147|u1037=0@217,0+0] which makes sense. Now if we introduce an

Re: [HarfBuzz] have markfiltersets ever worked?

Dear All, > > I meant 1.2.3 and 1.2.4 are broken. I may be wrong, but my understanding is that this error came in with revision: 9a13ed453ef96822a47d6e6f58332b87f38d5c59 which was released with 1.2.1. So it may be a bit wider than that. I don't think android has this in, at least not in

Re: [HarfBuzz] difference between harfbuzz and uniscribe?

Dear Behdad, Some pertinent further information and that is that both diacritics were previously attached to the base before these rules are applied. > Consider a font with the following FEA rule in it: > > pos u1014' 79 u1032 u1037; > > which adds 79 to the advance of the first glyph in the

[HarfBuzz] difference between harfbuzz and uniscribe?

Dear Behdad, Consider a font with the following FEA rule in it: pos u1014' 79 u1032 u1037; which adds 79 to the advance of the first glyph in the sequence u1014 u1032 u1037: [u1014=0+609|u1032=0@-89,-42+0|u1037=0@-55,0+0] vs without the 1037: [u1014=0+530|u1032=0@-10,-42+0] But if we

[HarfBuzz] have markfiltersets ever worked?

Dear All, Has anyone had any success with mark filter sets? According bug #238 there is a problem with hb-open-type-private.hh where the to_int() function is programmed wrongly (need to multiply sizeof() by 8 when shifting). This would mean that no mark filter set lookups will ever succeed.

Re: [HarfBuzz] Python: accessing harfbuzz enums

On Sun, 19 Jun 2016 16:34:23 -0400 Kelvin Ma wrote: > When using the python harfbuzz bindings, how do I access the harfbuzz enums > for direction, script, & language? Also can someone explain the difference > between direction and script because I thought script

[HarfBuzz] parse_one_feature

Dear Behdad, I notice that hb-shape has a parse_one_feature function, but that nothing refers to it and it is not in the public API. Does this mean that it is deprecated or that one day you will publicise it? Yours, Martin ___ HarfBuzz mailing list

Re: [HarfBuzz] harfbuzz: Branch 'master'

Dear Behdad, > - if (buffer == NULL) > + if (!buffer) Always wanting to learn. How does this cause a divide by zero? Or what led you to make the change? TIA, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org

[HarfBuzz] seems not to build without fallback shaper

Dear Behdad, I recently had occasion to build harfbuzz without setting -DHAVE_FALLBACK and it didn't do too well. I don't think it unreasonable to require the fallback shaper. But in that case, perhaps it's worth getting rid of the need for -DHAVE_FALLBACK. Minor point, etc. But wanted to

[HarfBuzz] unsafe to break

Dear Behdad, Please could you explain the purpose and function of HB_GLYPH_FLAG_UNSAFE_TO_BREAK. Is this about line breaking? grapheme clustering? TIA, Yours, Martin ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org

Re: [HarfBuzz] query cvXX feature name table references

On Mon, 30 Apr 2018 23:22:29 +0200 Khaled Hosny <khaledho...@eglug.org> wrote: > On Mon, Apr 30, 2018 at 08:50:57PM +0700, Martin Hosken wrote: > > Dear Behdad, > > > > Do you have any plans (pretty please) to add an API to enable a client > > to query a f

[HarfBuzz] script segmentation

Dear All, One problem I am facing as we add characters to Unicode, is that if a character is added to a block, it doesn't necessarily mean that an existing application will keep that character in the same run as other characters in the same script of that block. This means the app is broken

Re: [HarfBuzz] script segmentation

Dear Richard, I would reply to my own message, but that never comes back to me. Thank you (NOT) Google. So here goes. I've started a discussion document here: https://github.com/OpenType/opentype-layout/blob/master/docs/script_segmentation.md. Please feel free to interact with it. If you are

Re: [HarfBuzz] [docs][question] Emoji "shaping", etc.

Dear Nathan, > Kind of a high-level question about describing HB functionality > > Would you consider the handling of emoji variation-selector sequences to be > "shaping", or some other operation? > > Feels like kind of a gray area to me; when it comes to describing what > HarfBuzz does and

[HarfBuzz] Changes in SIL

Dear All, This is an internal name change announcement that may be relevant to folk here: Within SIL the team that has worked on scripts and writing system issues has been known as the NRSI (Non-Roman Scripts Initiative). But as the team's work has widened to include Roman scripts (since 2001)

Re: [HarfBuzz] Why mark glyphs are skipped when MarkBasePos matching?

Dear Shusaku, > For example, the following glyph sequence is input. B is a glyph in > Base Coverage Table, M is a glyph in Mark Coverage Table > and m is a mark glyph defined by GDEF. > > B m M > > The implementation of HarfBuzz finds B as base of M even though there > is m between B and

[HarfBuzz] problems with hmn

Dear Behdad, I notice that there are a lot of language tags that are mapped to the internal OT language tag HMN. Given that a bunch of those are Miao languages that have significant stylistic variation in them, I am interested in removing their unification. Do you know where this set of

Re: [HarfBuzz] Exposing attachment tree / Arabic joining to shaping clients

Dear Behdad, > For reason that many of you know (letter-spacing, Arabic elongation, other > postprocessing) I like to expose attachment data to the shaping clients. > There's two separate pieces so far: > > - The Arabic joining info, which is applicable to all Arabic-like fonts > even the ones