Re: [HarfBuzz] Fwd: Harfbuzz with linebreaking

[Simon Cozens]

On 14/06/2016 14:23, kelvinsthirt...@gmail.com wrote:
> each word has at least
> one (often many) breakpoints, but only one of them gets used per
> line.

Right.

> And the only way to know which one to use is to shape.

Well, no. You shaped already; that was the first thing you did. As Adam
told you, the only way to know which breakpoint to *use* is to run a
justification algorithm, which you need to code. You're currently
thinking about a simple first-fit algorithm, which chops the glyphs into
lines once they get to be longer than the target line length; that's
fine, although you may find that a best-fit algorithm which performs
dynamic programming over all possible breakpoints gives you a neater
paragraph.

Now, shaping determines the glyph widths for you (which is the input to
your line breaking algorithm), but it is your code which is responsible
for finding the *possible* breakpoints in the text, at the language
level, and your code which is responsible for determining the *actual*
breakpoints at the shaped-glyph level.

Here we go then. If you want to use Harfbuzz to shape lines into
paragraphs, here is what you need to do:

* Perform the first stage of the bidi algorithm to organise the text
into same-direction runs. (Really, don't leave this bit out, and don't
think "I'll add RTL/complex script support later", because that never
works out well and because we already have enough typesetters that only
handle Latin.) ICU does this.

* Shape the paragraph, keeping note of the connection between the glyphs
and the text. Harfbuzz does this.

* Find the breakpoints in the text, using language analysis on the
characters. ICU does this.

* Create a data structure representing the runs of Harfbuzz output
between the breakpoints - TeX and SILE call these runs "nnodes" - and
the potential breakpoints themselves - "penalty nodes" (for breaking
inside a "word") and "glue nodes" (for whitespace between "words").
Assign widths to the nnodes by summing the widths of the shaped glyphs
inside them. You can put each glyph into its own nnode instead of
consolidating each run into an nnode if it's conceptually easier, but it
just slows your justification algorithm down.

Here's what my data structure looks like at this stage:

N<19.71pt>(Take)G<2.6pt>N<22.06pt>(these)G<2.6pt>N<15.37pt>(five)G<2.6pt>N<40.42pt>(sentences)G<2.6pt>N<25.17pt>(which)G<2.6pt>N<2.97pt>(I)G<2.6pt>N<19.95pt>(need)G<2.6pt>N<8.47pt>(to)G<2.6pt>N<23.24pt>(break)G<2.6pt>N<16.69pt>(into)G<2.6pt>N<4.58pt>(a)G<2.6pt>N<42.68pt>(paragraph)N<2.29pt>(.)

(Each nnode also contains a list of glyph IDs and widths.) Each of the
glue nodes are potential break points; these were obtained by checking
the Unicode line break status of each character. The space character
0x20 is breakable, so it gets turned into a glue node.

* Run your justification algorithm to determine which breakpoints should
be used. Your code does this.

* If the algorithm does not produce a tight enough paragraph, break open
the nnodes by hyphenating the text, reshaping them into new nnodes, and
putting a discretionary breakpoint in the middle.

Now it looks like this:

N<19.71pt>(Take)G<2.64pt>N<22.06pt>(these)G<2.64pt>N<15.37pt>(five)G<2.64pt>N<13.99pt>(sen)D(N<3.36pt>(-)||)N<26.43pt>(tences)G<2.64pt>N<25.17pt>(which)G<2.64pt>N<2.97pt>(I)G<2.64pt>N<19.95pt>(need)G<2.64pt>N<8.47pt>(to)G<2.64pt>N<23.24pt>(break)G<2.64pt>N<16.69pt>(into)G<2.64pt>N<4.58pt>(a)G<2.64pt>N<18.43pt>(para)D(N<3.36pt>(-)||)N<24.24pt>(graph)N<2.29pt>(.)

* Run your justification algorithm again on this new node list.

On a 100pt column, my algorithm determined that the line breaks are at
position 10 and position 22 of the node list array.

* Organise your node list into a list of lines, based on the breakpoints
that were fired.

I split my node list at positions 10 and 22, so my lines are:

N<19.71pt>(Take)G<2.64pt>N<22.06pt>(these)G<2.64pt>N<15.37pt>(five)G<2.64pt>N<13.99pt>(sen)D(N<3.36pt>(-)||)N<26.43pt>(tences)G<2.6pt>

N<25.17pt>(which)G<2.6pt>N<2.97pt>(I)G<2.6pt>N<19.95pt>(need)G<2.6pt>N<8.47pt>(to)G<2.6pt>N<23.24pt>(break)G<2.6pt>N<16.69pt>(into)

N<4.58pt>(a)G<2.6pt>N<18.43pt>(para)D(N<3.36pt>(-)||)N<24.24pt>(graph)N<2.29pt>(.)

* For each line in the paragraph, apply the second part of the bidi
algorithm (ICU does this) and reshape where necessary. This splits and
recombines ligatures correctly. (I promise; we have a test case to prove
this.)

You only need to determine line breaks once, and you only need to
reshape once per line maximum. I'm not going to argue about whether it
works or not, because you can check out the code and the test suite for
yourself: https://github.com/simoncozens/sile

> In fact I don’t see any other way to do it

You need to put aside the idea that there is a connection between
shaping and determining which breakpoints to use. There isn't one, and
this is the mental block which is stopping you from seeing solutions to
your problem.

Simon

Re: [HarfBuzz] Fwd: Harfbuzz with linebreaking

[kelvinsthirteen]

No, shaping does tell you about breakpoints in the same way that shaping can 
link back to original character indexes for any other purpose.

For example, if the shaped sentence "make a difference!" overflows at the glyph 
“ff”, I know to test break points that come before the 11th character in the 
string ('f'), since “e”, the glyph that comes after “ff” has a cluster value of 
11. So I would shape "make a dif-", (and then "make a " instead if that didn’t 
fit). We know that  f|f  is a breakpoint from analysis of the text, but we only 
know to USE the  f|f  breakpoint, and not some other breakpoint like the  r|e  
breakpoint after it, from looking at the shaping.

In fact I don’t see any other way to do it; each word has at least one (often 
many) breakpoints, but only one of them gets used per line. And the only way to 
know which one to use is to shape.

And the safe to break API is an optimization, but a very important 
one—otherwise the amount of shaping that has to be done explodes out of 
proportion from the number of lines that are produced because of that 
multiplication.

On Jun 13, 2016, at 11:50 PM, Simon Cozens  wrote:
> 
>> On 14/06/2016 13:16, Kelvin Ma wrote:
>> The problem is, I have no idea where, in terms of x-coordinate, any of
>> these breakpoints are going to be until I shape them. So I will have to
>> shape the entire sentence.
> 
> You do have to shape the entire sentence, yes. But that's got nothing to
> do with where your breakpoints are. Shaping doesn't tell you anything
> about break points. That's why it's not Harfbuzz's job.
> 
> Let's take your example again. How do you get from:
> 
>> Take these five sentences which I need to break into a paragraph. The
>> shaper is always going to be involved in this. Did you only count two?
> 
> to:
> 
>> |Take |these |five |sen-|ten|-ces |which |I |need |to |break |into |a
>> |para-|graph. |The |sha-|per |is |al-|ways |go-|ing |to |be
>> |in-|vol-|ved |in |this. |Did |you |on-|ly |count |two?|
> 
> Shaping won't give you the hyphenation points. That's a matter of
> language analysis. So you have to do line breaking at the text layer;
> this is *why* Harfbuzz doesn't involve itself in line breaking.
> 
>> You are right. But I hope I explained why the shaping information has to
>> come before the textual-breakpoint information, because without shaping,
>> you don’t know *where* the breakpoints lie
> 
> Even with shaping, you don't know where the breakpoints lie. Shaping and
> breakpoints are completely orthogonal.
> 
> And line breaking *will* change your shaping - think about what happens
> when you break an RTL Hebrew sentence inside a majority LTR text - you
> need to swap around the two ends of the sentence. The safe-to-break API
> is just an optimization; it says, if you break at this point, you won't
> have to reshape. But it doesn't tell you anything about where you
> *should* or *shouldn't* break - that depends entirely on your
> justification strategy.
> 
> Simon
> 
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Fwd: Harfbuzz with linebreaking

[Simon Cozens]

On 14/06/2016 13:16, Kelvin Ma wrote:
> The problem is, I have no idea where, in terms of x-coordinate, any of
> these breakpoints are going to be until I shape them. So I will have to
> shape the entire sentence.

You do have to shape the entire sentence, yes. But that's got nothing to
do with where your breakpoints are. Shaping doesn't tell you anything
about break points. That's why it's not Harfbuzz's job.

Let's take your example again. How do you get from:

> Take these five sentences which I need to break into a paragraph. The
> shaper is always going to be involved in this. Did you only count two?

to:

> |Take |these |five |sen-|ten|-ces |which |I |need |to |break |into |a
> |para-|graph. |The |sha-|per |is |al-|ways |go-|ing |to |be
> |in-|vol-|ved |in |this. |Did |you |on-|ly |count |two?|

Shaping won't give you the hyphenation points. That's a matter of
language analysis. So you have to do line breaking at the text layer;
this is *why* Harfbuzz doesn't involve itself in line breaking.

> You are right. But I hope I explained why the shaping information has to
> come before the textual-breakpoint information, because without shaping,
> you don’t know *where* the breakpoints lie

Even with shaping, you don't know where the breakpoints lie. Shaping and
breakpoints are completely orthogonal.

And line breaking *will* change your shaping - think about what happens
when you break an RTL Hebrew sentence inside a majority LTR text - you
need to swap around the two ends of the sentence. The safe-to-break API
is just an optimization; it says, if you break at this point, you won't
have to reshape. But it doesn't tell you anything about where you
*should* or *shouldn't* break - that depends entirely on your
justification strategy.

Simon

___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Fwd: Harfbuzz with linebreaking

[Kelvin Ma]

On Mon, Jun 13, 2016 at 10:53 PM, Simon Cozens 
wrote:

> On 14/06/2016 12:42, Kelvin Ma wrote:
> > What I need is something to bridge that gap between the 1-line of
> > unbroken text that harfbuzz generates, and the fragments I need to be
> > able to assemble a multi-line paragraph.
>
> Right. You need that, but it's not Harfbuzz's job. Write some code. :-)
>
> > The only way to get these
> > pieces is to find the spots in the shaped text where the whole line can
> > be shaped in two pieces with an identical result.
>
> Wrong. What you need to find is the potential line breaks. That's not a
> shaping issue specifically; it's a text issue, and needs to be dealt
> with at the text level.


No, this is also a shaping issue and i’ll explain why.

*Take these five sentences which I need to break into a paragraph. The
shaper is always going to be involved in this. Did you only count two?*

It has potential breakpoints here:

|Take |these |five |sen-|ten|-ces |which |I |need |to |break |into |a
|para-|graph. |The |sha-|per |is |al-|ways |go-|ing |to |be |in-|vol-|ved
|in |this. |Did |you |on-|ly |count |two?|

The problem is, I have no idea where, in terms of x-coordinate, any of
these breakpoints are going to be until I shape them. So I will have to
shape the entire sentence.

Then I find that the first glyph that overruns the width of the line is the
‘e’ in “sentences”:

*Take these five sente*

Now I know that I can cut this down to a correct line break by just shaping
the text “*Take these five sen-*” and testing to see if that fits (with the
“safe-to-break” thing, I can probably just keep the old “Take these five
se” glyphs and append a newly shaped “n-”.)

The problem comes with what to do with the text that comes after the
breakpoint. Without “safe-to-break” I have to reshape the *entire*
remainder of the paragraph, the whole text “*tences which I need to break
into a paragraph. The shaper is always going to be involved in this. Did
you only count two?*”. If the paragraph is long, this can be a very long
string. If I had the “safe-to-break” thing, I could find that I could keep
that portion of the originally shaped line, or at worst, maybe have to
reshape a “te” or something and append the old “*nces which I need to break
into a paragraph. The shaper is always going to be involved in this. Did
you only count two?*” to it.

The amount of text that has to be laid out is the entire length of the
paragraph, PLUS *half the entire length of the paragraph times the number
of lines*. That last part is crucial. With “safe-to-break” it’s just the
length of the paragraph, plus a few bits and pieces of fractured text here
and there.


> Taking the example of a ligature, it *is* allowable to break (with
> hyphenation) in the middle of a ligature like "fi". Indeed, your
> justification engine might decide, for the good of the rest of the lines
> in the paragraph, that this is the best place to break. If all you are
> dealing with is the glyph output from Harfbuzz, you won't be able to
> spot that breakpoint.
>
> Once you get into non-Latin scripts, things get worse. Finding
> breakpoints is a matter that depends entirely on the rules of the script
> or language that your text is written in. Right now I'm fighting with
> Javanese, where line breaks are permissible at the end of syllables. You
> need to parse the text, not the glyphs, to determine the appropriate
> breaks. Like others have said: use ICU or similar.
>
> And so you need to deal with two sets of information at the same time:
> the text-level information about breaks, and the shaper-level
> information about glyphs. This is why Harfbuzz returns you an index into
> your text string, so that you can keep those two sets of information in
> sync. The hard part of writing a typesetting system is dealing with the
> interplay between those two representations of a text.
>

You are right. But I hope I explained why the shaping information has to
come before the textual-breakpoint information, because without shaping,
you don’t know *where* the breakpoints lie, and if you don’t know where
they lie, they don’t function as breakpoints anymore.


>
> It took me quite a while to get my head around this, and a lot of help
> from others. You can see the record of me banging my head against this
> particular wall at https://github.com/simoncozens/sile/issues/179 ,
> which has a nice explanation of the issues involved.
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Harfbuzz with linebreaking

[Simon Cozens]

On 14/06/2016 01:00, Kelvin Ma wrote:
> So I’ve not received an answer to this anywhere, so, how do I typeset
> paragraphs with Harfbuzz?

http://behdad.github.io/harfbuzz/hello-harfbuzz.html#what-harfbuzz-doesnt-do

Sorry, I need to finish writing the manual; will have more time soon.

Simon
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

[HarfBuzz] Fwd: Harfbuzz with linebreaking

[Kelvin Ma]

Also this might be jumping the gun a bit since we still don’t even have the
“safe-to-break” thing yet but how do we know when it’s okay to reshape a
broken segment or if it should just be moved to the next line? A ligature
like “ffi” can and should be broken into “f” and “fi” if needed (like
“ef-ficiency”) but a ligature like “‍‍‍” should never be shaped in
2 pieces

On Mon, Jun 13, 2016 at 10:33 PM, Kelvin Ma 
wrote:

> If you mean how does the linebreaking work in Knockout, it places
> characters/glyphs (it works 1-to-1 character-to-glyph, no ligatures,
> substitutions) until the glyphs overrun the line length. Then it goes
> backwards to the last space character and forward to the next space
> character to get the word that the line was broken on. Then it iterates
> through hyphenation points until one of them is short enough to accommodate
> both itself and a hyphen glyph. Because it works with a 1-to-1 relationship
> between glyphs and characters it just counts the number of glyphs printed
> and uses that as the index offset for laying out the next line.
>
> It’s a slightly more complicated process than that in reality because it
> also checks for other breaking characters like em dashes and slashes not
> just spaces, and it skips hyphenations that contain more than the original
> amount of characters. It can also do pairwise kerning (checks current and
> previous glyph to get kerning offset) but it doesn’t work often in practice
> because most modern fonts encode the kerning data in a different table than
> where freetype knows to look for it, and I’m currently using freetype to do
> what harfbuzz should be doing.
>
> It works in normal time which is nice but not ideal because there is no
> ligatures, little kerning, no substitutions, etc. So no opentype features
> basically. Which is why I need harfbuzz.
>
> On Mon, Jun 13, 2016 at 10:01 PM, Khaled Hosny 
> wrote:
>
>> On Mon, Jun 13, 2016 at 11:00:16AM -0400, Kelvin Ma wrote:
>> > So I’ve not received an answer to this anywhere, so, how do I typeset
>> > paragraphs with Harfbuzz? How do I use the ‘safe-to-break’ iterator?
>>
>> How are you doing line breaking right now?
>>
>> Regards,
>> Khaled
>>
>
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

[HarfBuzz] Fwd: Harfbuzz with linebreaking

[Kelvin Ma]

What I need is something to bridge that gap between the 1-line of unbroken
text that harfbuzz generates, and the fragments I need to be able to
assemble a multi-line paragraph.

What Freetype does is give me images of individual letters, which I can
then put together to make words and sentences, because words and sentences
are made up of individual letters. But I cannot do that with harfbuzz
because its output is one line of text, whereas I need smaller pieces that
I can use to build a paragraph. The only way to get these pieces is to find
the spots in the shaped text where the whole line can be shaped in two
pieces with an identical result. This requires knowledge of all the glyph
shaping rules and opentype features, which is why it sounds like it can
*only* be HarfBuzz’s responsibility.

There is only one “workaround” I’ve seen (the mozilla one) where it simply
assumes each space-separated word is a fragment that will not affect the
shaping of its neighbors. But this only works for justified text; in many
fonts the space is kerned with letters such as T and L.

On Mon, Jun 13, 2016 at 7:44 PM, Adam Twardoch (List) <
list.a...@twardoch.com> wrote:

> Both XeTeX and Mozilla Firefox have opensource code which uses HarfBuzz
> for line setting but combines it with other techniques to perform
> paragraph-level line breaking, hyphenation and justification.
>
> In a way, FreeType takes a font, glyph ids and their positions, and
> delivers an image. HarfBuzz takes a font and plain Unicode text, and
> delivers glyph ids and their positions in a line.
>
> But "something else" (e.g. code that needs to be written) would be needed
> for paragraphs.
>
> Paragraph setting is not at all easy, and can be done in a number of ways.
> If your paragraphs are always left-aligned, rectangular, and don't use
> hyphenation, then it’s easy. But if they need to wrap around graphics or
> use freeform margins, use hyphenation, multiple fonts etc. — the code needs
> to deal with situations that are completely out of scope for HarfBuzz. Yes,
> even setting one word in bold or italic would require the paragraph
> composer code to be aware of multiple fonts and some kind of “markup” or
> “rich text” — something that HarfBuzz doesn’t do and isn’t intended for.
>
> Best,
> Adam
>
> Sent from my mobile phone.
>
> On 13.06.2016, at 17:00, Kelvin Ma  wrote:
>
> So I’ve not received an answer to this anywhere, so, how do I typeset
> paragraphs with Harfbuzz? How do I use the ‘safe-to-break’ iterator?
>
> Because it sounds like right now I’d have to shape the *entire* paragraph
> as one line, then find the last shaped glyph that falls before the cutoff
> point, then iterate backwards from the character corresponding to the glyph
> following *that*, on each breakpoint, reshaping the whole line each time
> until something fits. Then I’d have to do that all over again for the
> second line until the entire paragraph is shaped. So to typeset 5 lines of
> text I’d have to shape about 22 lines of text. To typeset a 20 line
> paragraph I’d have to shape about 240 lines of text.
>
> Surely this is not how harfbuzz expects users to use it?
>
> (Asking because I need harfbuzz for my typesetter app
> )
>
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
>
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Harfbuzz with linebreaking

[Adam Twardoch (List)]

Both XeTeX and Mozilla Firefox have opensource code which uses HarfBuzz for 
line setting but combines it with other techniques to perform paragraph-level 
line breaking, hyphenation and justification. 

In a way, FreeType takes a font, glyph ids and their positions, and delivers an 
image. HarfBuzz takes a font and plain Unicode text, and delivers glyph ids and 
their positions in a line. 

But "something else" (e.g. code that needs to be written) would be needed for 
paragraphs. 

Paragraph setting is not at all easy, and can be done in a number of ways. If 
your paragraphs are always left-aligned, rectangular, and don't use 
hyphenation, then it’s easy. But if they need to wrap around graphics or use 
freeform margins, use hyphenation, multiple fonts etc. — the code needs to deal 
with situations that are completely out of scope for HarfBuzz. Yes, even 
setting one word in bold or italic would require the paragraph composer code to 
be aware of multiple fonts and some kind of “markup” or “rich text” — something 
that HarfBuzz doesn’t do and isn’t intended for. 

Best,
Adam

Sent from my mobile phone.

> On 13.06.2016, at 17:00, Kelvin Ma  wrote:
> 
> So I’ve not received an answer to this anywhere, so, how do I typeset 
> paragraphs with Harfbuzz? How do I use the ‘safe-to-break’ iterator?
> 
> Because it sounds like right now I’d have to shape the *entire* paragraph as 
> one line, then find the last shaped glyph that falls before the cutoff point, 
> then iterate backwards from the character corresponding to the glyph 
> following *that*, on each breakpoint, reshaping the whole line each time 
> until something fits. Then I’d have to do that all over again for the second 
> line until the entire paragraph is shaped. So to typeset 5 lines of text I’d 
> have to shape about 22 lines of text. To typeset a 20 line paragraph I’d have 
> to shape about 240 lines of text.
> 
> Surely this is not how harfbuzz expects users to use it?
> 
> (Asking because I need harfbuzz for my typesetter app)
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Harfbuzz with linebreaking

[Konstantin Ritt]

Like it was answered several times already, Harfbuzz doesn't do any
itemization, including line breaking. Feel free to choose ICU (or whatever
you like more) to do all the hard work for you.


Konstantin

2016-06-13 19:00 GMT+04:00 Kelvin Ma :

> So I’ve not received an answer to this anywhere, so, how do I typeset
> paragraphs with Harfbuzz? How do I use the ‘safe-to-break’ iterator?
>
> Because it sounds like right now I’d have to shape the *entire* paragraph
> as one line, then find the last shaped glyph that falls before the cutoff
> point, then iterate backwards from the character corresponding to the glyph
> following *that*, on each breakpoint, reshaping the whole line each time
> until something fits. Then I’d have to do that all over again for the
> second line until the entire paragraph is shaped. So to typeset 5 lines of
> text I’d have to shape about 22 lines of text. To typeset a 20 line
> paragraph I’d have to shape about 240 lines of text.
>
> Surely this is not how harfbuzz expects users to use it?
>
> (Asking because I need harfbuzz for my typesetter app
> )
>
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
>
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

[HarfBuzz] Harfbuzz with linebreaking

[Kelvin Ma]

So I’ve not received an answer to this anywhere, so, how do I typeset
paragraphs with Harfbuzz? How do I use the ‘safe-to-break’ iterator?

Because it sounds like right now I’d have to shape the *entire* paragraph
as one line, then find the last shaped glyph that falls before the cutoff
point, then iterate backwards from the character corresponding to the glyph
following *that*, on each breakpoint, reshaping the whole line each time
until something fits. Then I’d have to do that all over again for the
second line until the entire paragraph is shaped. So to typeset 5 lines of
text I’d have to shape about 22 lines of text. To typeset a 20 line
paragraph I’d have to shape about 240 lines of text.

Surely this is not how harfbuzz expects users to use it?

(Asking because I need harfbuzz for my typesetter app
)
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Windows build of harfbuzz with nmake

[Ebrahim Byagowi]

> He needs to be made aware that it's unbuildable with MSVC.

Is it really? Actually Visual Studio 2015 was only compiler I tested the
code against, but maybe older ones are broken. Anyway, the backend
shouldn't be used on any production code (specially as it's performance
issues) even the fact it currently partially supports an Arabic
typographical feature not offered by harfbuzz—which currently has not any
suitable way to be accessed from the API so nvm about it.

On Mon, Jun 13, 2016, 5:55 PM John Emmas  wrote:

> On 13/06/2016 13:51, Khaled Hosny wrote:
> > DirectWrite and Uniscribe backends should be disabled
> > by default since they are only for testing purposes
> >
> > [...]
> >
> > Or may be the issue is that you are using your own build system and just
> > included all the source files
> >
>
> Hi Khaled,
>
> You're right to an extent.  I did include 'hb-directwrite.cc' in my VC
> sources but it was mostly because of something I read in README.txt:-
>
>DIRECTWRITE: Enable (experimental) DirectWrite platform shaper
> support,
>requires a rather recent Windows SDK, and
> at least Windows Vista
>
> It looks like I misunderstood that because I took it to mean that
> 'hb-directwrite.cc' should be buildable with the more recent Windows
> compilers.  Thanks for the clarification.
>
> John
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Windows build of harfbuzz with nmake

[John Emmas]

On 13/06/2016 13:51, Khaled Hosny wrote:

DirectWrite and Uniscribe backends should be disabled
by default since they are only for testing purposes

[...]

Or may be the issue is that you are using your own build system and just
included all the source files



Hi Khaled,

You're right to an extent.  I did include 'hb-directwrite.cc' in my VC 
sources but it was mostly because of something I read in README.txt:-


  DIRECTWRITE: Enable (experimental) DirectWrite platform shaper 
support,
  requires a rather recent Windows SDK, and 
at least Windows Vista


It looks like I misunderstood that because I took it to mean that 
'hb-directwrite.cc' should be buildable with the more recent Windows 
compilers.  Thanks for the clarification.


John
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Windows build of harfbuzz with nmake

[Khaled Hosny]

On Mon, Jun 13, 2016 at 10:34:24AM +0100, John Emmas wrote:
> On 13/06/2016 10:09, Juha Martikainen wrote:
> > 
> > I had a second build attempt where I made my own vcxproj file. There I
> > get the following kind of errors:
> > 
> > 1>..\..\src\hb-directwrite.cc(246): error C2039: 'directwrite' : is not
> > a member of 'hb_shaper_data_t'
> > 1>..\..\src\hb-directwrite.cc(257): error C3861:
> > 'hb_directwrite_shaper_face_data_ensure': identifier not found
> > 
> 
> Hi Juha, I think you accidentally posted off-list.
> 
> You're right about the above and it's hugely frustrating.  I build with
> MSVC-8 and I had to comment out the whole of 'hb-directwrite.cc' because I
> couldn't make it compile.

That is why you need config.h file, to enable/disable optional features
as suitable. For example DirectWrite and Uniscribe backends should be
disabled by default since they are only for testing purposes (comparing
HarfBuzz output to MS implementations) and are not needed in production
code.

Or may be the issue is that you are using your own build system and just
included all the source files, in this case you can just skip
hb-directwrite.* and hb-uniscribe.* (among others, reading
src/Makefile.am should give some clues).

Regards,
Khaled
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Windows build of harfbuzz with nmake

[John Emmas]

On 13/06/2016 10:09, Juha Martikainen wrote:


I had a second build attempt where I made my own vcxproj file. There I 
get the following kind of errors:


1>..\..\src\hb-directwrite.cc(246): error C2039: 'directwrite' : is 
not a member of 'hb_shaper_data_t'
1>..\..\src\hb-directwrite.cc(257): error C3861: 
'hb_directwrite_shaper_face_data_ensure': identifier not found




Hi Juha, I think you accidentally posted off-list.

You're right about the above and it's hugely frustrating.  I build with 
MSVC-8 and I had to comment out the whole of 'hb-directwrite.cc' because 
I couldn't make it compile.  I added a note to say that I hoped it might 
build with some future version of MSVC but from what you're saying, it 
still won't compile.


Once again though...  commenting out the whole of 'hb-directwrite.cc' 
didn't seem to cause any problems.  I think it's a (comparatively) 
recent file so maybe its functionality isn't used very much as yet?  The 
file's author seems to be Ebrahim Byagowi. He needs to be made aware 
that it's unbuildable with MSVC.


John
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Windows build of harfbuzz with nmake

[John Emmas]

On 13/06/2016 08:36, Juha Martikainen wrote:


nmake /f Makefile.vc CFG=release /P

Resulting:
NMAKE : fatal error U1073: don't know how to make 'config.h.win32'
Stop.



Hi Juha,

In addition to what Cosimo said, it might be worthwhile to remember that 
it's not strictly necessary to have config.h for your eventual build.  
Wherever config.h gets #included in Harfbuzz, there's usually a guard, 
like this:-


  #ifdef HAVE_CONFIG_H
  #include "config.h"
  #endif

I build from git (using a VC project, rather than using nmake).  My 
project doesn't define HAVE_CONFIG_H and yet it's never caused any 
problems AFAIK.


John
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

[HarfBuzz] Windows build of harfbuzz with nmake

[Juha Martikainen]

Hello!

I am working with Windows 10 system and Visual Studio 2012 compiler. I 
would like to compile harfbuzz library for Scribus. I have taken the new 
harfbuzz codes with git.


I have read win32 directory readme file instructions and try to 
configure project using nmake. I run the following nmake command(in 
win32 directory):


nmake /f Makefile.vc CFG=release /P

Resulting:
NMAKE : fatal error U1073: don't know how to make 'config.h.win32'
Stop.

Whole output:

<<

release\Win32\harfbuzz-gobject:
flags:
dependents:
commands:   @-mkdir $@

release\Win32\harfbuzz-gobject.lib:
flags:
dependents: release\Win32\harfbuzz-gobject-vs11.dll
commands:

help:
flags:
dependents:
commands:   @echo.
@echo =
@echo Building HarfBuzz Using NMake
@echo =
@echo nmake /f Makefile.vc CFG=[release^|debug]