Re: [XeTeX] Diacritics in color
On Sat, Dec 03, 2011 at 08:45:24AM +0200, Tobias Schoel wrote: On 02.12.2011 21:48, Ross Moore wrote: Hi Tobias, On 03/12/2011, at 6:06, Tobias Schoelliesdieda...@googlemail.com wrote: As a teacher I can think of some more Applications. Of course, these are pedagogical: Teaching scripts to beginners (learning to write a primary school, learning to write in a different script when learning another language (or even in the same language: Mongol?): You might want to color single parts of a glyph in order to highlight them. So, for example in a handwritten (see http://de.wikipedia.org/wiki/Schulausgangsschrift or english equivalents I haven't found in the time) a the beginning or end-strokes might be colored. Yes, but are these examples really requiring parts of the same whole character coloured differently? Presuming that the font did allow access to individual glyphs, as if separate characters, then would not all meaningful aspects be equally well (if not better) encoded by an overlay? The result would be mostly the same. I don't know if some software might treat a partially colored glyph and two overlaid glyphs distuinguishably differently. Are overlaids encoded in a font and if yes, how are they accessible via XeTeX? Dunno about XeTeX, but in ConTeXt (when used with LuaTeX), you can have a sort of glyph color scheme that says glyph X takes color Y whenever it appears in the document (where X is glyph name whether it is encoded or not), if one wants to color, say the horizontal stroke of A, the font can be built where A is first decomposed in the Lambda-like part and the horizontal stroke, and then positioned together by some glyph positioning rule (or even using negative side bearing for such a simple case.) Someone can right something like that for LaTeX I think, but you would still need LuaTeX (unless someone extends XeTeX of course.) Regards, Khaled -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
OpenType has ligature caret info (in GDEF table) but is less flexible
than what Graphite offers (only horizontal position is provided) and
very few fonts, if any, have it.
Regards,
Khaled
On Thu, Dec 01, 2011 at 01:56:45PM -0500, Karljurgen Feuerherm wrote:
I seem to recall from the days when I was doing demos/mock-ups for the
cuneiform encoding proposal that SIL's graphite/WorldPad combo allowed
one to do things of this sort; it involved specifying sub-areas in
special font tables, which the software of course had to know about.
Not sure whether that's useful to this discussion, but thought I'd
mention it.
K
On Wed, Nov 30, 2011 at 6:31 PM, in message
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures (1/3
in
ffi etc), I'm not sure how this is done or if it is possible with PDF
at
all, but that is of limited value anyway specially for ligatures
that
can not be split vertically or into equal parts (ligatures carets
might
provide a clue in the later case, but they are rarely provided by
font
developers).
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Thanks, Khaled.
I realize the limitations etc.--just thought I'd note that these things
are in some measure possible, if one wishes to implement them (not that
one 'should', necessarily). In particular with regards to some recent
posts, I seem to remember that glyph sub-definition was not limited to
horizontal slices, either: one could define sub-bounding boxes of
whatever type. (And with WorldPad, the cursor could be placed within
glyphs, too, I think...)
K
On Thu, Dec 1, 2011 at 7:34 PM, in message
20111202003413.GE7306@khaled-laptop, Khaled Hosny
khaledho...@eglug.org
wrote:
OpenType has ligature caret info (in GDEF table) but is less
flexible
than what Graphite offers (only horizontal position is provided) and
very few fonts, if any, have it.
Regards,
Khaled
On Thu, Dec 01, 2011 at 01:56:45PM -0500, Karljurgen Feuerherm
wrote:
I seem to recall from the days when I was doing demos/mock-ups for
the
cuneiform encoding proposal that SIL's graphite/WorldPad combo
allowed
one to do things of this sort; it involved specifying sub-areas in
special font tables, which the software of course had to know
about.
Not sure whether that's useful to this discussion, but thought I'd
mention it.
K
On Wed, Nov 30, 2011 at 6:31 PM, in message
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures
(1/3
in
ffi etc), I'm not sure how this is done or if it is possible with
PDF
at
all, but that is of limited value anyway specially for ligatures
that
can not be split vertically or into equal parts (ligatures carets
might
provide a clue in the later case, but they are rarely provided by
font
developers).
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Tobias, On 03/12/2011, at 6:06, Tobias Schoel liesdieda...@googlemail.com wrote: As a teacher I can think of some more Applications. Of course, these are pedagogical: Teaching scripts to beginners (learning to write a primary school, learning to write in a different script when learning another language (or even in the same language: Mongol?): You might want to color single parts of a glyph in order to highlight them. So, for example in a handwritten (see http://de.wikipedia.org/wiki/Schulausgangsschrift or english equivalents I haven't found in the time) a the beginning or end-strokes might be colored. Yes, but are these examples really requiring parts of the same whole character coloured differently? Presuming that the font did allow access to individual glyphs, as if separate characters, then would not all meaningful aspects be equally well (if not better) encoded by an overlay? That is, position a coloured version of the required glyph over the full character in monochrome. In the pedagogical setting, you are presumably talking about the single stroke as a sub-part of the whole character, so it deserves to be placed as an entity in itself. This is quite different to a colored diacritical mark modifying the meaning of a character. Of course the font creator has to create sub-glyphs or other fancy stufff, but XeTeX should allow (re)composition of the glyph with different colors. bye Toscho Hope this helps, Ross -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On 02.12.2011 21:48, Ross Moore wrote: Hi Tobias, On 03/12/2011, at 6:06, Tobias Schoelliesdieda...@googlemail.com wrote: As a teacher I can think of some more Applications. Of course, these are pedagogical: Teaching scripts to beginners (learning to write a primary school, learning to write in a different script when learning another language (or even in the same language: Mongol?): You might want to color single parts of a glyph in order to highlight them. So, for example in a handwritten (see http://de.wikipedia.org/wiki/Schulausgangsschrift or english equivalents I haven't found in the time) a the beginning or end-strokes might be colored. Yes, but are these examples really requiring parts of the same whole character coloured differently? Presuming that the font did allow access to individual glyphs, as if separate characters, then would not all meaningful aspects be equally well (if not better) encoded by an overlay? The result would be mostly the same. I don't know if some software might treat a partially colored glyph and two overlaid glyphs distuinguishably differently. Are overlaids encoded in a font and if yes, how are they accessible via XeTeX? That is, position a coloured version of the required glyph over the full character in monochrome. In the pedagogical setting, you are presumably talking about the single stroke as a sub-part of the whole character, so it deserves to be placed as an entity in itself. As a single stroke wouldn't be recognised as such, it should be shown in conjunction with the whole character. For example print the whole character in gray and the interesting stroke in brigt red. This is quite different to a colored diacritical mark modifying the meaning of a character. Do you mean a technical difference or a semantic difference? Of course, the semantics differ. That's also the case with mathematical diacritics vs. text diacritics. The technical aspects should serve these semantics. Of course the font creator has to create sub-glyphs or other fancy stufff, but XeTeX should allow (re)composition of the glyph with different colors. bye Toscho Hope this helps, Ross -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Matthew Skala writes: Demonstration attached. Of course, the correct behavior in this case is that of Firefox: if I type: 기lt;FONT COLOR=REDgt;ᆷlt;/FONTgt; I get the precomposed Korean glyph all in black. Why? Because in the font (I don't have Jieubsida Dodum), the substitution rules tell us to replace the individual components with a ligature. What happens here in Firefox is what is supposed to happen -- the substitution rules are still honored and the color is ignored. If you want the ability to color individual Korean syllables (maybe for some pedagogical reason), you would have to design a font that does not use precomposed ligatures. Again, we're talking about two separate issue here. The first is coloring components of a precomposed ligature. Clearly, that is impossible. As someone on this list already said, it's like having a T where the top is red and the stem is black. The other issue is coloring separate glyphs that are positioned according to rules in GPOS. I don't think anywhere in the Unicode or OpenType standards it says that this should not be possible. In fact, it cannot say that in those standards because the standards don't address color! Color is handled by the layout manager, not by Unicode or OpenType. So the point then is that the software doing the layout (TeX, a browser, OpenOffice, whatever) should be able to color the glyphs while still honoring the positioning rules. Obviously, this would not be possible for a script that relies heavily on precomposition (e.g., Devanagari). But I am willing to submit that this is a difficult task, and probably one of limited importance -- I can see it being useful only for people who for some reason need their Hebrew or Arabic vowels in color (perhaps as a pedagogical tool) or for enthusiasts of Byzantine or Znamenny chant. Aleks -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 05:09:09PM -0600, msk...@ansuz.sooke.bc.ca wrote:
On Wed, 30 Nov 2011, Khaled Hosny wrote:
processed by the layout engine, which would require keeping account of
character to glyph mapping (which is doable, all text editing GUI's have
to do it).
It will, of course, have to do something sensible (even if that just means
complain) should the desired location of a reinserted special end up in
the middle of a ligature.
Suppose someone types
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures (1/3 in
ffi etc), I'm not sure how this is done or if it is possible with PDF at
all, but that is of limited value anyway specially for ligatures that
can not be split vertically or into equal parts (ligatures carets might
provide a clue in the later case, but they are rarely provided by font
developers).
Regards,
Khaled
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Thu, 1 Dec 2011, Khaled Hosny wrote:
Suppose someone types
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures (1/3 in
ffi etc), I'm not sure how this is done or if it is possible with PDF at
I don't think XeTeX should attempt to do that.
--
Matthew Skala
msk...@ansuz.sooke.bc.ca People before principles.
http://ansuz.sooke.bc.ca/
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Thu, 1 Dec 2011, Khaled Hosny wrote:
Suppose someone types
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures (1/3 in
ffi etc),
And if you look at the rendered output, is there a single
ff ligature-digraph glyph, or two f glyphs kerned ?
Philip Taylor
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
It's not given that that's the only conceivable application. But even if it is, that's plenty useful in and of itself. K But I am willing to submit that this is a difficult task, and probably one of limited importance -- I can see it being useful only for people who for some reason need their Hebrew or Arabic vowels in color (perhaps as a pedagogical tool) or for enthusiasts of Byzantine or Znamenny chant. Karljürgen G. Feuerherm, PhD Undergraduate Advisor Department of Archaeology and Classical Studies Wilfrid Laurier University 75 University Avenue West Waterloo, Ontario N2L 3C5 Tel. (519) 884-1970 x3193 Fax (519) 883-0991 (ATTN Arch. Classics) -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
I seem to recall from the days when I was doing demos/mock-ups for the
cuneiform encoding proposal that SIL's graphite/WorldPad combo allowed
one to do things of this sort; it involved specifying sub-areas in
special font tables, which the software of course had to know about.
Not sure whether that's useful to this discussion, but thought I'd
mention it.
K
On Wed, Nov 30, 2011 at 6:31 PM, in message
f\textcolor{red}{f}
In this case FireFox colourises half of resulting ff ligatures (1/3
in
ffi etc), I'm not sure how this is done or if it is possible with PDF
at
all, but that is of limited value anyway specially for ligatures
that
can not be split vertically or into equal parts (ligatures carets
might
provide a clue in the later case, but they are rarely provided by
font
developers).
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Thu, Dec 01, 2011 at 09:35:29AM -0500, Aleksandr Andreev wrote: coloring components of a precomposed ligature. Clearly, that is impossible. Not entirely: http://robert.ocallahan.org/2006/10/partial-ligatures_24.html (OK, it is a hack, but far from being impossible) -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Khaled Hosny writes: Not entirely: http://robert.ocallahan.org/2006/10/partial-ligatures_24.html The page reads: Basically it's a hack --- a ligature with N characters is divided into N vertical strips of equal width which I pretend are pseudocharacters While that could work for a sans (or even a serif) font for something like fi or ff or fl, it would be pretty hard to do for, say, a cursive-type font (or even for italics). And it would most certainly not work for something like Korean syllables. Aleks -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 05:10:07AM +0200, Khaled Hosny wrote:
A\textcolor{red}{0308}\\ % (3)
It seems Charis SIL composes accented glyphs, try Gentium Basic instead
(GenBasR.ttf).
Thanks, the result is correct, the box contents after luacolor's
processing is:
\EU2/GentiumBasic(0)/m/n/100 A
\pdfliteral direct1 0 0 rg 1 0 0 RG
\EU2/GentiumBasic(0)/m/n/100 Ì
The part in the PDF with/without color:
% without color:
1 0 0 1 72 590.679 Tm [0024]TJ
1 0 0 1 126.191 606.246 Tm [0301]TJ
% with color:
1 0 0 1 72 491.053 Tm [0024]TJ
1 0 0 rg 1 0 0 RG
1 0 0 1 126.191 506.619 Tm [0301]TJ
In case of XeTeX:
.\EU1/GentiumBasic(0)/m/n/100 A
.\special{color push rgb 1 0 0}
.\EU1/GentiumBasic(0)/m/n/100 Ì210
.\special{color pop}
The PDF stuff:
% without color:
0 -99.63 Td[0024]TJ
54.19 15.57 Td[0301]TJ
% with color:
-54.19 -115.2 Td[0024]TJ
ET 1 0 0 RG 1 0 0 rg BT /F1 99.626 Tf
59.3 -278.84 Td[0301]TJ
% with color via \special{pdf:code ...}:
0 -378.46 Td[0024]TJ
ET 1 0 0 rg 1 0 0 RG BT /F1 99.626 Tf
59.3 -378.46 Td[0301]TJ
It seems that in XeTeX the color cannot be inserted without
breaking the PDF text sections (BT...ET). \special{pdf:literal direct ...}
(or short \special{pdf:code ...}) is not equivalent to
\pdfliteral direct{...} but rather to \pdfliteral page{...}
that ends a text section in the PDF output.
If someone want's this issue fixed, make a feature request
for XeTeX/xdvipdfmx to provide a real \pdfliteral direct/
direct mode for colors without breaking text sections.
Yours sincerely
Heiko Oberdiek
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Khaled Hosny writes: use/build an OpenType font with proper combining mark positioning and apply the colors to individual glyphs à la what FireFox/LuaTeX and may be many other does. What I get out of this comment and the post by Heiko Oberdiek is that this should be possible to accomplish in XeTeX. My next question is this: is this then an issue with XeTeX, dvipdfmx, ICU or Ghostscript? I am just trying to understand here the workflow whereby a TeX source document becomes PDF in XeTeX and which component breaks the proper positioning. Thanks, Aleks -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Heiko, and others
On 30/11/2011, at 8:56 PM, Heiko Oberdiek wrote:
The PDF stuff:
% without color:
0 -99.63 Td[0024]TJ
54.19 15.57 Td[0301]TJ
% with color:
-54.19 -115.2 Td[0024]TJ
ET 1 0 0 RG 1 0 0 rg BT /F1 99.626 Tf
59.3 -278.84 Td[0301]TJ
% with color via \special{pdf:code ...}:
0 -378.46 Td[0024]TJ
ET 1 0 0 rg 1 0 0 RG BT /F1 99.626 Tf
59.3 -378.46 Td[0301]TJ
It seems that in XeTeX the color cannot be inserted without
breaking the PDF text sections (BT...ET). \special{pdf:literal direct ...}
(or short \special{pdf:code ...}) is not equivalent to
\pdfliteral direct{...} but rather to \pdfliteral page{...}
that ends a text section in the PDF output.
If someone want's this issue fixed, make a feature request
for XeTeX/xdvipdfmx to provide a real \pdfliteral direct/
direct mode for colors without breaking text sections.
In the experimental version of pdfTeX with support for Tagged PDF,
we need a similar kind of variant to \pdfliteral .
It is called 'enclose'.
Here is how, in the coding of the corresponding pdftex.web
the various options are read, following an occurrence
of \pdfliteral :
@ @Implement \.{\\pdfliteral}@=
begin
check_pdfoutput(\pdfliteral, true);
new_whatsit(pdf_literal_node, write_node_size);
if scan_keyword(direct) then
pdf_literal_mode(tail) := direct_always
else if scan_keyword(page) then
pdf_literal_mode(tail) := direct_page
else if scan_keyword(enclose) then
pdf_literal_mode(tail) := enclose
else
pdf_literal_mode(tail) := set_origin;
scan_pdf_ext_toks;
pdf_literal_data(tail) := def_ref;
end
and here is the documentation on those options:
@# {data structure for \.{\\pdfliteral}; node size = 2}
@d pdf_literal_data(#) == link(#+1) {data}
@d pdf_literal_mode(#) == info(#+1) {mode of resetting the text matrix
while writing data to the page stream}
@# {modes of setting the current transformation matrix (CTM)}
@d set_origin == 0 {end text (ET) if needed, set CTM to current
point}
@d direct_page == 1 {end text (ET) if needed, but don't change the
CTM}
@d direct_always == 2 {don't end text, don't change the CTM}
@d scan_special== 3 {look into special text}
@d enclose == 4 {like |direct_always|, but end current string
and sync pos}
The 'enclose' option is used to position the 'BDC' and 'EMC'
as closely around the text snippets as can be achieved,
without breaking the BT ... ET.
This seems to be the same kind of requirement here.
It would be nice if any extension to literal specials in xdvipdfmx
used the same keywords for similar functionality.
Yours sincerely
Heiko Oberdiek
Hope this helps,
Ross
Ross Moore ross.mo...@mq.edu.au
Mathematics Department office: E7A-419
Macquarie University tel: +61 (0)2 9850 8955
Sydney, Australia 2109 fax: +61 (0)2 9850 8114
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Khaled, I am afraid you are completely misconceived about what the subject matter is here. 1) adding a diacritic mark(glyph) is composing a a glyph, You are able to output it on its own. 2) There is a difference between the glyph ä and adding the diactirc mark of umlaut to an a. a) in the first case you can use two different colors because it is one glyph. b) in the second you can use two different colors because you two glyph in order two compose the ä glyph. So we are talking about composed glyphs whether you realize it or not. regards Keith. Am 30.11.2011 um 13:56 schrieb Khaled Hosny: On Wed, Nov 30, 2011 at 11:10:11AM +0100, Keith J. Schultz wrote: Hi All, I jump back in. I will cite anybody because what has been said is correct. But, 1) trying to compare a browser, XeTex engine and LuaTeX will not help as they have different methods of composing their output. That is how they compose and position their glyphs. As far as OpenType processing is concerned they all should give the same result, else there is a bug somewhere. 2) Most important a composed Unicode glyph is supposed to be just one color!! No one talked about composed glyphs (certainly not the OP), it was just a marginal and unrelated issue to the problem being discussed; coloring combining marks without breaking OpenType mark positioning. 3) Once you start using color a Unicode composed glyph you no longer are positioning a single composed glyph, but two or more glyphs. So? Again, coloring components of composed glyphs is not what is being discussed here. 3.a) color designed in TeX at.al is designed be applied to a box and not glyphs!! I've hard time understanding what this mean or how the difference, if any, is material. AFAIK, TeX knows nothing about color, it handled just like any other driver special which TeX makes no effort to interpret not to mention applying it. The question remains how to position the composed glyphs and where and how the color attribute is added to the output. There are therefore two solutions: 1) The Tex way: create a macro to compose the glyph and do the positioning and coloring. 2) The developer way: change the engine so that it firsts generates the composed glyph and then goes back and then applies color to the different glyphs. 3) The post-2000s way; use/build an OpenType font with proper combining mark positioning and apply the colors to individual glyphs à la what FireFox/LuaTeX and may be many other does. Regards, Khaled -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 02:17:07PM -0500, Aleksandr Andreev wrote:
Khaled Hosny writes:
use/build an OpenType font with proper combining
mark positioning and apply the colors to individual glyphs à la what
FireFox/LuaTeX and may be many other does.
What I get out of this comment and the post by Heiko Oberdiek is that
this should be possible to accomplish in XeTeX.
My next question is this: is this then an issue with XeTeX, dvipdfmx,
ICU or Ghostscript?
I am just trying to understand here the workflow whereby a TeX source
document becomes PDF in XeTeX and which component breaks the proper
positioning.
My understanding is that XeTeX reads input, converts characters into a
special native word node which is then processed by the layout engine
and converted into and array of positioned glyphs. Color is inserted
using \special{} command, when XeTeX encounters any non character
command is stops collecting characters at that positions, so the input
A\special{pdf:color stuff}0308 is split into two native word nodes
by the \special{} command, each processed by the layout engine
separately i.e. it sees an A on its own and an 0308 on its own,
which means no OpenType interaction would take place (mark positioning,
ligaturing, whatever).
A solution might involve e.g. not splitting characters at specials
instead, the special and its position in the text stream would be saved
somehow, and then inserted back in the glyph array somehow after being
processed by the layout engine, which would require keeping account of
character to glyph mapping (which is doable, all text editing GUI's have
to do it).
How easy/hard is that is beyond me.
Regards,
Khaled
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 09:47:31PM +0100, Keith J. Schultz wrote: Hi Khaled, I am afraid you are completely misconceived about what the subject matter is here. 1) adding a diacritic mark(glyph) is composing a a glyph, You are able to output it on its own. 2) There is a difference between the glyph ä and adding the diactirc mark of umlaut to an a. a) in the first case you can use two different colors because it is one glyph. b) in the second you can use two different colors because you two glyph in order two compose the ä glyph. So we are talking about composed glyphs whether you realize it or not. No we are not: 1) adding a mark to a glyphs is one thing, and composing them into a single glyph is another thing (in Latin fonts the later is a common result of the former, but that is mere implementation detail). The fully vowelled Arabic word مُحَمَّدٌ have a mark (or even two) in each character but no font will compose each basemark into a single glyph (not a font build by a sane person). The same is even true for versatile Latin fonts that allow proper positioning of arbitrary basemark combinations (e.g. many fonts from SIL). 2) the OP is developing his own font, which is neither covering Latin glyphs nor using precomposed glyphs. Regards, Khaled regards Keith. Am 30.11.2011 um 13:56 schrieb Khaled Hosny: On Wed, Nov 30, 2011 at 11:10:11AM +0100, Keith J. Schultz wrote: Hi All, I jump back in. I will cite anybody because what has been said is correct. But, 1) trying to compare a browser, XeTex engine and LuaTeX will not help as they have different methods of composing their output. That is how they compose and position their glyphs. As far as OpenType processing is concerned they all should give the same result, else there is a bug somewhere. 2) Most important a composed Unicode glyph is supposed to be just one color!! No one talked about composed glyphs (certainly not the OP), it was just a marginal and unrelated issue to the problem being discussed; coloring combining marks without breaking OpenType mark positioning. 3) Once you start using color a Unicode composed glyph you no longer are positioning a single composed glyph, but two or more glyphs. So? Again, coloring components of composed glyphs is not what is being discussed here. 3.a) color designed in TeX at.al is designed be applied to a box and not glyphs!! I've hard time understanding what this mean or how the difference, if any, is material. AFAIK, TeX knows nothing about color, it handled just like any other driver special which TeX makes no effort to interpret not to mention applying it. The question remains how to position the composed glyphs and where and how the color attribute is added to the output. There are therefore two solutions: 1) The Tex way: create a macro to compose the glyph and do the positioning and coloring. 2) The developer way: change the engine so that it firsts generates the composed glyph and then goes back and then applies color to the different glyphs. 3) The post-2000s way; use/build an OpenType font with proper combining mark positioning and apply the colors to individual glyphs à la what FireFox/LuaTeX and may be many other does. Regards, Khaled -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, 30 Nov 2011, Khaled Hosny wrote:
processed by the layout engine, which would require keeping account of
character to glyph mapping (which is doable, all text editing GUI's have
to do it).
It will, of course, have to do something sensible (even if that just means
complain) should the desired location of a reinserted special end up in
the middle of a ligature.
Suppose someone types
f\textcolor{red}{f}
and the font tries to replace the two fs with an ff ligature. In that
case there is no sensible way to colour just the second one red.
Changing the colour of a mark and not the base is different from changing
the colour of just part of a ligature, and the system can certainly
distinguish mark-to-base from ligatures; but especially if colour is
handled by a special that the TeX engine only understands as an opaque
blob of data, it'll be hard for the engine to know what is the right thing
to do with the above.
In the Korean fonts I'm currently working on, some syllables are converted
to single precomposed glyphs by ligature substitution, and others are
built up by overlaying zero-width glyphs, and the difference between the
two ways of drawing syllables should ideally not be visible to the user.
(This approach is driven by Unicode, which specifies roughly 11000 code
points for entire Korean syllables, but a more complicated way of
expressing the several hundred thousand imaginable syllables that do not
have code points of their own.) The zero-width glyphs also change their
shapes contextually. Suppose someone tries to change the colour of just
one of the sub-glyphs in what would otherwise be a precomposed syllable.
Well, if that breaks the glyph sequence for the purposes of the ligature
substitution, it's actually a good thing. It means my font will fall
back on the overlaid zero-width glyphs, and then just the one the user
picked will change colour, and the user will get what they wanted. The
layout of the overlaid combination won't be quite as good as if the font
had been allowed to use a precomposed glyph, but there isn't really any
better way it could work.
Trouble is, the overlaid zero-width glyph method also depends on
contextual substitutions (because there are actually six different layouts
for an overlaid syllable, used to approximate the more nuanced layout
decisions that go into the precomposed glyphs). If the colour special
interrupts the glyph sequence for the purpose of those contextual
substitutions, then the font will end up falling all the way back to its
default layout (or, even worse, different layout guesses for different
parts of the same syllable) and the result will be unacceptable. The
ideal for my fonts would be if colour specials interrupted the glyph
sequence in the liga feature - thus preventing ligature substitution from
occurring - but not in the ccmp, ljmo, or vjmo features. What would be
ideal in some other font might be different from that.
I don't know the right answer to how it should work, but I'm highlighting
this issue just because I hope anyone who tries to implement a solution
will think carefully about the consequences for multiple cases.
--
Matthew Skala
msk...@ansuz.sooke.bc.ca People before principles.
http://ansuz.sooke.bc.ca/
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, 30 Nov 2011, msk...@ansuz.sooke.bc.ca wrote: In the Korean fonts I'm currently working on, some syllables are converted to single precomposed glyphs by ligature substitution, and others are built up by overlaying zero-width glyphs, and the difference between the I was interested to test just what the effects of trying to colour part of a Korean syllable with my fonts actually were, so I did some experiments. Demonstration attached. -- Matthew Skala msk...@ansuz.sooke.bc.ca People before principles. http://ansuz.sooke.bc.ca/ colordemo.pdf Description: Adobe PDF document -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color (was Re: XETEX cannot access OpenType features in PUA?)
Ross Moore writes:
Would you be so kind as to post the PDF from this? And where does one obtain
the font MezenetsUnicode ?
Mezenets Unicode is a font I'm developing for Znamenny neumatic
notation and it is available here:
http://www.ponomar.net/files/mezen_uni.ttf
Attempting to encode Znamenny Notation in Unicode seems to be a
mind-boggling task because each character is able to take several
diacritical marks, each of which is governed by its own position rules
and each of which can have its own color.
The attached image shows what I am and am not able to do right now in
XeTeX and LuaTeX.
The code was:
\documentclass{article}
\usepackage{fontspec}
\usepackage{xcolor}
\usepackage{xunicode}
\usepackage{luacolor}
\newfontface\moo{MezenetsUnicode}
\begin{document}
\Huge
\moo
\\
\textcolor{red}{} \\
\textcolor{red}{} \\
\textcolor{red}{} \\
\textcolor{red}{} \\
\\
\end{document}
In XeTeX I am using the color package instead of luacolor.
Now, I realize that comparing TeX to Firefox is like apples to
oranges, but I'm just giving the Firefox example to show what is
supposed to happen.
Aleks
attachment: znamenny.png
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color (was Re: XETEX cannot access OpenType features in PUA?)
Aleksandr Andreev wrote: Mezenets Unicode is a font I'm developing for Znamenny neumatic notation Oohhh, this is exciting : [p]nuematic notation as in [p]neumes and as in [p]neumatic music ? Will this be a first for TeX, if you succeed ? Philip Taylor -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Hi Ross,
On Wed, Nov 30, 2011 at 06:23:54AM +1100, Ross Moore wrote:
Hi Heiko,
On 29/11/2011, at 9:29 AM, Heiko Oberdiek wrote:
The same also works in XeLaTeX:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{color}
\pagestyle{empty}
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00e4\\
a0308\\
a\textcolor{red}{0308}
\end{document}
stream
q 1 0 0 1 72 769.89 cm 0 G 0 g 0 G 0 g 0 G 0 g BT /F1 99.626 Tf 0 -63.86
Td[00a0]TJ 0 -99.63 Td[001c00ee]TJ 0 -99.62 Td[001c]TJ ET 1 0 0 RG 1
0 0 rg BT /F1 99.626 Tf 45.73 -263.11 Td[00ee]TJ ET 0 G 0 g 0 G 0 g 0 G
0 g Q
endstream
Yes, this is OK with lower-case letters.
But try with uppers:
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00e4\\
a0308\\
a\textcolor{red}{0308}\a
U\textcolor{red}{0308}\U
\end{document}
or
00c4\\
A0308\\
A\textcolor{red}{0308}
As you can see, this problem is not related to color,
both XeTeX and LuaTeX fail:
Both the vertical and horizontal position are now wrong.
The Unicode input neither seems to be normalized in XeTeX or LuaTeX
nor the accent seems to know its base letter. Manual fixing
of the accent position remains as (only?) option.
Yours sincerely
Heiko Oberdiek
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 10:58:23AM +1100, Ross Moore wrote:
On 30/11/2011, at 10:32 AM, Heiko Oberdiek wrote:
or
00c4\\
A0308\\
A\textcolor{red}{0308}
As you can see, this problem is not related to color,
both XeTeX and LuaTeX fail:
With this font (Latin Modern) yes. I noticed this too.
But try switching the font:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{color}
\pagestyle{empty}
\setmainfont{Charis SIL}
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00e4\\
a0308\\
a\textcolor{red}{0308}\a
\hbox{U0308} U\textcolor{red}{0308}\U
00c4\\
A0308\\
A\textcolor{red}{0308}
\end{document}
Now it *does* depend upon having the color commands.
\documentclass{minimal}
\usepackage{ifluatex}
\usepackage{fontspec}
\ifluatex
\usepackage{luacolor}
\pdfobjcompresslevel=0
\pdfcompresslevel=0
\else
\usepackage{color}
\fi
\pagestyle{empty}
\setmainfont{CharisSIL-R.ttf}
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00c4\\ % (1)
A0308\\ % (2)
A\textcolor{red}{0308}\\ % (3)
A\textcolor{red}{\hbox{0308}} % (4)
\end{document}
* LuaTeX: A + U+0308 gets combined to one glyph U+00C4, the
color attribute of the diaeresis vanishes and the result is
black (3). In the last line (4) \hbox prevents the recombination
and the diaeresis is red, but misplaced.
* XeTeX: The color special prevents the glyph recombination and
U+0308 is processed separately without knowing the base character
(3 and 4).
Thus the workaround would be to prevent the recombination and
to fix the placement manually.
Yours sincerely
Heiko Oberdiek
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
Heiko Oberdiek writes: * LuaTeX: A + U+0308 gets combined to one glyph U+00C4, the color attribute of the diaeresis vanishes and the result is black (3). On my machine, in LuaTeX (3) results in a correctly positioned *red* diaeresis over a black A. (4) results in a red diaeresis with incorrect positioning, both vertical and horizontal. I'm running Ubuntu 11.10 and TexLive 2011. XeTeX: The color special prevents the glyph recombination and U+0308 is processed separately without knowing the base character (3 and 4) Yes, I can confirm this also. Thus the workaround would be to prevent the recombination and to fix the placement manually. Would it be possible to read the positioning data out of GPOS in order to do this manual placement? It could then be handled by a macro. Aleks -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 02:09:47AM +0100, Heiko Oberdiek wrote:
On Wed, Nov 30, 2011 at 10:58:23AM +1100, Ross Moore wrote:
On 30/11/2011, at 10:32 AM, Heiko Oberdiek wrote:
or
00c4\\
A0308\\
A\textcolor{red}{0308}
As you can see, this problem is not related to color,
both XeTeX and LuaTeX fail:
With this font (Latin Modern) yes. I noticed this too.
But try switching the font:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{color}
\pagestyle{empty}
\setmainfont{Charis SIL}
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00e4\\
a0308\\
a\textcolor{red}{0308}\a
\hbox{U0308} U\textcolor{red}{0308}\U
00c4\\
A0308\\
A\textcolor{red}{0308}
\end{document}
Now it *does* depend upon having the color commands.
\documentclass{minimal}
\usepackage{ifluatex}
\usepackage{fontspec}
\ifluatex
\usepackage{luacolor}
\pdfobjcompresslevel=0
\pdfcompresslevel=0
\else
\usepackage{color}
\fi
\pagestyle{empty}
\setmainfont{CharisSIL-R.ttf}
\begin{document}
\fontsize{100pt}{100pt}\selectfont
\noindent
00c4\\ % (1)
A0308\\ % (2)
A\textcolor{red}{0308}\\ % (3)
A\textcolor{red}{\hbox{0308}} % (4)
\end{document}
* LuaTeX: A + U+0308 gets combined to one glyph U+00C4, the
color attribute of the diaeresis vanishes and the result is
black (3). In the last line (4) \hbox prevents the recombination
and the diaeresis is red, but misplaced.
It seems Charis SIL composes accented glyphs, try Gentium Basic instead
(GenBasR.ttf).
Regards,
Khaled
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color
On Wed, Nov 30, 2011 at 05:10:07AM +0200, Khaled Hosny wrote: It seems Charis SIL composes accented glyphs, After closer look, this is not entirely true; Charis SIL has AAT tables that, among other things, compose accented glyphs but the OpenType tables do mark positioning. LuaTeX shouldn't be executing AAT tables, so that is a bug (it is a side effect of using FontForge internallu which can read some AAT tables in present them in an OpenType-like format). Regards, Khaled -- Subscriptions, Archive, and List information, etc.: http://tug.org/mailman/listinfo/xetex
[XeTeX] Diacritics in color (was Re: XETEX cannot access OpenType features in PUA?)
Jonathan Kew writes:
Making this work in xetex would require a different approach to implementing
color.
I have been able to get it to work (the base glyph in black and the
diacritic in red) in LuaTeX using the luacolor package.
Here's a minimal example:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{xcolor}
\usepackage{luacolor}
\newfontface\moo{MezenetsUnicode}
\begin{document}
\moo
\textcolor{red}{}
\end{document}
I'm not much of an expert in the inner workings of TeX and I know
absolutely nothing about Lua (is that a derivative of LISP?) so I
can't comment on whether the luacolor package could be ported to
XeTeX.
Any insights?
Aleks
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color (was Re: XETEX cannot access OpenType features in PUA?)
Hi Aleks,
On 29/11/2011, at 6:18 AM, Aleksandr Andreev wrote:
Jonathan Kew writes:
Making this work in xetex would require a different approach to
implementing color.
I have been able to get it to work (the base glyph in black and the
diacritic in red) in LuaTeX using the luacolor package.
Here's a minimal example:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{xcolor}
\usepackage{luacolor}
\newfontface\moo{MezenetsUnicode}
\begin{document}
\moo
\textcolor{red}{}
\end{document}
Would you be so kind as to post the PDF from this?
And where does one obtain the font MezenetsUnicode ?
--- Google gives nothing with this name.
Furthermore my LuaTeX gives a Segmentation Fault, so I cannot
just try with a different font!
I'm not much of an expert in the inner workings of TeX and I know
absolutely nothing about Lua (is that a derivative of LISP?) so I
can't comment on whether the luacolor package could be ported to
XeTeX.
I'd doubt that this could work currently.
My guess is that you would need to do some post-processing
of the PDF code snippet returned from the OS positioning
the glyphs. Once positioned, you would need to wrap the colour
commands around the part which places the diacritic.
Any insights?
But XeTeX currently does not give you access to that PDF string,
and it is well past the place of macro-expansion in LaTeX, so
there wouldn't be a mechanism for such late adjustments.
It can be done with LuaTeX, since it does have the appropriate
mechanism for such post-processing.
Others more familiar with how LuaTeX works can confirm this
explanation -- or shoot it down, as appropriate.
Aleks
Hope this helps,
Ross
Ross Moore ross.mo...@mq.edu.au
Mathematics Department office: E7A-419
Macquarie University tel: +61 (0)2 9850 8955
Sydney, Australia 2109 fax: +61 (0)2 9850 8114
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
Re: [XeTeX] Diacritics in color (was Re: XETEX cannot access OpenType features in PUA?)
On Mon, Nov 28, 2011 at 02:18:56PM -0500, Aleksandr Andreev wrote:
Jonathan Kew writes:
Making this work in xetex would require a different approach to
implementing color.
I have been able to get it to work (the base glyph in black and the
diacritic in red) in LuaTeX using the luacolor package.
Here's a minimal example:
\documentclass{minimal}
\usepackage{fontspec}
\usepackage{xcolor}
\usepackage{luacolor}
\newfontface\moo{MezenetsUnicode}
\begin{document}
\moo
\textcolor{red}{}
\end{document}
I'm not much of an expert in the inner workings of TeX and I know
absolutely nothing about Lua (is that a derivative of LISP?) so I
can't comment on whether the luacolor package could be ported to
XeTeX.
No. Package luacolor uses LuaTeX's attributes and LuaTeX's ability
to process node lists.
Yours sincerely
Heiko Oberdiek
--
Subscriptions, Archive, and List information, etc.:
http://tug.org/mailman/listinfo/xetex
