On 9/22/2023 3:51 PM, Hamid,Idris wrote:

------ Original Message ------
 From "Hans Hagen" <j.ha...@xs4all.nl<mailto:j.ha...@xs4all.nl>>
To "ntg-context@ntg.nl<mailto:ntg-context@ntg.nl>" 
Date 9/22/2023 7:15:25 AM
Subject [NTG-context] Re: Toggling the symbol for the zero-width joiner and 
related Unicode control characters

** Caution: EXTERNAL Sender **

On 9/22/2023 2:39 PM, Hamid,Idris wrote:

b. we want all Unicode control symbols to be suppressed in final pdf output 
(for, e.g., printing).

they basically are unless some font features keeps them around which is
out of our control

irr it was you who wanted them to be wiped decades ago as some fonts
visualized them by default

Yes, that's exactly the point: Somewhere along the course of history, it became 
standard for Arabic-script fonts (and other cursive-script fonts as well) to 
include symbols for the control characters.

In typo-rep there is also

%D \starttyping
%D \definefontfeature[default][default][mode=node,formatters=strip]
%D \stoptyping

You included some notes about Khaled, so I guess he faced the same issue. His 
Amiri font displays the symbols by default, as do other Arabic fonts.

(It seems he never considered making it an opentype feature in the font itself, 
but since his focus is/was XeTeX/HB (HB is rather rigid and dictatorial) I 
guess that's not surprising.)

I admit that I don't follow what happens with xetex (they changed the rendere at some point indeed) not HB (I only notice that it gets updates frequently in the tex live repository which makes me wonder how one retains compatility unless one freezes). I actually kept the lib binding code that can use it around for your font testing (we wanted to see what uniscribe does), not sure if it still works.

Anyway, we're entering the bug cq. side effect becomes feature area here; just like yesterdays perfect bidi algorithm is todays less pefect one replaced by ...

But therein lies the problem: ConTeXt shows the rendering by default, and we 
need to turn it off. Since most non-Latin typography targets Uniscribe 
applications which allows for toggling, the font developers (commercial or 
free) don't have to concern themselves with this issue.

if context shows it then it is not a feature but hard coded shapes which is weird; how does one know what to 'remove' or not? And in what stage? If they are zero width it is simple to ignore them in the backend, if they have dimensions (w/h/d) then they contributed and wiping is tricky

Since Word rules the world, most font designers target it. Since Word provides 
for toggling the symbols -- needed for editing purposes -- there was no need 
for Arabic-script font designers to worry about the symbols showing up where 
they are not wanted.

(I suppose that InDesign behaves the same way.)

I don't know ... irr these dtp programs are more like "if you want this feature applied select a range of characters and apply it"

That's what was meant when I spoke of the continued effects of the WYSIWYG 
curse: It saved font designers from having to think much about this issue.

In some way it's also flaws in the open type approach. Basically that happens when application stuff becomes a standard and one forgets that it was (is) application driven. (And you haven't seen variable fonts and color fonts yet ... no pretty standards either.)

Not really -) This brings us to the point of consistency: For Arabic-script fonts, hard 
symbolic rendering of the Unicode control characters is the rule, not the exception. So 
not "an inconsistent mess" -- at least not as far as Arabic-script typography 
is concerned.

Funny rules ... but I'm not going top enable 'wipe' by default: after all, one gets what one deserves, nto what one likes (which can differ per day). But you can enable the wiping. We can of course ignore in the backend when zero width but then how to explain that they contributed to the ht/dp (unless we wipe these dimensions) ... all slow-downers

so you want to see soem zwj sumbol in a rendered text?

Only in verbatim/\type'd text where it is appropriate, even necessary. Thanks 
to Word/WYSIWYG, the rule is de facto, but it is not de jure -)

Ideally, Scintilla (Scite, Notepad++, etc.) should do the same, or provide a 
toggle, as MS Notepad does.

(Tangent: In terms of Unicode functionality, MS Notepad is still unrivalled, 
even in 2023!)

We agree that for final printed output it is not appropriate (except perhaps in 
a paper that discusses Unicode, fonts, etc., in which case it can be rendered 
using the figures or symbols mechanism -- or toggled as needed.)
so what is it now:

- for verbatim you can use almfixed and they show up (when they have a glyph) - for other fonts if they have them they show up (unless gone in the process ot rendering)
- but you can wipe them optionally

not sure what more we need


                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl

If your question is of interest to others as well, please add an entry to the 

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net

Reply via email to