Update of bug #59402 (group groff):
Category: Font - others/general => Preprocessor preconv
Severity: 1 - Wish => 3 - Normal
Item Group: Feature change => Incorrect behaviour
_______________________________________________________
Follow-up Comment #11:
Hi Dave,
The problem is the structure of groff's own font files. They are a table where
the first column is treated as a key to lookup information about a glyph, the
key can be an ascii character (a), a groff char name ('e), or (since
development of preconv) the unicode normalised NFD of the input character.
In choosing a key for a table it is important to use values which are always
unique, otherwise you are likely to run into problems.
Unicode gives every possible input character a unique code (usually expressed
as U+XXXX(X(X)). So what is the purpose of normalisation? It is to ensure that
input characters which have the same meaning can be matched, The normalised
form of 'fi' is 'fi' so that if you are searching for 'fi' (or 'fi') you
will find both. The same applies to sorting, you would want both to sort
together even though they are completely different input characters U+0066 +
U+0069 and U+FB01.
Groff does no sorting or searching of its input so normalisation is not
required. Now lets consider the greek iota, it is a letter in its own right,
but if it is used as an adscript a special mini form is used. See
<https://en.wikipedia.org/wiki/Iota_subscript>. Both forms mean 'iota', the
mini version is mute, so troff converts input character U+0389 to 'u03B9' and
due to normalisation the mini version U+1FBE also becomes u03B9. So both
different input characters lookup the same glyph entry in the groff font.
Because there are 2 entries with the same key it is tossup which will be
used.
I have created a tar file with the Tinos font prepped for groff here:-
<http://chuzzlewit.co.uk/Tinos.tar.xz>
When you unpack it will create a Tinos subdirectory, if you cd into it there
is test.sh which runs two groff commands, one for postscript and one for pdf.
If you view both you should see a difference, grops is using the second entry
(the one for U+1FBE, the mini iota) which is wrong for the context in which it
appears, and gropdf is using the glyph for the full size iota U+03B9. It then
greps the groff font TINOR showing the two different entries for u03B9.
Afmtodit used to warn when it was creating duplicate keys in a groff font,
this was silenced by commit 389b3e7f.
This mainly affects greek and to a lesser extent (but more duplicates) CJK
fonts. This is an interesting explanation of polytonic and monotonic versions
of greek:-
<https://www.opoudjis.net/unicode/unicode_gkbkgd.html>
Although modern greek is now the most common, it is not impossible that users
may require access to both forms if quoting ancient text. As a character an
iota is an iota (so it makes sense to normalise), but since Unicode
differenciates different iotas it does not make sense for troff to lose that
differentiation. Normalisation says that characters with a tonos are equal to
the same character with an oxia, which is true as a character, but a font may
use subtly different shapes for those two diacritics.
Because of the duplicate entries there is the chance the wrong entry is
chosen. We have seen that grops picks the wrong glyph for iota U+03B9, but
equally troff is using wrong information.
derij@pip:~/.groff/fonts/devpdf $ echo "ι=iota" | test-groff -Tpdf -ms -F..
-fTinos -k -Z
x T pdf
x res 72000 1 1
x init
p1
x font 7 TinosR
f7
s10000
md
DFd
V84000
H72000
Cu03B9
h3330 <-- note the horizontal movement
t=iota
n12000 0
V756000
H540000
n12000 0
x trailer
V792000
x stop
This should be the width of the u03B9 glyph * current font size (10). The
duplicate entries are:-
u03B9 269,459,10 0 541 iota -- 03B9
u03B9 333,0,217 1 2385 uni1FBE
Troff has used the width of the 'mini' iota (U+1FBE) rather than the correct
width for U+03B9 which would be 'h2690'.
> To play quasi-devil's advocate: "we" are typesetters, who are in a decent
> position to know what will look best on the page. What we don't know is
> whether someone has indeed gone to great trouble of decomposing a character
> because they want want the output to be composed of two glyphs, or whether
> they're using NFD for reasons wholly unrelated to troff, not knowing (nor,
> ideally, needing to care) which form typesets the best. Such users may
> reasonably expect troff, a typesetting program, to give them the best
> possible typeset output from their input.
Are "we" as good as we could be (as typesetters!), troff usiing wrong
meta-data if there are duplicate entries (i.e. the glyph width etc.), fall
back glyphs which are substandard and issue no warning the glyph is missing
from the selected font (see below). Although I like perl, I generally don't
like DWIM (do what I mean) which you are sort of advocating above. The
typesetter may alter spacing in order to make the finished output look nice,
but not change the characters the author selected.
Groff is generally not being helpful by silently using fall back characters
for missing glyphs. Consider this:-
$ echo "\s'48'Tinos=\[u0104]\[A ho] Times=\f[TR]\[u0104]\[A ho]" |
test-groff -Tpdf -ms -F.. -f Tinos -ww | okular -
In the attached image, Tinos has a glyph for U+0104 but our Times-Roman does
not, so troff uses a fallback glyph concoction which puts the ogonek in the
wrong place and gets the width wrong so the following glyph partially
overwrites. It does this without warning the user it is substituting the glyph
which was requested with a best guess at what will do. I don't think this is
good typesetting, if we have to have fallback glyphs a warning the precise
glyph requested is not available in the font used would be helpful.
(file #59194)
_______________________________________________________
Additional Item Attachment:
Name: u0104.png Size: 11KiB
<https://file.savannah.gnu.org/file/u0104.png?file_id=59194>
AGPL NOTICE
These attachments are served by Savane. You can download the corresponding
source code of Savane at
https://savannah.gnu.org/source/savane-99b9bb5c89ddd7e7187ecbf606b99ab99423d38f.tar.gz
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?59402>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
signature.asc
Description: PGP signature
