On Mon, Dec 03, 2007 at 02:16:00PM +1100, Russell Shaw wrote:
Hi,
I was thinking of making a multilingual text editor.
I don't get how glyphs are done outside of english.
I've read the Unicode Standard book.
When a paragraph of unicode characters is processed, the glyphs
are layed out according to the state contained in the unicode
character sequence.
Depending on this state, the same unicode characters can map to
multiple glyphs depending on context.
If multiple fonts exist for a language, then for all these font
files to work with an editor, then all these glyphs must be indexed
the same.
Where can i find the standard that specifies what glyphs are indexed
by what number? Or are these glyphs created on the fly by the unicode
paragraph layout processor?
The relevant standard is OpenType fonts, which contain the necessary
tables for mapping sequences of characters to glyphs. The glyph
indexing is specific to the particular font being used; there is no
standard across fonts, and in fact some fonts will use precomposed
glyphs while others will use constituent glyphs with positioning
information to achieve the same result.
OpenType was designed by Microsoft as an abstraction of TrueType and
Type1 fonts with the necessary features for proper Unicode rendering.
On Windows, Uniscribe/USP10.DLL is the code responsible for processing
these tables. Correctly multilingualized applications will use its
functions for text rendering (but all the standard Windows controls
will do that for apps).
The situation on Linux and *nix is a bit more diverse. Both GTK+ and
Qt widgets provide semi-correct OpenType handling, but with lots of
mistakes in handling scripts/languages their developers are not very
familiar with. Qt uses its own code for this, while GTK+ uses the
Pango library, an extremely slow “complex text layout” library which
does a lot more than is needed for most uses, and which duplicates
most of the font-specific logic in code, causing lots of headaches in
addition to bloat and bad performance (Firefox with Pango enabled is
many times slower than without; this is why many distributions still
have Pango support disabled by default, causing many languages not to
work...).
I’m very much hoping for a future direction of proper OpenType
rendering support without the need for Pango, but it requires someone
spending some time to understand the problem domain. Basically it’s
just a matter of applying substitution tables, and hard-coding lists
of which tables are needed for which scripts in Unicode and the order
in which they should be applied. (Originally they were intended to be
applied in the order they appear in the font files, but then MS went
and made their implementation hard-code the order, so other
implementations need to follow that in order to handle fonts properly
— or at least that’s my understanding.)
The OpenType specs themselves are available at Microsoft’s website,
but they’re very poorly documented. Reading them alone is insufficient
to make an implementation unless you already know basically what the
implementation must do, IMO — something like RFC 1459 in quality...
There’s a (semi-)new library called Harfbuzz which, as I understand
it, is purely the OpenType logic, without all the bloat of Pango. I’m
not sure what stage it’s at these days, but it might be a good place
to begin your search. Of course if your app depends on GTK+ or Qt you
can just use their widgets and forget about the whole issue, but I
hope someone will move things forward for OpenType font support
without the need for these toolkits.
Rich
--
Linux-UTF8: i18n of Linux on all levels
Archive: http://mail.nl.linux.org/linux-utf8/
