Hi Egmont, hi all,
This is a interesting discussion here. If only because I would have
thought that there is only minimal interest by the actual target
audience in supporting these scripts in a terminal, given the severe
limitations of that environment. The most important limitation seems to
me that a monospaced font must be used, which does not suite most
scripts that do shaping. On the script-level I am familiar with Arabic,
Syraic and Mandaic (I don't actually speak any of these languages, so if
you want a real expert, I am not that person). Monospaced Arabic
struggles and is not very elegant. I have not seen solutions for
monospaced Syriac or Mandaic but I have trouble to even to imagine them.
OTOH, that inelegance maybe can be an excuse (or a guide if you prefer)
to make the implementation simpler in other respects, because
expectations should be lower than for a graphical application.
Anyway, as a concrete addition to the discussion, I have a simple Arabic
shaping solution for Emacs on the terminal, especially on the Linux
console, and this discussion finally made me make it public on Gitlab,
see https://gitlab.com/cc_benny/termshape. The Gitlab CD is activated,
so (mostly) ready-make Emacs packages can be downloaded as build
artifacts. If anybody wants to discuss this implementation, we should
probably move that discussion somewhere else, like to the Emacs mailing
list (https://lists.gnu.org/mailman/listinfo/emacs-devel).
Some specific technical points from thinking about the problem on my
side:
Presentation forms: Termshape uses the Arabic presentation forms
available and so it is somewhat limited as mentioned by Eli. Given that
we need to keep the implementation simple anyway, I am not sure that
significantly more is really needed, at least given what Emacs provides
already. Additional character forms could be added, where the Unicode
repertoire is not sufficient. This could use PUA characters or other
means like terminal control sequences. In both cases a common
understanding would be needed between the terminal (or the font used by
it) and the application, outside of Unicode.
Ligatures: With most shaping one character is transformed into a
character form that still only occupies one cell. A ligature like
lam-alif OTOH only occupies one cell for two characters, so for
justification etc. the application will have to know that the two
characters together have a width of 1 on the screen. This is easier if
the applicaton does the selection of ligatures. If you want to do this
in the terminal, the application would probably need to have some way to
measure the display width of a string, so that it can handle the
situation. Be prepared though for the application to make quite a lot
of these requests. For my own main use case for Emacs on a terminal,
display over SSH, that could become a problem.
Diacritics: The application can know what is a non-spacing character and
what is not. So it can know that diacritics do not occupy their own
cell and it should be able to ignore whether the terminal supports a
specific diacritic or not. If the terminal does not support a diacritic
the terminal can either just leave it out or the terminal can mess up
the display more of less irreparably. In the first case, the worst is
that the user does not see the character, in the second case the
application cannot do anything about it with reasonable effort IMO.
A real problem is a combination of diacritics and ligatures. Any
diacritic applies to only one character in the ligature, and between the
application and the terminal it is currently not possible to determine
which one. This is one area where an implementation in the terminal
would clearly have the advantage. But a terminal control sequence could
also help. IMO we are talking about a luxury problem here, though. Do
we want to set as our first goal showing complete quranic verses in all
their glory, or are we satisfied with everyday Arabic like say the
website of a modern Arabic newspaper?
Thanks for your effort and for starting this discussion,
benny
