On Monday, 8 August 2016 12:33:44 UTC+2, Tony Mechelynck wrote: > On Mon, Aug 8, 2016 at 11:15 AM, manuelschiller.pimail via vim_dev > <[email protected]> wrote: > > On Monday, 8 August 2016 08:31:13 UTC+2, Kazunobu Kuriyama wrote: > >> 2016-08-07 20:27 GMT+09:00 manuelschiller.pimail via vim_dev > >> <[email protected]>: > >> > >> > >> On Monday, 19 October 2015 08:41:36 UTC+2, Hörmetjan Yiltiz wrote: > >> > >> > On Sunday, 20 April 2014 11:55:22 UTC-4, François Gannaz wrote: > >> > >> > > Hello > >> > >> > > > >> > >> > > In a few words, here is a patch that makes gvim work better with > >> > > ligatures > >> > >> > > in fonts, which can be useful even for programmers. Details follow. > >> > >> > > > >> > >> > > I tried to use a Hasklig[^1], a font with ligatures intended for the > >> > >> > > Haskell language. It serves the same objective as the Haskell Conceal > >> > >> > > script[^2], but with the added benefit that even a mouse copy-paste > >> > > works > >> > >> > > as intended. > >> > >> > > [^1]: https://github.com/i-tu/hasklig > >> > >> > > [^2]: http://www.vim.org/scripts/script.php?script_id=3200 > >> > >> > > > >> > >> > > Unfortunately, gvim doesn't support ligatures on ASCII characters. The > >> > >> > > following assertion fails at run-time: > >> > >> > > > >> > >> > > ascii_glyph_table_init: assertion 'gui.ascii_glyphs->num_glyphs == > >> > >> > > sizeof(ascii_chars)' failed > >> > >> > > > >> > >> > > and many characters are displayed with the wrong glyphs. > >> > >> > > The attached patch limits the function ascii_glyph_table_init() to > >> > >> > > spaces and alphanumeric chars. It solves the problem here. > >> > >> > > > >> > >> > > Yet I wonder if the current hack with ASCII characters is really > >> > > useful. > >> > >> > > Is there any performance test to check if a simpler behaviour wouldn't > >> > > be > >> > >> > > suitable, at least for modern desktop installations? > >> > >> > > As the code comment mentions spaces, maybe it should be restricted to > >> > >> > > blank lines? > >> > >> > > > >> > >> > > Regards > >> > >> > > -- > >> > >> > > François > >> > >> > > >> > >> > This would be so great to see implemented. > >> > >> > >> > >> It certainly would (and apologies for reopening such an old discussion). > >> > >> > >> > >> On my system, the patch François sent doesn't work for the ligatures in > >> PragmataPro (but flawlessly for other fonts like Hasklig), > >> > >> > >> According to http://www.fsd.it/shop/fonts/pragmatapro/ , PragmataPro has > >> "One typeface, 3 families for the right need": > >> > >> > >> > >> - PragmataPro is a spaced modularly font family with working ligatures. > >> - PragmataPro Mono is a very monospaced without ligatures. > >> - Essential PragmataPro is the cheap version to coding for English only. > >> > >> > >> As you know, Vim is implemented by design only to support monospace font > >> families. > >> > >> > >> So it looks to me that, among those three font families, the only > >> possible and suitable choice for Vim to use with is PragmataPro Mono alone. > > > > In PragmataPro, all ASCII glyphs etc. are monospaced. Just as they are in > > PragmatePro Mono or Courier, or whatever your favourite monospaced font is. > > The difference between the two is what happens during shaping into glyphs > > inside pango (or another library): For fonts like Hasklig or PragmataPro, > > some combinations of ASCII characters get assigned a new glyph (the > > ligature) which looks better (e.g. "<=" might be replaced with a new glyph > > that more closely mimics the mathematical notation), but fits into the > > normal monospaced character grid. These ligature characters are two and > > sometimes three times the width of a single ASCII glyph, depending on how > > many characters they replace (if it's a monospace font - in a proportional > > font like Times, the ligature for something like "fl" can be any width the > > font designer likes, and I'm not proposing to support this latter case). > > > > That kind of character/glyph juggling is okay because it fits with what vim > > is expecting to draw anyway (so vim's drawing code can largely remain > > unaware of the font shaping going on) - it's expecting to draw something > > that's e.g. 2 characters wide, and it draws (a single) glyph that wide. > > > > (In PragmataPro Mono, these ligature tables are absent, so shaping just > > returns the normal single-width glyphs.) > > > > The point of this patch is that it ensures that vim gets the right single > > width glyphs for all monospaced fonts, even in the code that bypasses the > > glyph shaping process and does its own thing. > > > > I've attached screenshots how unpatched vim (as shipped with the latest > > stable debian) and patched vim look with these fonts. In all cases, I do a > > ":version" before I take the screenshot. As you can see, it's totally > > garbled in case of the unpatched versions of the plots, despite the fact > > that the two fonts are monospaced (and are counted as such by all major > > terminal programs). > > > > This is a corner case where the optimisation to speed up rendering of ASCII > > glyphs in vim (by bypassing the shaping process) gets it wrong (namely the > > corner case where the font designer specified ligatures for a font). What > > happens in these cases is that pango returns a list with fewer glyphs than > > characters (because some character combinations return only a single > > glyph), and vim happily reads past the end of the glyph list and caches the > > result to render ascii glyphs. The effect is unreadable gibberish, as you > > can see. I think it should just be fixed, because it's ugly and needlessly > > astonishes unsuspecting users (neglecting for a moment the fact that there > > may also be security implications of such accesses past the end of a data > > structure)... > > > >> If that is the case, however, that font family is described as "without > >> ligatures," which makes your patch no sense. > > > > (See above.) > > > >> Accordingly, it appears my assumption was wrong; it might be better for me > >> to conclude that what you meant by PragmataPro was literally PragmataPro, > >> not its monospace version, or PragmataPro Mono. > > > > (See above again.) > > > >> Then, another question arises: What does "a spaced modularly font" mean? > >> Does it imply that the font family in question is technically considered > >> something equivalent to a monospaced font and hence deserve Vim's support? > > > > I'll try and render it yet another way: If you ask pango/... to draw a > > single character in a monospaced font, it's always the same width. But, in > > some monospaced fonts, for some two (and sometimes three) letter > > combinations, the font rendering library substitutes a glyph that looks > > better, but occupies exactly the width of two (three) individual > > characters. While you still fit totally into the monospaced character grid, > > you draw fewer glyphs than input characters now... > > > > I think this implies that such modern programmer's fonts like Hasklig or > > PragmataPro are effectively a monospaced fonts, and people will > > increasingly want to use them. (In fact, you're talking to a former > > die-hard text mode console adept, who after 15+ years started to see the > > point of GUIs...) These are questions that people will keep asking > > (probably increasingly often). In my opinion, vim would do well to include > > support for this kind of effectively monospaced fonts, so the answer to > > those questions can become "update to the latest version", and you guys can > > get back to some serious hacking again... ;) > > > >> I'd appreciate it if you would give me some clarification or correct my > >> possible misunderstanding. > > > > I hope this somehow clarifies the issue. I'm not proposing to implement > > proportional font support or anything like that. But the assumption in the > > current code that one ascii character renders to a single glyph is no > > longer true in today's world, and I think that needs to be fixed somehow. > > The fact that it also happens to make Hasklig or PragmataPro render > > correctly is, for me, a welcome side effect (and also happens to be the > > reason why I got started with this). But the longer I think about this > > patch, the more I tend to think of the current (unpatched) drawing code as > > a liability for vim because it reads past the end of a data structure > > without any checks. We're lucky that only the graphical output is garbled. > > > > I hope this clarifies things a bit. > > > > Best regards, > > > > Manuel > > > >> Best regards, > >> Kazunobu Kuriyama > >> > >> apparently because the ASCII-to-glyph mapping won't quite work - I suspect > >> that's due to PragmataPro having ligatures in that range, too. I solved > >> this by inserting a space between characters that we hand to pango for > >> shaping. Sane fonts will not have ligatures between a space and a > >> printable character, this way we still get one glyph per ASCII character. > >> I've attached the modified patch. This will work with normal fonts and > >> ones with programming ligatures like Hasklig and PragmataPro; I've tested > >> it with the gtk2 and gtk3 front ends. > >> > >> > >> > >> Please let me know if it would be possible to include this, and if not, at > >> least the patch is public now where people can find it if they want it. > >> > >> > >> > >> Best regards, > >> > >> > >> > >> Manuel > >> > >> > > Hm, the "unpatched hasklig" case seems to be just off-by-one > everywhere, while the other "unpatched" one looks pathologic. > > In current versions of Vim, there is no necessary equivalence between > the number of bytes and the number of screen cells used by a given > character. CJK "wide" characters take up two screen cells each > regardless of the number of bytes, and everything else takes up one > cell per character regardless of the number of bytes. (For instance, > in UTF-8 which is the recommended 'encoding' these days, codepoints > U+0080 to U+07FF use two bytes each, and anything from U+0800 up uses > at least three. FWIW, the "usual" Kangxi characters start at U+4E00 > with the single-line 一 character [Kangxi radical 1 plus no strokes] > meaning "one", but there are "extension CJK" characters as low as > U+3400, and the "Compatibility Ideographs Supplement" block starts at > U+2F800 and then goes up I don't know how high.) The horizontal > tabulation U+0009 is of course a special case, in a category by > itself. > > Best regards, > Tony.
Hi Tony, your musings on UTF-8 sound reasonable, although I'm far from an expert. In fact for unicode characters with character code >= 128, even the current vim code base will always do the full-blown character shaping, and apply whatever magic it applies to display the text. And it'll hopefully get things correct even without my patch. To render strings with codes < 128, however, the current code will apply an optimisation for skipping the shaping step inside pango. The character code -> glyph mapping is no longer 1:1 for a font with ligatures in that range, and that is why the cache of that mapping appears to display gibberish. So it's less a question of the mapping of characters (whatever their encoding) to screen cells, but that a speed optimisation inside the gtk gui code can be fooled by some fonts... Best regards, Manuel -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
