On Monday, 8 August 2016 12:49:15 UTC+2, Kazunobu Kuriyama wrote: > Sorry, but I still don't understand how you justify your patch that adds 0x20 > to each of alphanumeric characters and send them to Pango.
In my opinion, the optimisation in and of itself is a bit of a hack. The trouble is that it does the wrong thing in case of ligatures, since you get fewer glyphs than ASCII characters, which breaks the 1-to-1 mapping. The spaces are added to prevent ligatures (since most fonts with ligatures do not define a ligature between a space and another character). In some sense, this is still a hack, and what you would really want to do (if you want to keep the optimisation hack) is to shape one character one at a time, so are guaranteed to get one glyph per ascii character... > Actually, what is your point to make every text data length double? That is > 100% inefficient and totally ruins the existing optimization hack. Just as > what you wrote in your patch yourself, that's a ugly hack, indeed. It is, but bypassing the shaping to gain a bit of speed is ugly to start with, and adds quite a bit of code, data structures and complexity to vim's rendering routines. If I just disable that optimisation, vim doesn't have any issue with monospaced fonts, ligatures or otherwise. It's an optimisation, and one that isn't quite right. My point is that my patch contains more of the correct solution than does the code that's currently there. I don't claim it's nice, or beautiful, or anything. The double-length array is only a cost in memory, some small multiple of 128 bytes (likely 4 times that), which I doubt will be a consideration if you're running gvim, and an integer multiply (by 2, so likely a left shift) in the code that renders glyphs in the code path that is used by the optimisation... > If I wrote a patch for ligature support, I would remove the optimization hack > and add some code to cursor movement over a ligature, selection mechanisms > for visual mode and text objects when either of the region boundaries has a > ligatures on it, line wrapping handling at a ligature (to name but a few) in > order to decently accommodate ligatures in vim. > > What consideration is paid to those in your patch? I don't see any. The trick is to activate the optimisation not for all ASCII characters (chars < 128), but only for [A-Z], [a-z], [0-9] and space. If a string to be displayed contains other characters, it'll pass through the full blown shaping and has its glyphs drawn, including ligatures. > My question in the previous mail was whether or not PragmataPro was a > monospace font family as binary data. Not that what you felt about it nor > what the world saw about it. That was a purely technical question. Was it > too hard for you to check your PragmataPro file(s) with an appropriate > utility? I tried to answer your question as best as I could, and I'm sorry if I didn't quite hit the mark. In fact, with your new question, I don't even know what "monospace font family as binary data" means, maybe you could clarify. Unless you can read TrueType/OpenType fonts in hexdumps, and glean meaning from it, this will tell you very little. It's not a bitmap font, if that's what you mean. > I'm afraid your reply to that, which sounds somehow insulting old-school > people is completely out of place. I certainly never intended to insult anyone, and apologise if I have given offence. In fact, I've been programming for more than 25 years now, and am considered old-school by many of my colleagues. Changing one's preference in text editor user interface is certainly nothing that is carved in stone for all eternity, and people should have a freedom of choice. > I'm sorry that you don't care about technical aspects of the issue. I really > feel sick when I hear someone talks about old vs new to justify his view. I tried to present the technical issues at stake to the best of my (limited) ability and knowledge. I'm sorry if I didn't quite hit the mark. But to assert that I don't care about the technical aspects of the issue is a gross oversimplification. In fact, I'd welcome any constructive input I can get on this matter. Speaking of which: have you tried the patch with your favourite font? Does it work for you? (The patch may very well need improving, and I'm happy to do just that. But if you haven't given the patch a try, you might want to do that before judging and its author in such a harsh tone.) Concerning your request to send a patch that just fixes the assumption about "one ascii character == one glyph": One could either get rid of that speed optimisation code altogether (in which case all ligatures will work), or rewrite most of that optimisation, doing one shaping call into pango for each of (or some of) the characters with codes < 128. The latter versoin would amount to some overhead for all those calls, and reassembling the returned data structures in a suitable way... Instead, I preferred to go with a more minimal patch, but I'm more than willing to listen to whatever consensus emerges. Best regards, Manuel > 2016-08-08 18:15 GMT+09:00 manuelschiller.pimail via vim_dev > <[email protected]>: > On Monday, 8 August 2016 08:31:13 UTC+2, Kazunobu Kuriyama wrote: > > > > > 2016-08-07 20:27 GMT+09:00 manuelschiller.pimail via vim_dev > > <[email protected]>: > > > > > > > > > On Monday, 19 October 2015 08:41:36 UTC+2, Hörmetjan Yiltiz wrote: > > > > > > > On Sunday, 20 April 2014 11:55:22 UTC-4, François Gannaz wrote: > > > > > > > > Hello > > > > > > > > > > > > > > > > In a few words, here is a patch that makes gvim work better with > > > > ligatures > > > > > > > > in fonts, which can be useful even for programmers. Details follow. > > > > > > > > > > > > > > > > I tried to use a Hasklig[^1], a font with ligatures intended for the > > > > > > > > Haskell language. It serves the same objective as the Haskell Conceal > > > > > > > > script[^2], but with the added benefit that even a mouse copy-paste > > > > works > > > > > > > > as intended. > > > > > > > > [^1]: https://github.com/i-tu/hasklig > > > > > > > > [^2]: http://www.vim.org/scripts/script.php?script_id=3200 > > > > > > > > > > > > > > > > Unfortunately, gvim doesn't support ligatures on ASCII characters. The > > > > > > > > following assertion fails at run-time: > > > > > > > > > > > > > > > > ascii_glyph_table_init: assertion 'gui.ascii_glyphs->num_glyphs == > > > > > > > > sizeof(ascii_chars)' failed > > > > > > > > > > > > > > > > and many characters are displayed with the wrong glyphs. > > > > > > > > The attached patch limits the function ascii_glyph_table_init() to > > > > > > > > spaces and alphanumeric chars. It solves the problem here. > > > > > > > > > > > > > > > > Yet I wonder if the current hack with ASCII characters is really useful. > > > > > > > > Is there any performance test to check if a simpler behaviour wouldn't > > > > be > > > > > > > > suitable, at least for modern desktop installations? > > > > > > > > As the code comment mentions spaces, maybe it should be restricted to > > > > > > > > blank lines? > > > > > > > > > > > > > > > > Regards > > > > > > > > -- > > > > > > > > François > > > > > > > > > > > > > > This would be so great to see implemented. > > > > > > > > > > > > It certainly would (and apologies for reopening such an old discussion). > > > > > > > > > > > > On my system, the patch François sent doesn't work for the ligatures in > > PragmataPro (but flawlessly for other fonts like Hasklig), > > > > > > > > > According to http://www.fsd.it/shop/fonts/pragmatapro/ , PragmataPro has > > "One typeface, 3 families for the right need": > > > > > > > > > > > > - PragmataPro is a spaced modularly font family with working ligatures. > > > - PragmataPro Mono is a very monospaced without ligatures. > > > - Essential PragmataPro is the cheap version to coding for English only. > > > > > > > > > As you know, Vim is implemented by design only to support monospace font > > families. > > > > > > > > > So it looks to me that, among those three font families, the only possible > > and suitable choice for Vim to use with is PragmataPro Mono alone. > > > > In PragmataPro, all ASCII glyphs etc. are monospaced. Just as they are in > PragmatePro Mono or Courier, or whatever your favourite monospaced font is. > The difference between the two is what happens during shaping into glyphs > inside pango (or another library): For fonts like Hasklig or PragmataPro, > some combinations of ASCII characters get assigned a new glyph (the ligature) > which looks better (e.g. "<=" might be replaced with a new glyph that more > closely mimics the mathematical notation), but fits into the normal > monospaced character grid. These ligature characters are two and sometimes > three times the width of a single ASCII glyph, depending on how many > characters they replace (if it's a monospace font - in a proportional font > like Times, the ligature for something like "fl" can be any width the font > designer likes, and I'm not proposing to support this latter case). > > > > That kind of character/glyph juggling is okay because it fits with what vim > is expecting to draw anyway (so vim's drawing code can largely remain unaware > of the font shaping going on) - it's expecting to draw something that's e.g. > 2 characters wide, and it draws (a single) glyph that wide. > > > > (In PragmataPro Mono, these ligature tables are absent, so shaping just > returns the normal single-width glyphs.) > > > > The point of this patch is that it ensures that vim gets the right single > width glyphs for all monospaced fonts, even in the code that bypasses the > glyph shaping process and does its own thing. > > > > I've attached screenshots how unpatched vim (as shipped with the latest > stable debian) and patched vim look with these fonts. In all cases, I do a > ":version" before I take the screenshot. As you can see, it's totally garbled > in case of the unpatched versions of the plots, despite the fact that the two > fonts are monospaced (and are counted as such by all major terminal programs). > > > > This is a corner case where the optimisation to speed up rendering of ASCII > glyphs in vim (by bypassing the shaping process) gets it wrong (namely the > corner case where the font designer specified ligatures for a font). What > happens in these cases is that pango returns a list with fewer glyphs than > characters (because some character combinations return only a single glyph), > and vim happily reads past the end of the glyph list and caches the result to > render ascii glyphs. The effect is unreadable gibberish, as you can see. I > think it should just be fixed, because it's ugly and needlessly astonishes > unsuspecting users (neglecting for a moment the fact that there may also be > security implications of such accesses past the end of a data structure)... > > > > > If that is the case, however, that font family is described as "without > > ligatures," which makes your patch no sense. > > > > (See above.) > > > > > Accordingly, it appears my assumption was wrong; it might be better for me > > to conclude that what you meant by PragmataPro was literally PragmataPro, > > not its monospace version, or PragmataPro Mono. > > > > (See above again.) > > > > > Then, another question arises: What does "a spaced modularly font" mean? > > Does it imply that the font family in question is technically considered > > something equivalent to a monospaced font and hence deserve Vim's support? > > > > I'll try and render it yet another way: If you ask pango/... to draw a single > character in a monospaced font, it's always the same width. But, in some > monospaced fonts, for some two (and sometimes three) letter combinations, the > font rendering library substitutes a glyph that looks better, but occupies > exactly the width of two (three) individual characters. While you still fit > totally into the monospaced character grid, you draw fewer glyphs than input > characters now... > > > > I think this implies that such modern programmer's fonts like Hasklig or > PragmataPro are effectively a monospaced fonts, and people will increasingly > want to use them. (In fact, you're talking to a former die-hard text mode > console adept, who after 15+ years started to see the point of GUIs...) These > are questions that people will keep asking (probably increasingly often). In > my opinion, vim would do well to include support for this kind of effectively > monospaced fonts, so the answer to those questions can become "update to the > latest version", and you guys can get back to some serious hacking again... ;) > > > > > I'd appreciate it if you would give me some clarification or correct my > > possible misunderstanding. > > > > I hope this somehow clarifies the issue. I'm not proposing to implement > proportional font support or anything like that. But the assumption in the > current code that one ascii character renders to a single glyph is no longer > true in today's world, and I think that needs to be fixed somehow. The fact > that it also happens to make Hasklig or PragmataPro render correctly is, for > me, a welcome side effect (and also happens to be the reason why I got > started with this). But the longer I think about this patch, the more I tend > to think of the current (unpatched) drawing code as a liability for vim > because it reads past the end of a data structure without any checks. > > > Why don't you send us a patch on that? > We're lucky that only the graphical output is garbled. > > > > I hope this clarifies things a bit. > > > > Best regards, > > > > Manuel > > > > > Best regards, > > > Kazunobu Kuriyama > > > > > > apparently because the ASCII-to-glyph mapping won't quite work - I suspect > > that's due to PragmataPro having ligatures in that range, too. I solved > > this by inserting a space between characters that we hand to pango for > > shaping. Sane fonts will not have ligatures between a space and a printable > > character, this way we still get one glyph per ASCII character. I've > > attached the modified patch. This will work with normal fonts and ones with > > programming ligatures like Hasklig and PragmataPro; I've tested it with the > > gtk2 and gtk3 front ends. > > > > > > > > > > > > Please let me know if it would be possible to include this, and if not, at > > least the patch is public now where people can find it if they want it. > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Manuel > > > > > > > > > > > > -- > > > > > > -- > > > > > > You received this message from the "vim_dev" maillist. > > > > > > Do not top-post! Type your reply below the text you are replying to. > > > > > > For more information, visit http://www.vim.org/maillist.php > > > > > > > > > > > > --- > > > > > > You received this message because you are subscribed to the Google Groups > > "vim_dev" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > -- > > You received this message from the "vim_dev" maillist. > > Do not top-post! Type your reply below the text you are replying to. > > For more information, visit http://www.vim.org/maillist.php > > > > --- > > You received this message because you are subscribed to the Google Groups > "vim_dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
