2016-08-08 20:37 GMT+09:00 manuelschiller.pimail via vim_dev <
[email protected]>:
> On Monday, 8 August 2016 12:49:15 UTC+2, Kazunobu Kuriyama wrote:
> > Sorry, but I still don't understand how you justify your patch that adds
> 0x20 to each of alphanumeric characters and send them to Pango.
>
> In my opinion, the optimisation in and of itself is a bit of a hack. The
> trouble is that it does the wrong thing in case of ligatures, since you get
> fewer glyphs than ASCII characters, which breaks the 1-to-1 mapping. The
> spaces are added to prevent ligatures (since most fonts with ligatures do
> not define a ligature between a space and another character).
>
> In some sense, this is still a hack, and what you would really want to do
> (if you want to keep the optimisation hack) is to shape one character one
> at a time, so are guaranteed to get one glyph per ascii character...
>
> > Actually, what is your point to make every text data length double?
> That is 100% inefficient and totally ruins the existing optimization hack.
> Just as what you wrote in your patch yourself, that's a ugly hack, indeed.
>
> It is, but bypassing the shaping to gain a bit of speed is ugly to start
> with, and adds quite a bit of code, data structures and complexity to vim's
> rendering routines. If I just disable that optimisation, vim doesn't have
> any issue with monospaced fonts, ligatures or otherwise.
>
> It's an optimisation, and one that isn't quite right. My point is that my
> patch contains more of the correct solution than does the code that's
> currently there. I don't claim it's nice, or beautiful, or anything. The
> double-length array is only a cost in memory, some small multiple of 128
> bytes (likely 4 times that), which I doubt will be a consideration if
> you're running gvim, and an integer multiply (by 2, so likely a left shift)
> in the code that renders glyphs in the code path that is used by the
> optimisation...
>
> > If I wrote a patch for ligature support, I would remove the optimization
> hack and add some code to cursor movement over a ligature, selection
> mechanisms for visual mode and text objects when either of the region
> boundaries has a ligatures on it, line wrapping handling at a ligature (to
> name but a few) in order to decently accommodate ligatures in vim.
> >
> > What consideration is paid to those in your patch? I don't see any.
>
> The trick is to activate the optimisation not for all ASCII characters
> (chars < 128), but only for [A-Z], [a-z], [0-9] and space. If a string to
> be displayed contains other characters, it'll pass through the full blown
> shaping and has its glyphs drawn, including ligatures.
>
> > My question in the previous mail was whether or not PragmataPro was a
> monospace font family as binary data. Not that what you felt about it nor
> what the world saw about it. That was a purely technical question. Was it
> too hard for you to check your PragmataPro file(s) with an appropriate
> utility?
>
> I tried to answer your question as best as I could, and I'm sorry if I
> didn't quite hit the mark. In fact, with your new question, I don't even
> know what "monospace font family as binary data" means, maybe you could
> clarify. Unless you can read TrueType/OpenType fonts in hexdumps, and glean
> meaning from it, this will tell you very little. It's not a bitmap font, if
> that's what you mean.
>
If your windowing system is a recent X11 with fontconfig, do
$ fc-scan <your favorite ligature font file>
then you'll have something human readable.
Take a look at an item called 'spacing.'
If you have none or the number zero, then the font is proportional.
If you have the number 90, then the font is dual.
If you have the number 100, then the font is monospace.
If you have the number 110, then the font is charcell
> > I'm afraid your reply to that, which sounds somehow insulting old-school
> people is completely out of place.
>
> I certainly never intended to insult anyone, and apologise if I have given
> offence. In fact, I've been programming for more than 25 years now, and am
> considered old-school by many of my colleagues. Changing one's preference
> in text editor user interface is certainly nothing that is carved in stone
> for all eternity, and people should have a freedom of choice.
>
> > I'm sorry that you don't care about technical aspects of the issue. I
> really feel sick when I hear someone talks about old vs new to justify his
> view.
>
> I tried to present the technical issues at stake to the best of my
> (limited) ability and knowledge. I'm sorry if I didn't quite hit the mark.
> But to assert that I don't care about the technical aspects of the issue is
> a gross oversimplification. In fact, I'd welcome any constructive input I
> can get on this matter. Speaking of which: have you tried the patch with
> your favourite font? Does it work for you? (The patch may very well need
> improving, and I'm happy to do just that. But if you haven't given the
> patch a try, you might want to do that before judging and its author in
> such a harsh tone.)
>
> Concerning your request to send a patch that just fixes the assumption
> about "one ascii character == one glyph": One could either get rid of that
> speed optimisation code altogether (in which case all ligatures will work),
> or rewrite most of that optimisation, doing one shaping call into pango for
> each of (or some of) the characters with codes < 128. The latter versoin
> would amount to some overhead for all those calls, and reassembling the
> returned data structures in a suitable way...
>
> Instead, I preferred to go with a more minimal patch, but I'm more than
> willing to listen to whatever consensus emerges.
>
> Best regards,
>
> Manuel
>
>
> > 2016-08-08 18:15 GMT+09:00 manuelschiller.pimail via vim_dev <
> [email protected]>:
> > On Monday, 8 August 2016 08:31:13 UTC+2, Kazunobu Kuriyama wrote:
> >
> >
> >
> > > 2016-08-07 20:27 GMT+09:00 manuelschiller.pimail via vim_dev <
> [email protected]>:
> >
> > >
> >
> > >
> >
> > > On Monday, 19 October 2015 08:41:36 UTC+2, Hörmetjan Yiltiz wrote:
> >
> > >
> >
> > > > On Sunday, 20 April 2014 11:55:22 UTC-4, François Gannaz wrote:
> >
> > >
> >
> > > > > Hello
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > In a few words, here is a patch that makes gvim work better with
> ligatures
> >
> > >
> >
> > > > > in fonts, which can be useful even for programmers. Details follow.
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > I tried to use a Hasklig[^1], a font with ligatures intended for
> the
> >
> > >
> >
> > > > > Haskell language. It serves the same objective as the Haskell
> Conceal
> >
> > >
> >
> > > > > script[^2], but with the added benefit that even a mouse
> copy-paste works
> >
> > >
> >
> > > > > as intended.
> >
> > >
> >
> > > > > [^1]: https://github.com/i-tu/hasklig
> >
> > >
> >
> > > > > [^2]: http://www.vim.org/scripts/script.php?script_id=3200
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > Unfortunately, gvim doesn't support ligatures on ASCII characters.
> The
> >
> > >
> >
> > > > > following assertion fails at run-time:
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > ascii_glyph_table_init: assertion
> 'gui.ascii_glyphs->num_glyphs ==
> >
> > >
> >
> > > > > sizeof(ascii_chars)' failed
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > and many characters are displayed with the wrong glyphs.
> >
> > >
> >
> > > > > The attached patch limits the function ascii_glyph_table_init() to
> >
> > >
> >
> > > > > spaces and alphanumeric chars. It solves the problem here.
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > Yet I wonder if the current hack with ASCII characters is really
> useful.
> >
> > >
> >
> > > > > Is there any performance test to check if a simpler behaviour
> wouldn't be
> >
> > >
> >
> > > > > suitable, at least for modern desktop installations?
> >
> > >
> >
> > > > > As the code comment mentions spaces, maybe it should be restricted
> to
> >
> > >
> >
> > > > > blank lines?
> >
> > >
> >
> > > > >
> >
> > >
> >
> > > > > Regards
> >
> > >
> >
> > > > > --
> >
> > >
> >
> > > > > François
> >
> > >
> >
> > > >
> >
> > >
> >
> > > > This would be so great to see implemented.
> >
> > >
> >
> > >
> >
> > >
> >
> > > It certainly would (and apologies for reopening such an old
> discussion).
> >
> > >
> >
> > >
> >
> > >
> >
> > > On my system, the patch François sent doesn't work for the ligatures
> in PragmataPro (but flawlessly for other fonts like Hasklig),
> >
> > >
> >
> > >
> >
> > > According to http://www.fsd.it/shop/fonts/pragmatapro/ , PragmataPro
> has "One typeface, 3 families for the right need":
> >
> > >
> >
> > >
> >
> > >
> >
> > > - PragmataPro is a spaced modularly font family with working ligatures.
> >
> > > - PragmataPro Mono is a very monospaced without ligatures.
> >
> > > - Essential PragmataPro is the cheap version to coding for English
> only.
> >
> > >
> >
> > >
> >
> > > As you know, Vim is implemented by design only to support monospace
> font families.
> >
> > >
> >
> > >
> >
> > > So it looks to me that, among those three font families, the only
> possible and suitable choice for Vim to use with is PragmataPro Mono alone.
> >
> >
> >
> > In PragmataPro, all ASCII glyphs etc. are monospaced. Just as they are
> in PragmatePro Mono or Courier, or whatever your favourite monospaced font
> is. The difference between the two is what happens during shaping into
> glyphs inside pango (or another library): For fonts like Hasklig or
> PragmataPro, some combinations of ASCII characters get assigned a new glyph
> (the ligature) which looks better (e.g. "<=" might be replaced with a new
> glyph that more closely mimics the mathematical notation), but fits into
> the normal monospaced character grid. These ligature characters are two and
> sometimes three times the width of a single ASCII glyph, depending on how
> many characters they replace (if it's a monospace font - in a proportional
> font like Times, the ligature for something like "fl" can be any width the
> font designer likes, and I'm not proposing to support this latter case).
> >
> >
> >
> > That kind of character/glyph juggling is okay because it fits with what
> vim is expecting to draw anyway (so vim's drawing code can largely remain
> unaware of the font shaping going on) - it's expecting to draw something
> that's e.g. 2 characters wide, and it draws (a single) glyph that wide.
> >
> >
> >
> > (In PragmataPro Mono, these ligature tables are absent, so shaping just
> returns the normal single-width glyphs.)
> >
> >
> >
> > The point of this patch is that it ensures that vim gets the right
> single width glyphs for all monospaced fonts, even in the code that
> bypasses the glyph shaping process and does its own thing.
> >
> >
> >
> > I've attached screenshots how unpatched vim (as shipped with the latest
> stable debian) and patched vim look with these fonts. In all cases, I do a
> ":version" before I take the screenshot. As you can see, it's totally
> garbled in case of the unpatched versions of the plots, despite the fact
> that the two fonts are monospaced (and are counted as such by all major
> terminal programs).
> >
> >
> >
> > This is a corner case where the optimisation to speed up rendering of
> ASCII glyphs in vim (by bypassing the shaping process) gets it wrong
> (namely the corner case where the font designer specified ligatures for a
> font). What happens in these cases is that pango returns a list with fewer
> glyphs than characters (because some character combinations return only a
> single glyph), and vim happily reads past the end of the glyph list and
> caches the result to render ascii glyphs. The effect is unreadable
> gibberish, as you can see. I think it should just be fixed, because it's
> ugly and needlessly astonishes unsuspecting users (neglecting for a moment
> the fact that there may also be security implications of such accesses past
> the end of a data structure)...
> >
> >
> >
> > > If that is the case, however, that font family is described as
> "without ligatures," which makes your patch no sense.
> >
> >
> >
> > (See above.)
> >
> >
> >
> > > Accordingly, it appears my assumption was wrong; it might be better
> for me to conclude that what you meant by PragmataPro was literally
> PragmataPro, not its monospace version, or PragmataPro Mono.
> >
> >
> >
> > (See above again.)
> >
> >
> >
> > > Then, another question arises: What does "a spaced modularly font"
> mean? Does it imply that the font family in question is technically
> considered something equivalent to a monospaced font and hence deserve
> Vim's support?
> >
> >
> >
> > I'll try and render it yet another way: If you ask pango/... to draw a
> single character in a monospaced font, it's always the same width. But, in
> some monospaced fonts, for some two (and sometimes three) letter
> combinations, the font rendering library substitutes a glyph that looks
> better, but occupies exactly the width of two (three) individual
> characters. While you still fit totally into the monospaced character grid,
> you draw fewer glyphs than input characters now...
> >
> >
> >
> > I think this implies that such modern programmer's fonts like Hasklig or
> PragmataPro are effectively a monospaced fonts, and people will
> increasingly want to use them. (In fact, you're talking to a former
> die-hard text mode console adept, who after 15+ years started to see the
> point of GUIs...) These are questions that people will keep asking
> (probably increasingly often). In my opinion, vim would do well to include
> support for this kind of effectively monospaced fonts, so the answer to
> those questions can become "update to the latest version", and you guys can
> get back to some serious hacking again... ;)
> >
> >
> >
> > > I'd appreciate it if you would give me some clarification or correct
> my possible misunderstanding.
> >
> >
> >
> > I hope this somehow clarifies the issue. I'm not proposing to implement
> proportional font support or anything like that. But the assumption in the
> current code that one ascii character renders to a single glyph is no
> longer true in today's world, and I think that needs to be fixed somehow.
> The fact that it also happens to make Hasklig or PragmataPro render
> correctly is, for me, a welcome side effect (and also happens to be the
> reason why I got started with this). But the longer I think about this
> patch, the more I tend to think of the current (unpatched) drawing code as
> a liability for vim because it reads past the end of a data structure
> without any checks.
> >
> >
> > Why don't you send us a patch on that?
> > We're lucky that only the graphical output is garbled.
> >
> >
> >
> > I hope this clarifies things a bit.
> >
> >
> >
> > Best regards,
> >
> >
> >
> > Manuel
> >
> >
> >
> > > Best regards,
> >
> > > Kazunobu Kuriyama
> >
> > >
> >
> > > apparently because the ASCII-to-glyph mapping won't quite work - I
> suspect that's due to PragmataPro having ligatures in that range, too. I
> solved this by inserting a space between characters that we hand to pango
> for shaping. Sane fonts will not have ligatures between a space and a
> printable character, this way we still get one glyph per ASCII character.
> I've attached the modified patch. This will work with normal fonts and ones
> with programming ligatures like Hasklig and PragmataPro; I've tested it
> with the gtk2 and gtk3 front ends.
> >
> > >
> >
> > >
> >
> > >
> >
> > > Please let me know if it would be possible to include this, and if
> not, at least the patch is public now where people can find it if they want
> it.
> >
> > >
> >
> > >
> >
> > >
> >
> > > Best regards,
> >
> > >
> >
> > >
> >
> > >
> >
> > > Manuel
> >
> > >
> >
> > >
> >
> > >
> >
> > > --
> >
> > >
> >
> > > --
> >
> > >
> >
> > > You received this message from the "vim_dev" maillist.
> >
> > >
> >
> > > Do not top-post! Type your reply below the text you are replying to.
> >
> > >
> >
> > > For more information, visit http://www.vim.org/maillist.php
> >
> > >
> >
> > >
> >
> > >
> >
> > > ---
> >
> > >
> >
> > > You received this message because you are subscribed to the Google
> Groups "vim_dev" group.
> >
> > >
> >
> > > To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected].
> >
> >
> >
> > >
> >
> > > For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
> > --
> >
> > --
> >
> > You received this message from the "vim_dev" maillist.
> >
> > Do not top-post! Type your reply below the text you are replying to.
> >
> > For more information, visit http://www.vim.org/maillist.php
> >
> >
> >
> > ---
> >
> > You received this message because you are subscribed to the Google
> Groups "vim_dev" group.
> >
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected].
> >
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups
> "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
