Re: [O] mis-alignment in org-tables with Tibetan characters
Eric Abrahamsen writes: > The patch series that would not die! :) Applied, thanks! -- Bastien
Re: [O] mis-alignment in org-tables with Tibetan characters
Bastien writes: > Hi Eric, > > Eric Abrahamsen writes: > >> Sorry this took a while to get to... >> >> I think it was a little simpler than I thought -- at least I hope that's >> true, and I'm not missing something really obvious. There are two >> patches attached, a simple one that handles re-justification of table >> fields during field movement, and another that allows for narrowing of >> columns with double-width strings. The second patch is uglier, and >> doesn't work 100% well (you get misalignment if you try to narrow a >> double-wide to an odd number of single-width characters), but it's >> better than nothing. > > I want to apply the patches on master -- can you give them a commit > message with an Emacs changelog? Also, the first one (0001-...) does > not apply on master anymore. > > Thanks in advance! The patch series that would not die! I'm not a git wizard, but I think I was able to apply both these patches to master with no problem: they're now sitting on top of 56b47fb and nothing is going wrong (I think). Can you try again? Here they are with better commit messages. E >From 3e6fd0e2790156a9bd68401bdabbf1b170408ba6 Mon Sep 17 00:00:00 2001 From: Eric Abrahamsen Date: Wed, 13 Feb 2013 11:56:18 +0800 Subject: [PATCH 1/2] Use string width in table field justification lisp/org-table.el (org-table-justify-field-maybe): Use org-string-width instead of length to calculate the visual width of table cells -- helps with double-width charsets, etc. --- lisp/org-table.el | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lisp/org-table.el b/lisp/org-table.el index 6a7935e..5e0177f 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -981,13 +981,16 @@ Optional argument NEW may specify text to replace the current field content." (progn (setq s (match-string 1) o (match-string 0) - l (max 1 (- (match-end 0) (match-beginning 0) 3)) + l (max 1 + (- (org-string-width + (buffer-substring-no-properties +(match-end 0) (match-beginning 0))) 3)) e (not (= (match-beginning 2) (match-end 2 (setq f (format (if num " %%%ds %s" " %%-%ds %s") l (if e "|" (setq org-table-may-need-update t) "")) n (format f s)) (if new - (if (<= (length new) l) ;; FIXME: length -> str-width? + (if (<= (org-string-width new) l) (setq n (format f new)) (setq n (concat new "|") org-table-may-need-update t))) (if (equal (string-to-char n) ?-) (setq n (concat " " n))) -- 1.9.3 >From 7850f739a6ef21bcc92763150861a67798282f12 Mon Sep 17 00:00:00 2001 From: Eric Abrahamsen Date: Wed, 13 Feb 2013 16:46:33 +0800 Subject: [PATCH 2/2] Improve table column narrowing with variable-width strings lisp/org-table.el (org-table-align): Use org-string-width to make a better guess as to how wide a narrowed table column should actually be. This won't work perfectly with variable-width scripts, as it still only narrows in full-column increments, but it's better than before. --- lisp/org-table.el | 19 ++- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/lisp/org-table.el b/lisp/org-table.el index 5e0177f..d1f9686 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -765,7 +765,7 @@ When nil, simply write \"#ERROR\" in corrupted fields.") (hfmt1 (concat (make-string sp2 ?-) "%s" (make-string sp1 ?-) "+")) emptystrings links dates emph raise narrow - falign falign1 fmax f1 len c e space) + falign falign1 fmax f1 f2 len c e space) (untabify beg end) (remove-text-properties beg end '(org-cwidth t org-dwidth t display t)) ;; Check if we have links or dates @@ -851,10 +851,19 @@ When nil, simply write \"#ERROR\" in corrupted fields.") (unless (> f1 1) (user-error "Cannot narrow field starting with wide link \"%s\"" (match-string 0 xx))) - (add-text-properties f1 (length xx) (list 'org-cwidth t) xx) - (add-text-properties (- f1 2) f1 - (list 'display org-narrow-column-arrow) - xx) + (setq f2 (length xx)) + (if (= (org-string-width xx) + f2) + (setq f2 f1) + (setq f2 1) + (while (< (org-string-width (substring xx 0 f2)) + f1) + (setq f2 (1+ f2 + (add-text-properties f2 (length xx) (list 'org-cwidth t) xx) + (add-text-properties (if (>= (string-width (substring xx (1- f2) f2)) 2) + (1- f2) (- f2 2)) f2 + (list 'display org-narrow-column-arrow) + xx) ;; Get the maximum width for each column (push (apply 'max (or fmax 1) 1 (mapcar 'org-string-width column)) lengths) -- 1.9.3
Re: [O] mis-alignment in org-tables with Tibetan characters
Hi Eric, Eric Abrahamsen writes: > Sorry this took a while to get to... > > I think it was a little simpler than I thought -- at least I hope that's > true, and I'm not missing something really obvious. There are two > patches attached, a simple one that handles re-justification of table > fields during field movement, and another that allows for narrowing of > columns with double-width strings. The second patch is uglier, and > doesn't work 100% well (you get misalignment if you try to narrow a > double-wide to an odd number of single-width characters), but it's > better than nothing. I want to apply the patches on master -- can you give them a commit message with an Emacs changelog? Also, the first one (0001-...) does not apply on master anymore. Thanks in advance! -- Bastien
Re: [O] mis-alignment in org-tables with Tibetan characters
Ah yes, this won't work if your font draws characters at variable widths. I use Chinese a lot, and some Chinese fonts will still create misalignment, simply because a Chinese character as drawn as 192% the width of an ascii character. Dunno how to get around that. On 02/11/14 10:50 AM, Steffan Iverson wrote: > Thanks Eric - I've used this patch but I doesn't seem to solve the > problem. I'm working on an earlier suggestion by Michael about the > unicode type that my Tibetan font is. I very much appreciate all this > help! > > Steffan > > > On Sun, Feb 9, 2014 at 8:09 PM, Eric Abrahamsen < > e...@ericabrahamsen.net> wrote: > > Bastien writes: > > > Eric Abrahamsen writes: > > > >> I've been using that patch or something like it for nearly a > year now, > >> with no adverse effects. I'm on the road right now, give me a > day and > >> I'll take a closer look at what I've got... > > > > Great -- thanks in advance! I'll then wait before releasing a > new > > minor version and merging it into Emacs for Emacs 24.4. > > > > To other core maintainers: if you see important issues that > needs to > > be fixed in maint, let me know. > > Sorry this took a while to get to... > > I think it was a little simpler than I thought -- at least I hope > that's > true, and I'm not missing something really obvious. There are two > patches attached, a simple one that handles re-justification of > table > fields during field movement, and another that allows for > narrowing of > columns with double-width strings. The second patch is uglier, > and > doesn't work 100% well (you get misalignment if you try to narrow > a > double-wide to an odd number of single-width characters), but > it's > better than nothing. > > Please test! > > Eric >
Re: [O] mis-alignment in org-tables with Tibetan characters
Thanks Eric - I've used this patch but I doesn't seem to solve the problem. I'm working on an earlier suggestion by Michael about the unicode type that my Tibetan font is. I very much appreciate all this help! Steffan On Sun, Feb 9, 2014 at 8:09 PM, Eric Abrahamsen wrote: > Bastien writes: > > > Eric Abrahamsen writes: > > > >> I've been using that patch or something like it for nearly a year now, > >> with no adverse effects. I'm on the road right now, give me a day and > >> I'll take a closer look at what I've got... > > > > Great -- thanks in advance! I'll then wait before releasing a new > > minor version and merging it into Emacs for Emacs 24.4. > > > > To other core maintainers: if you see important issues that needs to > > be fixed in maint, let me know. > > Sorry this took a while to get to... > > I think it was a little simpler than I thought -- at least I hope that's > true, and I'm not missing something really obvious. There are two > patches attached, a simple one that handles re-justification of table > fields during field movement, and another that allows for narrowing of > columns with double-width strings. The second patch is uglier, and > doesn't work 100% well (you get misalignment if you try to narrow a > double-wide to an odd number of single-width characters), but it's > better than nothing. > > Please test! > > Eric > >
Re: [O] mis-alignment in org-tables with Tibetan characters
Bastien writes: > Eric Abrahamsen writes: > >> I've been using that patch or something like it for nearly a year now, >> with no adverse effects. I'm on the road right now, give me a day and >> I'll take a closer look at what I've got... > > Great -- thanks in advance! I'll then wait before releasing a new > minor version and merging it into Emacs for Emacs 24.4. > > To other core maintainers: if you see important issues that needs to > be fixed in maint, let me know. Sorry this took a while to get to... I think it was a little simpler than I thought -- at least I hope that's true, and I'm not missing something really obvious. There are two patches attached, a simple one that handles re-justification of table fields during field movement, and another that allows for narrowing of columns with double-width strings. The second patch is uglier, and doesn't work 100% well (you get misalignment if you try to narrow a double-wide to an odd number of single-width characters), but it's better than nothing. Please test! Eric >From f03f4755b2d3e62d9f88845d93390a364805c131 Mon Sep 17 00:00:00 2001 From: Eric Abrahamsen Date: Wed, 13 Feb 2013 11:56:18 +0800 Subject: [PATCH 1/9] Make table field justification respect string width --- lisp/org-table.el | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lisp/org-table.el b/lisp/org-table.el index 520ac8a..33ae63e 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -973,14 +973,16 @@ Optional argument NEW may specify text to replace the current field content." (progn (setq s (match-string 1) o (match-string 0) - l (max 1 (- (match-end 0) (match-beginning 0) 3)) + l (max 1 + (- (org-string-width + (buffer-substring +(match-end 0) (match-beginning 0))) 3)) e (not (= (match-beginning 2) (match-end 2 (setq f (format (if num " %%%ds %s" " %%-%ds %s") l (if e "|" (setq org-table-may-need-update t) "")) n (format f s)) (if new - (if (<= (length new) l) ;; FIXME: length -> str-width? + (if (<= (org-string-width new) l) (setq n (format f new)) (setq n (concat new "|") org-table-may-need-update t))) (if (equal (string-to-char n) ?-) (setq n (concat " " n))) -- 1.8.5.4 >From 85aee797a97194d9c29d7baac641f3849f7f Mon Sep 17 00:00:00 2001 From: Eric Abrahamsen Date: Wed, 13 Feb 2013 16:46:33 +0800 Subject: [PATCH 2/9] Make table column narrowing play nice with variable-width strings --- lisp/org-table.el | 19 ++- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/lisp/org-table.el b/lisp/org-table.el index 33ae63e..32dd025 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -758,7 +758,7 @@ When nil, simply write \"#ERROR\" in corrupted fields.") (hfmt1 (concat (make-string sp2 ?-) "%s" (make-string sp1 ?-) "+")) emptystrings links dates emph raise narrow - falign falign1 fmax f1 len c e space) + falign falign1 fmax f1 f2 len c e space) (untabify beg end) (remove-text-properties beg end '(org-cwidth t org-dwidth t display t)) ;; Check if we have links or dates @@ -843,10 +843,19 @@ When nil, simply write \"#ERROR\" in corrupted fields.") (unless (> f1 1) (user-error "Cannot narrow field starting with wide link \"%s\"" (match-string 0 xx))) - (add-text-properties f1 (length xx) (list 'org-cwidth t) xx) - (add-text-properties (- f1 2) f1 - (list 'display org-narrow-column-arrow) - xx) + (setq f2 (length xx)) + (if (= (org-string-width xx) + f2) + (setq f2 f1) + (setq f2 1) + (while (< (org-string-width (substring xx 0 f2)) + f1) + (setq f2 (1+ f2 + (add-text-properties f2 (length xx) (list 'org-cwidth t) xx) + (add-text-properties (if (>= (string-width (substring xx (1- f2) f2)) 2) + (1- f2) (- f2 2)) f2 + (list 'display org-narrow-column-arrow) + xx) ;; Get the maximum width for each column (push (apply 'max (or fmax 1) 1 (mapcar 'org-string-width column)) lengths) -- 1.8.5.4
Re: [O] mis-alignment in org-tables with Tibetan characters
Hi Steffan On Wed, Feb 5, 2014 at 6:35 AM, Michael Brand wrote: > Is there any font that is monospaced even in the Tibetan range at all? On one of my "trips" of learning more about unicode I just discovered the monospaced bitmap font GNU Unifont [1] [2] which implements most of (!) the range U+ to U+ and should be ok for a whole lot of tests. See PNG image of e. g. version 6.3.20131006 [3]. The unicode script Tibetan you need starts at 0F00. Michael [1] http://en.wikipedia.org/wiki/GNU_Unifont [2] http://unifoundry.com/unifont.html [3] http://upload.wikimedia.org/wikipedia/commons/e/e3/Unifont-6.3.20131006.png
Re: [O] mis-alignment in org-tables with Tibetan characters
Hi Steffan On Wed, Feb 5, 2014 at 1:00 AM, Steffan Iverson wrote: > Michael, that test case you suggest does indeed cause the same spacing > problem. So this isn't a problem from the vowel stacking in Tibetan, Such combining characters and and also double width characters may still remain as separate issues for Org table alignment and maybe the patches from Eric Abrahamsen are required to resolve one or both. > because of the type of font it is? Sorry, generally new to Emacs so > I'm not sure how to move forward with that insight. Several steps to go with choosing font(s) and probably tweaking set-fontset-font which I don't have much insight neither. Regarding the choice of font: Is there any font that is monospaced even in the Tibetan range at all? What does a "cat" (not "less" or any other ncurses) of your file look like in different terminal emulators like xterm, gnome-terminal etc. on different OSes? What does "locale" from these terminal emulators tell? Regarding a possible first step with Emacs set-fontset-font: How did the glyph for INFINITY from the test look like, was this aligned for you?: xx| ∞∞| Michael
Re: [O] mis-alignment in org-tables with Tibetan characters
Michael, that test case you suggest does indeed cause the same spacing problem. So this isn't a problem from the vowel stacking in Tibetan, because of the type of font it is? Sorry, generally new to Emacs so I'm not sure how to move forward with that insight. On Tue, Feb 4, 2014 at 10:13 AM, Michael Brand wrote: > Hi Steffan > > On Mon, Feb 3, 2014 at 6:33 AM, Steffan Iverson > wrote: > > Screenshot: http://oi59.tinypic.com/lz893.jpg > > Judging from e. g. this part > | རྐུས་ | rkus | > | གསོས་ | gsos | > in your screenshot my speculation is that you get glyphs from not only your > default monospace font but also from other (maybe variable width) font(s). > Either because some glyphs are not available in your default monospace font > or are not used due to an incomplete configuration with set-fontset-font. > > For a diagnosis in this direction I would check the alignment of "|" after > some characters just repeated like here: > > xx| > ་་| > གག| > རར| > སས| > ུུ| > ོོ| > ྐྐ| > xx| > ∞∞| > > Michael >
Re: [O] mis-alignment in org-tables with Tibetan characters
Hi Steffan On Mon, Feb 3, 2014 at 6:33 AM, Steffan Iverson wrote: > Screenshot: http://oi59.tinypic.com/lz893.jpg Judging from e. g. this part | རྐུས་ | rkus | | གསོས་ | gsos | in your screenshot my speculation is that you get glyphs from not only your default monospace font but also from other (maybe variable width) font(s). Either because some glyphs are not available in your default monospace font or are not used due to an incomplete configuration with set-fontset-font. For a diagnosis in this direction I would check the alignment of "|" after some characters just repeated like here: xx| ་་| གག| རར| སས| ུུ| ོོ| ྐྐ| xx| ∞∞| Michael
Re: [O] mis-alignment in org-tables with Tibetan characters
Eric Abrahamsen writes: > I've been using that patch or something like it for nearly a year now, > with no adverse effects. I'm on the road right now, give me a day and > I'll take a closer look at what I've got... Great -- thanks in advance! I'll then wait before releasing a new minor version and merging it into Emacs for Emacs 24.4. To other core maintainers: if you see important issues that needs to be fixed in maint, let me know. -- Bastien
Re: [O] mis-alignment in org-tables with Tibetan characters
Bastien writes: > Hi Steffan, > > thanks for reporting this. > > Steffan Iverson writes: > >> Any way to solve this problem? > > This is a recurring problem, and Eric is the one who tried > to solve it. > > Eric, I don't remember why we didn't follow-up on your patch > here: > > http://lists.gnu.org/archive/html/emacs-orgmode/2013-02/msg00664.html > > Would that help in this situation? I've been using that patch or something like it for nearly a year now, with no adverse effects. I'm on the road right now, give me a day and I'll take a closer look at what I've got... E
Re: [O] mis-alignment in org-tables with Tibetan characters
Hi Steffan, thanks for reporting this. Steffan Iverson writes: > Any way to solve this problem? This is a recurring problem, and Eric is the one who tried to solve it. Eric, I don't remember why we didn't follow-up on your patch here: http://lists.gnu.org/archive/html/emacs-orgmode/2013-02/msg00664.html Would that help in this situation? Thanks, -- Bastien
Re: [O] mis-alignment in org-tables with Tibetan characters
Screenshot: http://oi59.tinypic.com/lz893.jpg On Sun, Feb 2, 2014 at 7:29 PM, Steffan Iverson wrote: > Hello all, > > > I've made an org table that includes both English and Tibetan characters, > and the columns don't line up. I suspect this is because of the way > Tibetan > characters are displayed - they "stack" below and above each other, making > the characters vertically taller than the English characters. You can see > this > same problem in the text pasted below, it behaves the same way in an org > table. Any way to solve this problem? Perhaps somehow changing the height > of all the text in the table? Thanks! > > | རྐུ་བ་| rku-ba | བརྐུས་པ་ | brkus-pa| བརྐུ་བ་ | > brku-ba| རྐུས་ | rkus| > | གསོ་བ་ | gso-ba | གསོས་པ་ | gsos-pa | གསོ་བ་ | gso-ba | > གསོས་ | gsos| > | འཆོར་བ་ | 'chor-ba | ཤོར་བ་ | shor-ba | འཆོར་བ་ | 'chor-ba | >
[O] mis-alignment in org-tables with Tibetan characters
Hello all, I've made an org table that includes both English and Tibetan characters, and the columns don't line up. I suspect this is because of the way Tibetan characters are displayed - they "stack" below and above each other, making the characters vertically taller than the English characters. You can see this same problem in the text pasted below, it behaves the same way in an org table. Any way to solve this problem? Perhaps somehow changing the height of all the text in the table? Thanks! | རྐུ་བ་| rku-ba | བརྐུས་པ་ | brkus-pa| བརྐུ་བ་ | brku-ba | རྐུས་ | rkus| | གསོ་བ་ | gso-ba | གསོས་པ་ | gsos-pa | གསོ་བ་ | gso-ba | གསོས་ | gsos| | འཆོར་བ་ | 'chor-ba | ཤོར་བ་ | shor-ba | འཆོར་བ་ | 'chor-ba |
