Re: broken table formatting in psql
On Thu, Sep 8, 2022 at 12:39 PM John Naylor wrote: > > On Fri, Sep 2, 2022 at 3:19 PM Kyotaro Horiguchi > wrote: > > > > At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor > > wrote in > > > If there is any doubt about including all of Cf, we could also just > > > add a branch in wchar.c to hard-code the 200B-200F range. > > > > If every way has defect to the similar extent, I think we will choose > > to use authoritative data at least for the first step. We might want > > to have additional filtering on it but it would be another issue, > > maybe. > > > > Attached is the first cut of that. (The commit messages is not great, > > though.) > > Okay, the patch looks good to me overall. As discussed, I pushed to master only, with only one additional comment in the perl script to describe Me/Mn/Cf. -- John Naylor EDB: http://www.enterprisedb.com
Re: broken table formatting in psql
po 12. 9. 2022 v 10:29 odesílatel John Naylor napsal: > On Mon, Sep 12, 2022 at 12:44 PM Pavel Stehule > wrote: > > This is not a critical issue, really. On second thought, I don't see > the point in releasing fresh Postgres with this bug, where there is know > bugfix - and this bugfix should be compatible (at this moment) with 16. > > I agree the actual logic/data change is low-risk. The patch renames > two files, which seems a bit much this late in the cycle. Maybe that's > okay, but I'd like someone else to opine before doing so. > understand Pavel > > -- > John Naylor > EDB: http://www.enterprisedb.com >
Re: broken table formatting in psql
On Mon, Sep 12, 2022 at 12:44 PM Pavel Stehule wrote: > This is not a critical issue, really. On second thought, I don't see the > point in releasing fresh Postgres with this bug, where there is know bugfix - > and this bugfix should be compatible (at this moment) with 16. I agree the actual logic/data change is low-risk. The patch renames two files, which seems a bit much this late in the cycle. Maybe that's okay, but I'd like someone else to opine before doing so. -- John Naylor EDB: http://www.enterprisedb.com
Re: broken table formatting in psql
po 12. 9. 2022 v 7:37 odesílatel John Naylor napsal: > On Thu, Sep 8, 2022 at 12:51 PM Pavel Stehule > wrote: > > > > > >> Does anyone want to advocate for backpatching these missing ranges to > >> v12 and up? v12 still has a table in-line so trivial to remedy, but > >> v13 and up use a script, so these exceptions would likely have to use > >> hard-coded branches to keep from bringing in new changes. > >> > >> If so, does anyone want to advocate for including this patch in v15? > >> It claims Unicode 14.0.0, and this would make that claim more > >> technically correct as well as avoiding additional branches. > > > > > > I think it can be fixed just in v15 and master. This issue has no > impact on SQL. > > Well, if the regressions from v11 are not important enough to > backpatch, there is not as much of a case to backpatch the full fix to > v15 either. > This is not a critical issue, really. On second thought, I don't see the point in releasing fresh Postgres with this bug, where there is know bugfix - and this bugfix should be compatible (at this moment) with 16. PostgreSQL 15 was not released yet. Regards Pavel -- > John Naylor > EDB: http://www.enterprisedb.com >
Re: broken table formatting in psql
On Thu, Sep 8, 2022 at 12:51 PM Pavel Stehule wrote: > > >> Does anyone want to advocate for backpatching these missing ranges to >> v12 and up? v12 still has a table in-line so trivial to remedy, but >> v13 and up use a script, so these exceptions would likely have to use >> hard-coded branches to keep from bringing in new changes. >> >> If so, does anyone want to advocate for including this patch in v15? >> It claims Unicode 14.0.0, and this would make that claim more >> technically correct as well as avoiding additional branches. > > > I think it can be fixed just in v15 and master. This issue has no impact on > SQL. Well, if the regressions from v11 are not important enough to backpatch, there is not as much of a case to backpatch the full fix to v15 either. -- John Naylor EDB: http://www.enterprisedb.com
Re: broken table formatting in psql
čt 8. 9. 2022 v 7:39 odesílatel John Naylor
napsal:
> On Fri, Sep 2, 2022 at 3:19 PM Kyotaro Horiguchi
> wrote:
> >
> > At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor <
> john.nay...@enterprisedb.com> wrote in
> > > If there is any doubt about including all of Cf, we could also just
> > > add a branch in wchar.c to hard-code the 200B-200F range.
> >
> > If every way has defect to the similar extent, I think we will choose
> > to use authoritative data at least for the first step. We might want
> > to have additional filtering on it but it would be another issue,
> > maybe.
> >
> > Attached is the first cut of that. (The commit messages is not great,
> > though.)
>
> Okay, the patch looks good to me overall. Comparing releases, some
> other ranges were in v11 but left out in v12 with the transition to
> using a script:
>
> 0x070F
> {0x200B, 0x200F}
> {0x202A, 0x202E}
> {0x206A, 0x206F}
> 0xFEFF
> {0xFFF9, 0xFFFB}
>
> Does anyone want to advocate for backpatching these missing ranges to
> v12 and up? v12 still has a table in-line so trivial to remedy, but
> v13 and up use a script, so these exceptions would likely have to use
> hard-coded branches to keep from bringing in new changes.
>
> If so, does anyone want to advocate for including this patch in v15?
> It claims Unicode 14.0.0, and this would make that claim more
> technically correct as well as avoiding additional branches.
>
I think it can be fixed just in v15 and master. This issue has no impact
on SQL.
Thank you for fixing this issue
Regards
Pavel
> --
> John Naylor
> EDB: http://www.enterprisedb.com
>
Re: broken table formatting in psql
On Fri, Sep 2, 2022 at 3:19 PM Kyotaro Horiguchi
wrote:
>
> At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor
> wrote in
> > If there is any doubt about including all of Cf, we could also just
> > add a branch in wchar.c to hard-code the 200B-200F range.
>
> If every way has defect to the similar extent, I think we will choose
> to use authoritative data at least for the first step. We might want
> to have additional filtering on it but it would be another issue,
> maybe.
>
> Attached is the first cut of that. (The commit messages is not great,
> though.)
Okay, the patch looks good to me overall. Comparing releases, some
other ranges were in v11 but left out in v12 with the transition to
using a script:
0x070F
{0x200B, 0x200F}
{0x202A, 0x202E}
{0x206A, 0x206F}
0xFEFF
{0xFFF9, 0xFFFB}
Does anyone want to advocate for backpatching these missing ranges to
v12 and up? v12 still has a table in-line so trivial to remedy, but
v13 and up use a script, so these exceptions would likely have to use
hard-coded branches to keep from bringing in new changes.
If so, does anyone want to advocate for including this patch in v15?
It claims Unicode 14.0.0, and this would make that claim more
technically correct as well as avoiding additional branches.
--
John Naylor
EDB: http://www.enterprisedb.com
Re: broken table formatting in psql
On 2022-Sep-02, Kyotaro Horiguchi wrote: > UnicodeData.txt > 174:00AD;SOFT HYPHEN;Cf;0;BN;N; > > Soft-hyphen seems like not zero-width.. usually... Soft-hyphen *is* zero width. It should not be displayed. It's just a marker so that typesetting software knows where to add real hyphens in case a word is too long to appear in a single line. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Re: broken table formatting in psql
At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor
wrote in
> On Fri, Sep 2, 2022 at 12:17 PM Kyotaro Horiguchi
> wrote:
> > > Including them into unicode_combining_table.h actually worked, but I'm
> > > not sure it is valid to include Cf's among Mn/Me's..
>
> Looking at the definition, Cf means "other, format" category, "Format
> character that affects the layout of text or the operation of text
> processes, but is not normally rendered". [1]
>
> > UnicodeData.txt
> > 174:00AD;SOFT HYPHEN;Cf;0;BN;N;
> >
> > Soft-hyphen seems like not zero-width.. usually...
>
> I gather it only appears at line breaks, which I doubt we want to handle.
Yeah. Sounds reasonable. (Emacs always renders it, though..)
> > 0600;ARABIC NUMBER SIGN;Cf;0;AN;N;
> > 110BD;KAITHI NUMBER SIGN;Cf;0;L;N;
> >
> > Mmm. These looks like not zero-width?
>
> There are glyphs, but there is something special about the first one:
>
> select U&'\0600';
>
> Looks like this in psql (substituting 'X' to avoid systemic differences):
>
> +--+
> | ?column? |
> +--+
> | X |
> +--+
> (1 row)
>
> Copy from psql to vim or nano:
>
> +--+
> | ?column? |
> +--+
> | X|
> +--+
> (1 row)
>
> ...so it does mess up the border the same way. The second
> (U&'\+0110bd') doesn't render for me.
Anyway it is inevitably rendering-environment dependent.
> > However, it seems like basically a win if we include "Cf"s to the
> > "combining" table?
>
> There seems to be a case for that. If we did include those, we should
> rename the table to match.
Agreed:)
> I found this old document from 2002 on "default ignorable" characters
> that normally have no visible glyph:
>
> https://unicode.org/L2/L2002/02368-default-ignorable.html
Mmm. Too old?
> If there is any doubt about including all of Cf, we could also just
> add a branch in wchar.c to hard-code the 200B-200F range.
If every way has defect to the similar extent, I think we will choose
to use authoritative data at least for the first step. We might want
to have additional filtering on it but it would be another issue,
maybe.
Attached is the first cut of that. (The commit messages is not great,
though.)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
>From fefff124798c3b716a64893ce04e9331b301a379 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi
Date: Fri, 2 Sep 2022 17:16:03 +0900
Subject: [PATCH v1] Treat Unicode characters of category "Format" as
non-spacing
At PostgreSQL12, the table for non-spacing characters that
ucs_wcwidth() uses was updated based on the UnicodeData.txt so that it
consists of the characters of category Mn and Me. By this change the
table loses some actually zero-width characters. This change broke
indentation of psql outputs when these characters are involved.
Add all characters of category Cf (Format) which are not normally
rendered. However, some corner-case can be left alone.
SOFT_HYPHEN (U+00AD) can be rendered for some conditions but we don't
break a line at it so we assume it is not rendered in psql output.
ARABIC_NUMBER_SIGN(U+0600), KAITHI_NUMBER_SIGN (U+110BD) and some
similar characters seems to have glyphs so they might cause the
calculation wrong.
---
src/common/unicode/Makefile | 4 +--
...l => generate-unicode_nonspacing_table.pl} | 10 +++---
src/common/wchar.c| 8 ++---
...ing_table.h => unicode_nonspacing_table.h} | 33 ---
4 files changed, 32 insertions(+), 23 deletions(-)
rename src/common/unicode/{generate-unicode_combining_table.pl => generate-unicode_nonspacing_table.pl} (68%)
rename src/include/common/{unicode_combining_table.h => unicode_nonspacing_table.h} (91%)
diff --git a/src/common/unicode/Makefile b/src/common/unicode/Makefile
index 60e01e748f..382da476cf 100644
--- a/src/common/unicode/Makefile
+++ b/src/common/unicode/Makefile
@@ -18,7 +18,7 @@ LIBS += $(PTHREAD_LIBS)
# By default, do nothing.
all:
-update-unicode: unicode_norm_table.h unicode_combining_table.h unicode_east_asian_fw_table.h unicode_normprops_table.h unicode_norm_hashfunc.h
+update-unicode: unicode_norm_table.h unicode_nonspacing_table.h unicode_east_asian_fw_table.h unicode_normprops_table.h unicode_norm_hashfunc.h
mv $^ $(top_srcdir)/src/include/common/
$(MAKE) normalization-check
@@ -35,7 +35,7 @@ unicode_norm_hashfunc.h: unicode_norm_table.h
unicode_norm_table.h: generate-unicode_norm_table.pl UnicodeData.txt CompositionExclusions.txt
$(PERL) $<
-unicode_combining_table.h: generate-unicode_combining_table.pl UnicodeData.txt
+unicode_nonspacing_table.h: generate-unicode_nonspacing_table.pl UnicodeData.txt
$(PERL) $^ >$@
unicode_east_asian_fw_table.h: generate-unicode_east_asian_fw_table.pl EastAsianWidth.txt
diff --git a/src/common/unicode/generate-unicode_combining_table.pl b/src/common/unicode/generate-unicode_nonspacing_table.pl
similarity index 68%
rename from
Re: broken table formatting in psql
On Fri, Sep 2, 2022 at 12:17 PM Kyotaro Horiguchi wrote: > > At Thu, 01 Sep 2022 18:22:06 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Thu, 1 Sep 2022 15:00:38 +0700, John Naylor > > wrote in > > > UnicodeData.txt has this: > > > > > > 200B;ZERO WIDTH SPACE;Cf;0;BN;N; > > > 200C;ZERO WIDTH NON-JOINER;Cf;0;BN;N; > > > 200D;ZERO WIDTH JOINER;Cf;0;BN;N; > > > 200E;LEFT-TO-RIGHT MARK;Cf;0;L;N; > > > 200F;RIGHT-TO-LEFT MARK;Cf;0;R;N; > > > > > > So maybe we need to take Cf characters in this file into account, in > > > addition to Me and Mn (combining characters). > > > > Including them into unicode_combining_table.h actually worked, but I'm > > not sure it is valid to include Cf's among Mn/Me's.. Looking at the definition, Cf means "other, format" category, "Format character that affects the layout of text or the operation of text processes, but is not normally rendered". [1] > UnicodeData.txt > 174:00AD;SOFT HYPHEN;Cf;0;BN;N; > > Soft-hyphen seems like not zero-width.. usually... I gather it only appears at line breaks, which I doubt we want to handle. > 0600;ARABIC NUMBER SIGN;Cf;0;AN;N; > 110BD;KAITHI NUMBER SIGN;Cf;0;L;N; > > Mmm. These looks like not zero-width? There are glyphs, but there is something special about the first one: select U&'\0600'; Looks like this in psql (substituting 'X' to avoid systemic differences): +--+ | ?column? | +--+ | X | +--+ (1 row) Copy from psql to vim or nano: +--+ | ?column? | +--+ | X| +--+ (1 row) ...so it does mess up the border the same way. The second (U&'\+0110bd') doesn't render for me. > However, it seems like basically a win if we include "Cf"s to the > "combining" table? There seems to be a case for that. If we did include those, we should rename the table to match. I found this old document from 2002 on "default ignorable" characters that normally have no visible glyph: https://unicode.org/L2/L2002/02368-default-ignorable.html If there is any doubt about including all of Cf, we could also just add a branch in wchar.c to hard-code the 200B-200F range. -- John Naylor EDB: http://www.enterprisedb.com
Re: broken table formatting in psql
At Thu, 01 Sep 2022 18:22:06 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 1 Sep 2022 15:00:38 +0700, John Naylor > wrote in > > UnicodeData.txt has this: > > > > 200B;ZERO WIDTH SPACE;Cf;0;BN;N; > > 200C;ZERO WIDTH NON-JOINER;Cf;0;BN;N; > > 200D;ZERO WIDTH JOINER;Cf;0;BN;N; > > 200E;LEFT-TO-RIGHT MARK;Cf;0;L;N; > > 200F;RIGHT-TO-LEFT MARK;Cf;0;R;N; > > > > So maybe we need to take Cf characters in this file into account, in > > addition to Me and Mn (combining characters). > > Including them into unicode_combining_table.h actually worked, but I'm > not sure it is valid to include Cf's among Mn/Me's.. UnicodeData.txt 174:00AD;SOFT HYPHEN;Cf;0;BN;N; Soft-hyphen seems like not zero-width.. usually... 0600;ARABIC NUMBER SIGN;Cf;0;AN;N; 110BD;KAITHI NUMBER SIGN;Cf;0;L;N; Mmm. These looks like not zero-width? However, it seems like basically a win if we include "Cf"s to the "combining" table? 174:00AD;SOFT HYPHEN;Cf;0;BN;N; 1499:0600;ARABIC NUMBER SIGN;Cf;0;AN;N; 1500:0601;ARABIC SIGN SANAH;Cf;0;AN;N; 1501:0602;ARABIC FOOTNOTE MARKER;Cf;0;AN;N; 1502:0603;ARABIC SIGN SAFHA;Cf;0;AN;N; 1503:0604;ARABIC SIGN SAMVAT;Cf;0;AN;N; 1504:0605;ARABIC NUMBER MARK ABOVE;Cf;0;AN;N; 1527:061C;ARABIC LETTER MARK;Cf;0;AL;N; 1720:06DD;ARABIC END OF AYAH;Cf;0;AN;N; 1769:070F;SYRIAC ABBREVIATION MARK;Cf;0;AL;N; 2124:0890;ARABIC POUND MARK ABOVE;Cf;0;AN;N; 2125:0891;ARABIC PIASTRE MARK ABOVE;Cf;0;AN;N; 2200:08E2;ARABIC DISPUTED END OF AYAH;Cf;0;AN;N; 5517:180E;MONGOLIAN VOWEL SEPARATOR;Cf;0;BN;N; 7365:200B;ZERO WIDTH SPACE;Cf;0;BN;N; 7366:200C;ZERO WIDTH NON-JOINER;Cf;0;BN;N; 7367:200D;ZERO WIDTH JOINER;Cf;0;BN;N; 7368:200E;LEFT-TO-RIGHT MARK;Cf;0;L;N; 7369:200F;RIGHT-TO-LEFT MARK;Cf;0;R;N; 7396:202A;LEFT-TO-RIGHT EMBEDDING;Cf;0;LRE;N; 7397:202B;RIGHT-TO-LEFT EMBEDDING;Cf;0;RLE;N; 7398:202C;POP DIRECTIONAL FORMATTING;Cf;0;PDF;N; 7399:202D;LEFT-TO-RIGHT OVERRIDE;Cf;0;LRO;N; 7400:202E;RIGHT-TO-LEFT OVERRIDE;Cf;0;RLO;N; 7450:2060;WORD JOINER;Cf;0;BN;N; 7451:2061;FUNCTION APPLICATION;Cf;0;BN;N; 7452:2062;INVISIBLE TIMES;Cf;0;BN;N; 7453:2063;INVISIBLE SEPARATOR;Cf;0;BN;N; 7454:2064;INVISIBLE PLUS;Cf;0;BN;N; 7455:2066;LEFT-TO-RIGHT ISOLATE;Cf;0;LRI;N; 7456:2067;RIGHT-TO-LEFT ISOLATE;Cf;0;RLI;N; 7457:2068;FIRST STRONG ISOLATE;Cf;0;FSI;N; 7458:2069;POP DIRECTIONAL ISOLATE;Cf;0;PDI;N; 7459:206A;INHIBIT SYMMETRIC SWAPPING;Cf;0;BN;N; 7460:206B;ACTIVATE SYMMETRIC SWAPPING;Cf;0;BN;N; 7461:206C;INHIBIT ARABIC FORM SHAPING;Cf;0;BN;N; 7462:206D;ACTIVATE ARABIC FORM SHAPING;Cf;0;BN;N; 7463:206E;NATIONAL DIGIT SHAPES;Cf;0;BN;N; 7464:206F;NOMINAL DIGIT SHAPES;Cf;0;BN;N; 16660:FEFF;ZERO WIDTH NO-BREAK SPACE;Cf;0;BN;N;BYTE ORDER MARK 16886:FFF9;INTERLINEAR ANNOTATION ANCHOR;Cf;0;ON;N; 16887:FFFA;INTERLINEAR ANNOTATION SEPARATOR;Cf;0;ON;N; 16888:FFFB;INTERLINEAR ANNOTATION TERMINATOR;Cf;0;ON;N; 19731:110BD;KAITHI NUMBER SIGN;Cf;0;L;N; 19737:110CD;KAITHI NUMBER SIGN ABOVE;Cf;0;L;N; 24043:13430;EGYPTIAN HIEROGLYPH VERTICAL JOINER;Cf;0;L;N; 24044:13431;EGYPTIAN HIEROGLYPH HORIZONTAL JOINER;Cf;0;L;N; 24045:13432;EGYPTIAN HIEROGLYPH INSERT AT TOP START;Cf;0;L;N; 24046:13433;EGYPTIAN HIEROGLYPH INSERT AT BOTTOM START;Cf;0;L;N; 24047:13434;EGYPTIAN HIEROGLYPH INSERT AT TOP END;Cf;0;L;N; 24048:13435;EGYPTIAN HIEROGLYPH INSERT AT BOTTOM END;Cf;0;L;N; 24049:13436;EGYPTIAN HIEROGLYPH OVERLAY MIDDLE;Cf;0;L;N; 24050:13437;EGYPTIAN HIEROGLYPH BEGIN SEGMENT;Cf;0;L;N; 24051:13438;EGYPTIAN HIEROGLYPH END SEGMENT;Cf;0;L;N; 27838:1BCA0;SHORTHAND FORMAT LETTER OVERLAP;Cf;0;BN;N; 27839:1BCA1;SHORTHAND FORMAT CONTINUING OVERLAP;Cf;0;BN;N; 27840:1BCA2;SHORTHAND FORMAT DOWN STEP;Cf;0;BN;N; 27841:1BCA3;SHORTHAND FORMAT UP STEP;Cf;0;BN;N; 28386:1D173;MUSICAL SYMBOL BEGIN BEAM;Cf;0;BN;N; 28387:1D174;MUSICAL SYMBOL END BEAM;Cf;0;BN;N; 28388:1D175;MUSICAL SYMBOL BEGIN TIE;Cf;0;BN;N; 28389:1D176;MUSICAL SYMBOL END TIE;Cf;0;BN;N; 28390:1D177;MUSICAL SYMBOL BEGIN SLUR;Cf;0;BN;N; 28391:1D178;MUSICAL SYMBOL END SLUR;Cf;0;BN;N; 28392:1D179;MUSICAL SYMBOL BEGIN PHRASE;Cf;0;BN;N; 28393:1D17A;MUSICAL SYMBOL END PHRASE;Cf;0;BN;N; 34286:E0001;LANGUAGE TAG;Cf;0;BN;N; 34287:E0020;TAG SPACE;Cf;0;BN;N; 34288:E0021;TAG EXCLAMATION MARK;Cf;0;BN;N; 34289:E0022;TAG QUOTATION
Re: broken table formatting in psql
At Thu, 1 Sep 2022 15:00:38 +0700, John Naylor
wrote in
> On Thu, Sep 1, 2022 at 2:13 PM Pavel Stehule wrote:
> > problem is in bad width of invisible char 200E
>
> I removed this comment in bab982161e since it didn't match the code.
> I'd be interested to see what happened after v12.
>
> - * - Other format characters (general category code Cf in the Unicode
> - * database) and ZERO WIDTH SPACE (U+200B) have a column
> width of 0.
>
> UnicodeData.txt has this:
>
> 200B;ZERO WIDTH SPACE;Cf;0;BN;N;
> 200C;ZERO WIDTH NON-JOINER;Cf;0;BN;N;
> 200D;ZERO WIDTH JOINER;Cf;0;BN;N;
> 200E;LEFT-TO-RIGHT MARK;Cf;0;L;N;
> 200F;RIGHT-TO-LEFT MARK;Cf;0;R;N;
>
> So maybe we need to take Cf characters in this file into account, in
> addition to Me and Mn (combining characters).
Including them into unicode_combining_table.h actually worked, but I'm
not sure it is valid to include Cf's among Mn/Me's..
> diff --git a/src/common/unicode/generate-unicode_combining_table.pl
> b/src/common/unicode/generate-unicode_combining_table.pl
> index 8177c20260..7030bc637b 100644
> --- a/src/common/unicode/generate-unicode_combining_table.pl
> +++ b/src/common/unicode/generate-unicode_combining_table.pl
> @@ -25,7 +25,7 @@ foreach my $line ()
> my @fields = split ';', $line;
> $codepoint = hex $fields[0];
>
> - if ($fields[2] eq 'Me' || $fields[2] eq 'Mn')
> + if ($fields[2] eq 'Me' || $fields[2] eq 'Mn' || $fields[2] eq 'Cf')
> {
> # combining character, save for start of range
> if (!defined($range_start))
By the way I was super annoyed that it was super-hard to reflect the
changes under src/common to the final binary. There are two hops of
missing dependencies and finally ccache stood in my way.. I find that
Andres once meant to try that using --dependency-files but I hope we
make that reflection automated even if we do define the dependencies
manually..
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Re: broken table formatting in psql
On Thu, Sep 1, 2022 at 2:13 PM Pavel Stehule wrote: > problem is in bad width of invisible char 200E I removed this comment in bab982161e since it didn't match the code. I'd be interested to see what happened after v12. - * - Other format characters (general category code Cf in the Unicode - * database) and ZERO WIDTH SPACE (U+200B) have a column width of 0. UnicodeData.txt has this: 200B;ZERO WIDTH SPACE;Cf;0;BN;N; 200C;ZERO WIDTH NON-JOINER;Cf;0;BN;N; 200D;ZERO WIDTH JOINER;Cf;0;BN;N; 200E;LEFT-TO-RIGHT MARK;Cf;0;L;N; 200F;RIGHT-TO-LEFT MARK;Cf;0;R;N; So maybe we need to take Cf characters in this file into account, in addition to Me and Mn (combining characters). -- John Naylor EDB: http://www.enterprisedb.com
broken table formatting in psql
Hi
I found some data that are badly formatted in psql
create table foo(a varchar);
insert into foo values('Dětská šperkovnice Goki 15545');
insert into foo values('Tlakoměr Omron Evolv s Bluetooth připojením');
insert into foo values('Řetěz KMC BE08SEP22 stříbrný');
psql older than 12 shows this table correctly
(2022-09-01 08:42:44) postgres=# select * from foo;
┌─┐
│ a │
╞═╡
│ Dětská šperkovnice Goki 15545 │
│ Tlakoměr Omron Evolv s Bluetooth připojením │
│ Řetěz KMC BE08SEP22 stříbrný│
└─┘
(3 rows)
psql 12 and later breaks border little bit
(2022-09-01 08:42:49) postgres=# select * from foo;
┌─┐
│ a │
╞═╡
│ Dětská šperkovnice Goki 15545 │
│ Tlakoměr Omron Evolv s Bluetooth připojením │
│ Řetěz KMC BE08SEP22 stříbrný │
└─┘
(3 rows)
problem is in bad width of invisible char 200E
https://unicodeplus.com/U+200E
(2022-09-01 09:10:05) postgres=# select e'Ahoj\u200eNazdar';
┌─┐
│ ?column? │
╞═╡
│ AhojNazdar │
└─┘
(1 row)
Regards
Pavel
