[issue12568] Add functions to get the width in columns of a character

STINNER Victor added the comment: I close the issue as WONTFIX. -- resolution: -> wont fix stage: test needed -> resolved status: open -> closed ___ Python tracker ___

[issue12568] Add functions to get the width in columns of a character

Terry J. Reedy added the comment: I suggest reclosing this issue, for the same reason I suggested closure of #24665 in msg321291: abstract unicode 'characters' (graphemes) do not, in general, have fixed physical widths of 0, 1, or 2 n-pixel columns (or spaces). I based that fairly long

[issue12568] Add functions to get the width in columns of a character

Change by Bian Jiaping : -- nosy: +bianjp ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue12568] Add functions to get the width in columns of a character

Change by Robert Booth : -- nosy: +ishigoya ___ Python tracker ___ ___ Python-bugs-list

[issue12568] Add functions to get the width in columns of a character

Changes by Socob <206a8...@opayq.com>: -- nosy: +Socob ___ Python tracker ___ ___ Python-bugs-list mailing

[issue12568] Add functions to get the width in columns of a character

Guillaume Sanchez added the comment: Hello, I come from bugs.python.org/issue30717 . I have a pending PR that needs review ( https://github.com/python/cpython/pull/2673 ) adding a function that breaks unicode strings into grapheme clusters (aka what one would intuitively call "a character").

[issue12568] Add functions to get the width in columns of a character

STINNER Victor added the comment: You need users who use CJK and understand locale issues especially the width of characters. Ask maybe Xiang Zhang and Naoki INADA? -- ___ Python tracker

[issue12568] Add functions to get the width in columns of a character

STINNER Victor added the comment: > At least two other issues depend on this: issue17048 and issue24665. I removed the dependency from bpo-24665 (CJK support for textwrap) to this issue, since its current PR uses unicodedata.east_asian_width(), not the C function wcswidth(). --

[issue12568] Add functions to get the width in columns of a character

Serhiy Storchaka added the comment: At least two other issues depend on this: issue17048 and issue24665. If Victor lost interest in this issue I take it. I'm going to push at least imperfect solution which may be improved in time. -- resolution: rejected -> stage: resolved ->

[issue12568] Add functions to get the width in columns of a character

R. David Murray added the comment: Interestingly, this just came up again in issue 30717. -- nosy: +r.david.murray ___ Python tracker ___

[issue12568] Add functions to get the width in columns of a character

STINNER Victor added the comment: Since we failed to agree on this feature, I close the issue. -- resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue12568] Add functions to get the width in columns of a character

Serhiy Storchaka added the comment: I think this function would be very useful in many parts of interpreter core and standard library. From displaying tracebacks to formatting helps. Otherwise we are doomed to implement imperfect variants in multiple places. -- resolution: out of date

[issue12568] Add functions to get the width in columns of a character

STINNER Victor added the comment: Since no consensus was found on the definition of the function, and this issue has no activity since 2 years, I close the issue as out of date. -- resolution: - out of date status: open - closed ___ Python tracker

[issue12568] Add functions to get the width in columns of a character

Changes by Terry J. Reedy tjre...@udel.edu: -- stage: - patch review type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___

[issue12568] Add functions to get the width in columns of a character

Terry J. Reedy added the comment: In this part of width.py, w = unicodedata.east_asian_width(c) if c == 'A': # ambiguous raise ValueError(ambiguous character %x % (ord(c))) I presume that 'c' should be 'w'. -- nosy: +terry.reedy

[issue12568] Add functions to get the width in columns of a character

STINNER Victor victor.stin...@gmail.com added the comment: Martin: I agree that there are going to be cases where it is not correct because the terminal does something strange, but what we need is something that gets as close as possible to what the terminal is likely to be doing Can't we

[issue12568] Add functions to get the width in columns of a character

Serhiy Storchaka storch...@gmail.com added the comment: Has anyone tested wcswidth on FreeBSD, old Solaris? With non-utf8 locales? -- nosy: +storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568

[issue12568] Add functions to get the width in columns of a character

Éric Araujo mer...@netwok.org added the comment: eo -- nosy: +benjamin.peterson, eric.araujo, lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___

[issue12568] Add functions to get the width in columns of a character

Changes by Éric Araujo mer...@netwok.org: -- Removed message: http://bugs.python.org/msg156059 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___

[issue12568] Add functions to get the width in columns of a character

poq p...@gmx.com added the comment: Martin, I agree that wcswidth is incorrect with respect to Unicode. However I don't think that's relevant at all. Python should only try to match the behaviour of the terminal. Since terminals do slightly different things, trying to match them exactly - in

[issue12568] Add functions to get the width in columns of a character

Nicholas Cole nicholas.c...@gmail.com added the comment: Poq: I agree. Guessing from the Unicode standard is going to lead to users having to write some complicated code that people are going have to reinvent over and over, and is not going to be accurate with respect to curses. I'd favour

[issue12568] Add functions to get the width in columns of a character

Nicholas Cole nicholas.c...@gmail.com added the comment: Martin: sorry to be completely dense, but I can't get this to work properly with the python3.3a1 build. Could you post some example code? -- ___ Python tracker rep...@bugs.python.org

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: Please see the attached width.py for an example -- Added file: http://bugs.python.org/file24773/width.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568

[issue12568] Add functions to get the width in columns of a character

poq p...@gmx.com added the comment: Martin, I think you meant to write if w == 'A':. Some very common characters have ambiguous widths though (e.g. the Greek alphabet), so you can't just raise an error for them. http://unicode.org/reports/tr11/ says: Ambiguous characters occur in East Asian

[issue12568] Add functions to get the width in columns of a character

Tom Christiansen tchr...@perl.com added the comment: I would encourage you to look at the Perl CPAN module Unicode::LineBreak, which fully implements tr11. It includes Unicode::GCString, a class that has a columns() method to determine the print columns. This is very fancy in the case of Asian

[issue12568] Add functions to get the width in columns of a character

Nicholas Cole nicholas.c...@gmail.com added the comment: Marting and Poq: I think the sample code shows up a real problem. Ambiguous characters according to unicode may be rendered by curses in different ways. Don't we need a function that actually reports how curses is going to print a given

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: Martin, I think you meant to write if w == 'A':. Some very common characters have ambiguous widths though (e.g. the Greek alphabet), so you can't just raise an error for them. That's precisely why I don't think this should be in the

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: I would encourage you to look at the Perl CPAN module Unicode::LineBreak, which fully implements tr11. Thanks for the pointer! If you'd like, I can show you a program that uses these, a rewrite the standard Unix fmt(1) filter that works

[issue12568] Add functions to get the width in columns of a character

Tom Christiansen tchr...@perl.com added the comment: Martin v. L=C3=B6wis mar...@v.loewis.de added the comment: Martin, I think you meant to write if w =3D=3D 'A':. Some very common characters have ambiguous widths though (e.g. the Greek = alphabet), so you can't just raise an error for them.

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: Don't we need a function that actually reports how curses is going to print a given string, rather than just reporting what the unicode standard says? That may be useful, but a) this patch doesn't provide that, and b) it may not actually

[issue12568] Add functions to get the width in columns of a character

Changes by Antoine Pitrou pit...@free.fr: -- resolution: works for me - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___

[issue12568] Add functions to get the width in columns of a character

Tom Christiansen tchr...@perl.com added the comment: Martin v. L=C3=B6wis mar...@v.loewis.de added the comment: I would encourage you to look at the Perl CPAN module Unicode::LineBreak, which fully implements tr11. Thanks for the pointer! If you'd like, I can show you a program that uses

[issue12568] Add functions to get the width in columns of a character

poq p...@gmx.com added the comment: It seems this is a bit of a minefield... GNOME Terminal/libvte has an environment variable (VTE_CJK_WIDTH) to override the handling of ambiguous width characters. It bases its default on the locale (with the comment 'This is basically what GNU libc does').

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: Tom: I don't think Unicode::GCString implements UAX#11 correctly (but this is really out of scope of this issue). In particular, it contains an ad-hoc decision to introduce the EA_Z east-asian width that UAX#11 doesn't talk about. In most

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: poq: I still remain opposed to exposing wcswidth, since it is just as incorrect as any of the other solutions that people have circulated. I could agree to it if it was called wcswidth, making it clear that it does whatever the C library

[issue12568] Add functions to get the width in columns of a character

Nicholas Cole nicholas.c...@gmail.com added the comment: Could we have an update on the status of this? I ask because if 3.3 is going to (finally) fix unicode for curses, it would be really nice if it were possible to calculate the width of what's being displayed! It looks as if there was

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: Nicholas: I consider this issue fixed. There already *is* any API to compute the width of a character. Closing this as works for me. -- resolution: - works for me status: open - closed ___

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: I'm -1 on using wcswidth, though. When you write text into a console on Linux (e.g. displayed by gnome-terminal or konsole), I suppose that wcswidth() can be used to compute the width of a line. It would help to fix #2382. Or do you

[issue12568] Add functions to get the width in columns of a character

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___

[issue12568] Add functions to get the width in columns of a character

STINNER Victor victor.stin...@haypocalc.com added the comment: I'm -1 on using wcswidth, though. When you write text into a console on Linux (e.g. displayed by gnome-terminal or konsole), I suppose that wcswidth() can be used to compute the width of a line. It would help to fix #2382. Or do

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: I think the WideCharToMultibyte approach is just incorrect. I'm -1 on using wcswidth, though. We already have unicodedata.east_asian_width, which implements http://unicode.org/reports/tr11/ The outcomes of this function are these: - F:

[issue12568] Add functions to get the width in columns of a character

Tom Christiansen tchr...@perl.com added the comment: Martin v. Löwis mar...@v.loewis.de added the comment: I think the WideCharToMultibyte approach is just incorrect. I'm -1 on using wcswidth, though. Like you, I too seriously question using wcswidth() for this at all: The wcswidth()

[issue12568] Add functions to get the width in columns of a character

STINNER Victor victor.stin...@haypocalc.com added the comment: There might be something you can steal from ... I don't think that Python should reinvent the wheel. We should just reuse wcswidth(). Here is a simple patch exposing wcswidth() function as locale.width(). Example: import

[issue12568] Add functions to get the width in columns of a character

STINNER Victor victor.stin...@haypocalc.com added the comment: Oh, unicode_width.patch of issue #2382 implements the width on Windows using: WideCharToMultiByte(CP_ACP, 0, buf, len, NULL, 0, NULL, NULL); It computes the length of byte string encoded to the ANSI code page. I don't know if it

[issue12568] Add functions to get the width in columns of a character

Tom Christiansen tchr...@perl.com added the comment: I can attest that being able to get the columns of a grapheme cluster is very important for printing, because you need this to do correct linebreaking. There might be something you can steal from

[issue12568] Add functions to get the width in columns of a character

Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___ ___

[issue12568] Add functions to get the width in columns of a character

Martin v. Löwis mar...@v.loewis.de added the comment: In the #2382 code, how is the Windows case supposed to work? Also, what about systems that don't have wcswidth? IOW, the patch appears to be incorrect. I like the #6755 approach better, except that it shouldn't be using hard-coded tables,

[issue12568] Add functions to get the width in columns of a character

Changes by Nicholas Cole nicholas.c...@gmail.com: -- nosy: +Nicholas.Cole ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___ ___

[issue12568] Add functions to get the width in columns of a character

Changes by Christian Hofstaedtler ch+pythonb...@zeha.at: -- nosy: +zeha ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12568 ___ ___

[issue12568] Add functions to get the width in columns of a character

New submission from STINNER Victor victor.stin...@haypocalc.com: Some characters take more than one column in a terminal, especially CJK (chinese, japanese, korean) characters. If you use such character in a terminal without taking care of the width in columns of each character, the text