Re: Using "textwrap" package for unwrappable languages (Japanese)
On 2023-08-30 13:18:25 +, c.buhtz--- via Python-list wrote: > Am 30.08.2023 14:07 schrieb Peter J. Holzer via Python-list: > > another caveat: Japanese characters are usually double-width. So > > (unless your line length is 130 characters for English) you would > > want to add that line break every 32 characters. > > I don't get your calculation here. Original line length is 130 but for > "double-with" characters you would break at 32 instead of 65 ? No, I wrote "*unless* your original line length was 130 characters". I assumed that you want your line to be 65 latin characters wide since this is what fits nicely on an A4 (or letter) page with a bit of a margin on both sides. Or on an 80 character terminal screen or window. And it's also generally considered to be a good line length for readability. But Asian "full width" or "wide" characters are twice as wide, so you can fit only half as many in a single line. Hence 65 // 2 = 32. But that was only my assumption. I considered it possible that you started with 130 characters per line (many terminals back in the day had a 132 character mode, and that's also approximately the line length in landscape mode or when using a compressed typeface - so 132 is also a common length limit, although rarely for text (too wide to read comfortably) and more for code, tables, etc.), divided that by two and arrived at 65 Japanese characters per line that way. So I mentioned that to indicate that I had considered the possibility but concluded that it probably wasn't what you meant. (And as usual when I write a short sentence to clarify something I wind up writing 4 paragraphs clarifying the clarification :-/) > Then I will do something like this > > unicodedata.east_asian_width(mystring[0]) > > W is "wide". But there is also "F" (full-width). > What is the difference between "wide" and "full-width"? I'm not an expert on Japanese typography by any means. But they have some full width variants of latin characters and halfwidth variants of katakana characters. I assume that the categories 'F' and 'H' are for those, while "normal" Japanese characters are "W": >>> unicodedata.east_asian_width("\N{DIGIT ONE}") 'Na' >>> unicodedata.east_asian_width("\N{FULLWIDTH DIGIT ONE}") 'F' >>> unicodedata.east_asian_width("\N{KATAKANA LETTER ME}") 'W' >>> unicodedata.east_asian_width("\N{HALFWIDTH KATAKANA LETTER ME}") 'H' hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Using "textwrap" package for unwrappable languages (Japanese)
Dear Peter, thanks for your reply. That is a very interesting topic. I was a bit wrong. I realized that textwrap.wrap() do insert linebreaks when "words" are to long. So even a string without any blank space well get wrapped. Am 30.08.2023 14:07 schrieb Peter J. Holzer via Python-list: another caveat: Japanese characters are usually double-width. So (unless your line length is 130 characters for English) you would want to add that line break every 32 characters. I don't get your calculation here. Original line length is 130 but for "double-with" characters you would break at 32 instead of 65 ? ( Then I will do something like this unicodedata.east_asian_width(mystring[0]) W is "wide". But there is also "F" (full-width). What is the difference between "wide" and "full-width"? My application do support (currently 46) languages including Simplified and Traditional Chinese, Vietnamese, Korean, Japanese, Cyrylic. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using "textwrap" package for unwrappable languages (Japanese)
On 2023-08-30 11:32:02 +, c.buhtz--- via Python-list wrote: > I do use "textwrap" package to wrap longer texts passages. Works well with > English. > But the source string used is translated via gettext before it is wrapped. > > Using languages like Japanese or Chinese would IMHO result in unwrapped > text. Japanese rules do allow to break a line nearly where ever you want. > > How can I handle it with "textwrap"? > > At runtime I don't know which language is really used. So I'm not able to > decide using "textwrap" or just inserting "\n" every 65 characters. I don't have a solution but want to add another caveat: Japanese characters are usually double-width. So (unless your line length is 130 characters for English) you would want to add that line break every 32 characters. (unicodedata.east_asian_width() seems to be the canonical name to find the width of a character, but it returns a code (like 'W' or 'Na') not a number.) hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Using "textwrap" package for unwrappable languages (Japanese)
Hi, I do use "textwrap" package to wrap longer texts passages. Works well with English. But the source string used is translated via gettext before it is wrapped. Using languages like Japanese or Chinese would IMHO result in unwrapped text. Japanese rules do allow to break a line nearly where ever you want. How can I handle it with "textwrap"? At runtime I don't know which language is really used. So I'm not able to decide using "textwrap" or just inserting "\n" every 65 characters. Another approach would be to let the translators handle the line breaks. But I would like to avoid it because some of them don't know what "\n" means and they don't know the length rule (in my case 65 characters). Any ideas about it? Kind Christian -- https://mail.python.org/mailman/listinfo/python-list