Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-12-02 Thread MauMau
Hi, Tom san, From: Tom Lane t...@sss.pgh.pa.us I've committed this with some editorialization (mostly, I used a case statement not a constant array, because that's more like the other places that switch on errnos in this file). As I said, lack of %m string has been making troubleshooting

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-12-02 Thread Andres Freund
On 2013-12-02 19:36:01 +0900, MauMau wrote: I'll be glad if you could back-port this. Personally, in practice, 9.1 and later will be sufficient. Already happened: Author: Tom Lane t...@sss.pgh.pa.us Branch: REL9_3_STABLE [e3480438e] 2013-11-07 16:33:18 -0500 Branch: REL9_2_STABLE [64f5962fe]

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-11-06 Thread Tom Lane
MauMau maumau...@gmail.com writes: I did as Tom san suggested. Please review the attached patch. I chose as common errnos by selecting those which are used in PosttgreSQL source code out of the error numbers defined in POSIX 2013. I've committed this with some editorialization (mostly, I

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-10 Thread Greg Stark
On Mon, Sep 9, 2013 at 11:27 PM, MauMau maumau...@gmail.com wrote: 1. Recreate the database with LC_CTYPE = ja_JP.UTF-8. This changes various behaviors such as ORDER BY, index scan, and the performance of LIKE clause. This is almost impossible. Wait, why does the ctype of the database affect

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-10 Thread MauMau
From: Greg Stark st...@mit.edu Wait, why does the ctype of the database affect the ctype of the messages? Shouldn't these be two separate things? One describes the character set being used to store data in the database and the other the character set the log file and clients are in. At

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread MauMau
From: MauMau maumau...@gmail.com OK, I'll take this approach. That is: I did as Tom san suggested. Please review the attached patch. I chose as common errnos by selecting those which are used in PosttgreSQL source code out of the error numbers defined in POSIX 2013. As I said, lack of

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Peter Eisentraut
On 9/6/13 10:37 AM, Tom Lane wrote: BTW: personally, I would say that what you're looking at is a glibc bug. I always thought the contract of gettext was to return the ASCII version if it fails to produce a translated version. That might not be what the end user really wants to see, but

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: My suggestion in this matter is to disable gettext processing when LC_CTYPE is set to C. We could log a warning when LC_MESSAGES is set to something and LC_CTYPE is set to C. Or just do the warning and keep logging. Something like that. Meh. Seems

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Peter Eisentraut
On 9/9/13 10:25 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: My suggestion in this matter is to disable gettext processing when LC_CTYPE is set to C. We could log a warning when LC_MESSAGES is set to something and LC_CTYPE is set to C. Or just do the warning and keep

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 9/9/13 10:25 AM, Tom Lane wrote: Meh. Seems that would only prevent one specific instance of the general problem that strerror can fail to translate its result. Other locale combinations might create the same kind of failure. True. There isn't

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Tom Lane
Noah Misch n...@leadboat.com writes: ... I think MauMau's original bind_textdomain_codeset() proposal was on the right track. It might well be. My objection was to the proposal for back-patching it when we have little idea of the possible side-effects. I would be fine with handling that as a

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread MauMau
From: Tom Lane t...@sss.pgh.pa.us Noah Misch n...@leadboat.com writes: ... I think MauMau's original bind_textdomain_codeset() proposal was on the right track. It might well be. My objection was to the proposal for back-patching it when we have little idea of the possible side-effects. I

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread MauMau
From: Peter Eisentraut pete...@gmx.net Does anyone know why the PostgreSQL-supplied part of the error message does not get messed up? That is because bind_textdomain_codeset() is called for postgres.mo in src/backend/utils/mb/mbutils.c, specifying the database encoding as the second

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Peter Eisentraut
On 9/6/13 9:40 AM, MauMau wrote: $ psql -d postgres -c SELECT * FROM a ... This outputs, in Japanese, a message meaning could not open file base/xxx/yyy: ???. The problem is that strerror() returns ???, which hides the cause of the trouble. The cause is that gettext() called by

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread Peter Eisentraut
On 9/9/13 4:42 PM, MauMau wrote: We are using 9.1/9.2 and 9.2 is probably dominant, so I would be relieved with either of the following choices: 1. Take the approach that doesn't use bind_textdomain_codeset(libc) (i.e. the second version of errno_str.patch) for 9.4 and older releases. 2.

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-09 Thread MauMau
From: Peter Eisentraut pete...@gmx.net On 9/9/13 4:42 PM, MauMau wrote: 1. Take the approach that doesn't use bind_textdomain_codeset(libc) (i.e. the second version of errno_str.patch) for 9.4 and older releases. 2. Use bind_textdomain_codeset(libc) (i.e. take strerror_codeset.patch) for 9.4,

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-07 Thread MauMau
Thank you for your opinions and ideas. From: Tom Lane t...@sss.pgh.pa.us Greg Stark st...@mit.edu writes: What would be nicer would be to display the C define, EINVAL, EPERM, etc. Afaik there's no portable way to do that though. I suppose we could just have a small array or hash table of all

[HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread MauMau
Hello, I've been suffering from PostgreSQL's problems related to character encoding for some time. I really wish to solve those problems, because they make troubleshooting difficult. I'm going to propose fixes for them, and I would appreciate if you could help release the official patches

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Tom Lane
MauMau maumau...@gmail.com writes: I've been suffering from PostgreSQL's problems related to character encoding for some time. I really wish to solve those problems, because they make troubleshooting difficult. I'm going to propose fixes for them, and I would appreciate if you could help

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Andres Freund
On 2013-09-06 10:37:16 -0400, Tom Lane wrote: MauMau maumau...@gmail.com writes: I've been suffering from PostgreSQL's problems related to character encoding for some time. I really wish to solve those problems, because they make troubleshooting difficult. I'm going to propose fixes

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I have no clue about the gettext stuff but I am in favor of including the raw errno in strerror() messages (no backpatching tho). I dislike that on grounds of readability and translatability; and I'm also of the opinion that errno codes aren't really

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Andres Freund
On 2013-09-06 10:52:03 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I have no clue about the gettext stuff but I am in favor of including the raw errno in strerror() messages (no backpatching tho). I dislike that on grounds of readability and translatability; and

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Greg Stark
On Fri, Sep 6, 2013 at 3:57 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-09-06 10:52:03 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I have no clue about the gettext stuff but I am in favor of including the raw errno in strerror() messages (no

Re: [HACKERS] [bug fix] strerror() returns ??? in a UTF-8/C database with LC_MESSAGES=non-ASCII

2013-09-06 Thread Tom Lane
Greg Stark st...@mit.edu writes: What would be nicer would be to display the C define, EINVAL, EPERM, etc. Afaik there's no portable way to do that though. I suppose we could just have a small array or hash table of all the errors we know about and look it up. Yeah, I was just thinking the