Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 10, 2011, at 11:43 PM, Alex Hunsaker wrote: I'd like to quibble with you over this point if I may. :-) Per perldoc: JSON::XS utf8 flag disabled When utf8 is disabled (the default), then encode/decode generate and expect Unicode strings ... So - If you are on 9.1 and a

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Andrew Dunstan
On 02/11/2011 12:16 PM, David E. Wheeler wrote: [long discussion elided] Is there an action item here? Is the documentation inadequate or inaccurate? If so, please make a suggestion for improved wording. I certainly expect the change to be covered in the release notes. You can check on

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 10:16, David E. Wheeler da...@kineticode.com wrote: On Feb 10, 2011, at 11:43 PM, Alex Hunsaker wrote: Like I said, the terminology is awful. Yeah I use encode and decode to mean the same thing frequently :-(. In the the cited case he was passing %C3%A9 to

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 10:44, Alex Hunsaker bada...@gmail.com wrote: On Fri, Feb 11, 2011 at 10:16, David E. Wheeler da...@kineticode.com wrote: That *looks* like it is decoding the input string, which it is, but actually that will double utf8 encode your string. It does not seem to in this

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 9:44 AM, Alex Hunsaker wrote: It is decoded... the input string %C3%A9 actually is the _same_ string utf-8, latin1 and SQL_ASCII decoded or not. Those are all ascii characters. Calling utf8::decode(%C3%A9) is essentially a noop. No, it's not decoded. It doesn't matter

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 9:34 AM, Andrew Dunstan wrote: Is there an action item here? Is the documentation inadequate or inaccurate? If so, please make a suggestion for improved wording. I certainly expect the change to be covered in the release notes. You can check on the prominence given the

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 10:01 AM, Alex Hunsaker wrote: That *looks* like it is decoding the input string, which it is, but actually that will double utf8 encode your string. It does not seem to in this case because we are dealing with all ascii input. The trick here is its also telling perl to

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 11:07, David E. Wheeler da...@kineticode.com wrote: I don't understand where the bug is. If a string is encoded in utf-8 Perl will not treat it as such unless the utf-8 flag is set. Ok so I think we agreed this is right: $ perl -E 'use URI::Escape; my $str =

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 12:57 PM, Alex Hunsaker wrote: Yay! 1 Yes, this is all good. But it still seems to me that: * If your database is neither utf-8 no sql_ascii * And your PL/Perl functions expect arguments that are byte soup * Once you upgrade to 9.1 they won't be * So you'll need to encode

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 16:42, David E. Wheeler da...@kineticode.com wrote: On Feb 11, 2011, at 12:57 PM, Alex Hunsaker wrote: Yay! 1 Yes, this is all good. But it still seems to me that: * If your database is neither utf-8 no sql_ascii You mean... we have been talking past each other this

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 3:58 PM, Alex Hunsaker wrote: You mean... we have been talking past each other this whole time? Well, since my second post, I think. I was wrong in the first one. Olegs case _was_ a utf8 database. From his original bug: Hi there, below is the problem, which I don't

[HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread David E. Wheeler
Hackers, With regard to this (very welcome) commit: commit 50d89d422f9c68a52a6964e5468e8eb4f90b1d95 Author: Andrew Dunstan and...@dunslane.net Date: Sun Feb 6 17:29:26 2011 -0500 Force strings passed to and from plperl to be in UTF8 encoding. String are converted to UTF8

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread Alex Hunsaker
On Thu, Feb 10, 2011 at 16:28, David E. Wheeler da...@kineticode.com wrote: Hackers, With regard to this (very welcome) commit: commit 50d89d422f9c68a52a6964e5468e8eb4f90b1d95 Author: Andrew Dunstan and...@dunslane.net Date:   Sun Feb 6 17:29:26 2011 -0500     Force strings passed to and

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 5:28 PM, Alex Hunsaker wrote: Hrm? For UTF-8 databases, in practice, nothing should have changed-- we already passed strings in as utf8. What I fixed was some corner cases where some strings did not always have character semantics. See The Unicode Bug and Forcing Unicode

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread Alex Hunsaker
On Thu, Feb 10, 2011 at 21:53, David E. Wheeler da...@kineticode.com wrote: On Feb 10, 2011, at 5:28 PM, Alex Hunsaker wrote: The other thing that changed is non UTF-8 databases now also get character semantics. That is we convert from the database encoding into utf8 and visa versa on output.