Re: Unicode and Sybase univarchar

2010-06-07 Thread Martin Evans
Alexander Foken wrote: On 04.06.2010 15:41, Dave Rolsky wrote: On Fri, 4 Jun 2010, Alexander Foken wrote: That's why I proposed to switch to DBD::ODBC: It is well tested and supports Unicode as good as the ODBC driver does. And as I said in private email, that's not really feasible for us.

Re: Unicode and Sybase univarchar

2010-06-05 Thread Alexander Foken
On 04.06.2010 15:41, Dave Rolsky wrote: On Fri, 4 Jun 2010, Alexander Foken wrote: That's why I proposed to switch to DBD::ODBC: It is well tested and supports Unicode as good as the ODBC driver does. And as I said in private email, that's not really feasible for us. Yes, it would cause you a

Re: Unicode and Sybase univarchar

2010-06-04 Thread Alexander Foken
... should just work. It doesn't quite, because the hex string is not just a dump of a 16 Bit Unicode encoding, but it is a UTF-8 byte stream written with a 16 Bit Hex Format for each byte. Each and every 16-Bit-Word has its most significant byte set to 0. If it was a dump of a 16 Bit

Re: Unicode and Sybase univarchar

2010-06-04 Thread Dave Rolsky
On Fri, 4 Jun 2010, Alexander Foken wrote: Right. (But remember that DBI was there before Unicode support was added to Perl, and also most DBDs are older that the Unicode support. Before Unicode was there, you just passed bytes around and everything just worked.) Only if by everything you

Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
I'm working on an i18n project, and we use Sybase (sigh). Newer versions of Sybase have built-in support for Unicode with the univarchar (and other uni*) type. However, it seems like DBD::Sybase doesn't have any support for this. Specifically, if I take a Perl unicode string (utf8 flag is

Re: Unicode and Sybase univarchar

2010-06-03 Thread Michael Peppler
Hi, Which version of Sybase, which version of Sybase OpenClient, and which version of DBD::Sybase? Are you setting the connection charset to utf8 (in the connect() call?) Thanks, Michael On Jun 3, 2010, at 5:22 PM, Dave Rolsky wrote: I'm working on an i18n project, and we use Sybase

Re: Unicode and Sybase univarchar

2010-06-03 Thread Michael Peppler
On Jun 3, 2010, at 7:29 PM, Michael Peppler wrote: Hi, Which version of Sybase, which version of Sybase OpenClient, and which version of DBD::Sybase? Are you setting the connection charset to utf8 (in the connect() call?) I just gave this a try - I'm under linux, with ASE 15.5. I

Re: Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
On Thu, 3 Jun 2010, Michael Peppler wrote: I just gave this a try - I'm under linux, with ASE 15.5. I created a table with a univarchar column, entered some data via isql, then wrote a minimal perl script to fetch the data. If I use a UTF8 locale (i.e. LANG=en_us.UTF8) I get the correct

Re: Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
On Thu, 3 Jun 2010, Michael Peppler wrote: Which version of Sybase, which version of Sybase OpenClient, and which version of DBD::Sybase? Ah, I was using the old libs (11.0), which may have been the problem. I was also using DBD::Sybase 1.07. I switch to Sybase 15.0 (OCS 15.0 if that

Re: Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
On Thu, 3 Jun 2010, Dave Rolsky wrote: If I _don't_ set that, the data goes in and comes out as bytes, rather than the bizarro hex string. However, the data does have the utf8 flag set when it comes back from Sybase, so I have to run it through Encode::decode. Doh, the data _does not_ have

Re: Unicode and Sybase univarchar

2010-06-03 Thread Michael Peppler
On Jun 3, 2010, at 8:14 PM, Dave Rolsky wrote: On Thu, 3 Jun 2010, Michael Peppler wrote: Which version of Sybase, which version of Sybase OpenClient, and which version of DBD::Sybase? Ah, I was using the old libs (11.0), which may have been the problem. I was also using DBD::Sybase

Re: Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
On Thu, 3 Jun 2010, Michael Peppler wrote: 2010/06/03 14:08:11 unicode CRITICAL: FATAL: DBD::Sybase::db do failed: Server message number=2402 severity=16 state=1 line=1 server=HDATADEV1 text=Error converting characters into server's character set. Some character(s) could not be converted.

Re: Unicode and Sybase univarchar

2010-06-03 Thread Alexander Foken
Sorry, forgot to CC to the list ... Original Message Subject:Re: Unicode and Sybase univarchar Date: Thu, 03 Jun 2010 22:20:37 +0200 From: Alexander Foken alexan...@foken.de To: Dave Rolsky auta...@urth.org References: alpine.deb.0..1006031018000.9

Re: Unicode and Sybase univarchar

2010-06-03 Thread Dave Rolsky
On Thu, 3 Jun 2010, Alexander Foken wrote: Really strange way to avoid pack()/unpack(). At least, you can get rid of the Yes, yes, this is hack code, not production. ... should just work. It doesn't quite, because the hex string is not just a dump of a 16 Bit Unicode encoding, but it is a