On 09/11/2011 15:49, H.Merijn Brand wrote:
On Tue, 08 Nov 2011 21:12:13 +0000, "Martin J. Evans"
<martin.ev...@easysoft.com> wrote:
I've just checked in unicode_test.pl to DBI's subversion trunk in /ex dir.
It won't run right now without changing the do_connect sub as you have
to specify how to connect to the DB.
Also, there is a DBD specific section at the start where you might have
to add a DBD it does not know about (anything other than DBD::ODBC,
DBD::Oracle, DBD::SQLite, DBD::CSV, DBD::mysql) if it needs something
other than the defaults e.g., the name of the length function in SQL,
the column type for unicode columns and binary columns, the setting to
enable UTF8/Unicode support. It could be a bit of a pain if your DBD
does not support type_info_all but I'm around on irc and in this list if
anyone wants any help making it work.
It needs rather a lot of tidying up so I'm not putting it forward as
code-of-the-year but it is a start.
BTW, you'll need Test::More::UTF8 and perhaps a couple of other non core
modules to run it.
Martin
I'll have some deeper look at both Unify and CSV ...
Attached is a revised version of the script (first argument is the
driver to test, some need more work)
$ perl /tmp/unicode_test.pl Unify
# Driver DBD::Unify-0.78
# Using DBMS_NAME 'Unify DataServer'
# Using DBMS_VER undef
# Using DRIVER_NAME '/pro/asql/v83I/lib/perl/5.10.1/DBD/Unify.pm'
# Using DRIVER_VER '00.78.0000'
# SQL_IDENTIFIER_CASE 3
# LANGDIR = dutch
print() on closed filehandle $fh at /tmp/unicode_test.pl line 438.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
# Found type (HUGE AMOUNT) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
# Found type (AMOUNT) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
# Found type (BINARY) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
Use of uninitialized value in concatenation (.) or string at
/tmp/unicode_test.pl line 421.
:
:
# Found type (TIME) size=
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 422.
DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for Statement
"create table fredĀ ( a int)"] at /tmp/unicode_test.pl line 214.
not ok 1 - unicode table name supported
# Failed test 'unicode table name supported'
# at /tmp/unicode_test.pl line 216.
# died: DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for
Statement "create table fredÄ ( a int)"] at /tmp/unicode_test.pl line 214.
ok 2 # skip Failed to create unicode table name
ok 3 # skip Failed to create unicode table name
DBD::Unify::db prepare failed: Syntax error in SQL dynamic statement. [for Statement
"create table fred ( daveĀ int)"] at /tmp/unicode_test.pl line 214.
not ok 4 - unicode column name supported
Your going to have a lot of problems with this test code and DBD::Unify
as we previously discovered that DBD::Unify does not decode the data
coming back from the database itself but it can be decoded by any Perl
script using DBD::Unify into the correct data. Any chance you could
change the test code to print out the results of type_info_all for
DBD::Unify and send me them?
I'd like to fix all those Use of unitialized value warnings so I'll look
into that. The test code has changed so the line numbers are a problem
so I'll have to compare them with your attachment. I'm guessing they are
all in the processing of the results from type_info_all. To be honest,
this is not a unicode/encoding issue but one of my biggest annoyances
when working with DBDs and that is finding the right column type for a
table is really hit and miss - many DBDs don't support type_info_all and
some that do only provide a subset of the columns DBI defines.
Somewhere (I've lost it right now but I'll find it later) I have a
document which details my experiences of trying to write DBD independent
code which worked with DBD::mysql, DBD::DB2 and DBD::Oracle. In the end
we gave up - it was just too difficult. It wasn't just the different SQL
syntax/support. We ended up putting all the logic into
procedures/functions and simply calling them from Perl which avoided SQL
syntax issues and loads of other differences. Perhaps I'll go back to
that if I find some time.
$ perl /tmp/unicode_test.pl
# Driver DBD::SQLite-1.33
# Using DBMS_NAME 'SQLite'
# Using DBMS_VER '3.7.6.3'
# Using DRIVER_NAME undef
# Using DRIVER_VER undef
# SQL_IDENTIFIER_CASE undef
# LANGDIR = dutch
print() on closed filehandle $fh at /tmp/unicode_test.pl line 438.
ok 1 - unicode table name supported
ok 2 - unicode table found in unqualified table_info
ok 3 - unicode table found by qualified table_info
ok 4 - unicode column name supported
ok 5 - unicode column found in unqualified column_info
ok 6 - unicode column found by qualified column_info
ok 7 - table for unicode data
ok 8 - insert unicode data into table
ok 9 - unicode data out = unicode data in, no where
ok 10 - length of output data the same
ok 11 - db length of unicode data correct
ok 12 - select unicode data via parameterised where
ok 13 - select unicode data via inline where
ok 14 - table for unicode data
ok 15 - insert unicode data and blob into table
ok 16 - unicode data out = unicode data in, no where with blob
ok 17 - utf8 flag not set on blob data
Use of uninitialized value in string eq at /tmp/unicode_test.pl line 373.
ok 18 - retrieved blob = inserted blob
ok 19 - test table for unicode parameter markers
DBD::SQLite::st bind_param failed: Unknown named parameter: fred⬠[for Statement
"insert into fred (a) values (:fred€)"] at /tmp/unicode_test.pl line 390.
not ok 20 - bind parameter with unicode parameter marker
# Failed test 'bind parameter with unicode parameter marker'
# at /tmp/unicode_test.pl line 392.
# died: DBD::SQLite::st bind_param failed: Unknown named parameter: fred⬠[for
Statement "insert into fred (a) values (:fredâ¬)"] at /tmp/unicode_test.pl line
390.
1..20
# Looks like you failed 1 test of 20.
Martin
