Re: Truncation on single byte to UTF-8 conversion (was: Re: ANNOUNCE: DBD::Oracle 1.17 release candidate)

[Tim Bunce]

On Thu, Dec 29, 2005 at 11:56:27AM +0100, Honza Pazdziora wrote:
> On Wed, Dec 28, 2005 at 11:11:31PM +0000, Tim Bunce wrote:
> >   http://www.data-plan.com/public/DBD-Oracle-1.17.tar.gz
> > 
> > =head1 Changes in DBD-Oracle 1.17 (svn rev 2334)
> 
> Hello Tim,
> 
> it seems like the problem that I reported in
> 
>       
> http://groups.google.com/group/perl.dbi.users/browse_frm/thread/27696a644a85a0a3/fdb8ad2e1fa8f8bf
> 
> is still present in 1.17. The current code in oci8.c is
> 
>         case  96:                               /* CHAR         */
>                 fbh->disize = fbh->dbsize;
>                 if (CS_IS_UTF8(fbh->csid))
>                     fbh->disize = fbh->dbsize * 4;
>                 fbh->prec   = fbh->disize;
>                 break;
> 
> but on a database with character set EE8ISO8859P2 (single byte) and
> with NLS_LANG=AMERICAN_AMERICA.UTF8, the
> 
>       ORA-24345: A Truncation or null fetch error occurred (DBD ERROR: 
> ORA-01406 error on field 1 of 98, ora_type 1)
> 
> is still there. If I remove the
> 
>       if (CS_IS_UTF8(fbh->csid))
> 
> line and just multiply the disize by four for all CHARs, the fetch
> works.
> 
> If I understand that CS_IS_UTF8 macro correctly, it compares fbh->csid
> to utf8_csid or to al32utf8_csid. However, for my column (database
> in EE8ISO8859P2, NLS_LANG=AMERICAN_AMERICA.UTF8), the trace shows
> 
>     col  1: dbtype 1, scale 0, prec 80, nullok 0, name ID_FIR
>           : dbsize 20, char_used 0, char_size 20, csid 32, csform 1, disize 80
> 
> where csid 32 seems to be the id of (server's) EE8ISO8859P2, not
> of client's UTF8. So the CS_IS_UTF8(fbh->csid) evaluates to false,
> the buffer is not extended and fetching data from single byte database
> to UTF8 client fails.
> 
> Specifying
> 
>       ora_csid => 871
> 
> attribute on connect does not help either.
> 
> Any idea what should I try next to make CS_IS_UTF8 (and fbh->csid)
> behave like it is supposed to?

Try the appended patch. Should work for NCHAR as well.

Thanks Honza.

Tim.

Index: oci8.c
===================================================================
--- oci8.c      (revision 2342)
+++ oci8.c      (working copy)
@@ -1223,7 +1223,7 @@
                /* FALLTHRU */
        case  96:                               /* CHAR         */
                fbh->disize = fbh->dbsize;
-               if (CS_IS_UTF8(fbh->csid)) 
+               if (CSFORM_IMPLIES_UTF8(fbh->csform))
                    fbh->disize = fbh->dbsize * 4;
                fbh->prec   = fbh->disize;
                break;
@@ -1249,7 +1249,7 @@
                break;
 
        case   8:                               /* LONG         */
-                if (CS_IS_UTF8(fbh->csid)) 
+               if (CSFORM_IMPLIES_UTF8(fbh->csform))
                     fbh->disize = long_readlen * 4;
                 else
                     fbh->disize = long_readlen;