Re: [AOLSERVER] ns_db and multibyte support

[Darren Ferguson]

I had similar issues and in the head component of my adp being generated 
i had to add the following:

<meta http-equiv="content-type" content="text/html; charset=utf-8">

Once that was added the pages were rendered correctly.

Respectfully,
Darren Ferguson

Rajesh nair wrote:
Oops , forgot to attach the adp

-- Rajesh Nair

Rajesh Nair wrote:
Bas,

Apologies for the delayed response!

Our setup is a complex set of components with a java component 
inserting records
and tcl based RESTful service fetching the records back.
I have isolated this issue to a tcl script and am sending this out to 
replicate the issue

1. My MySQL version is 5.1.22
2. create a table
          /*CREATE TABLE multibytetest (value VARCHAR(255)  CHARACTER 
SET utf8)*/
3. Copy the adp in your aolserver installation
   and run it. You will need to make a modification to change the 
database name

The adp present you with a HTML form to let you set a string which is 
inserted into the multibytetest table
using SQL
/*        INSERT INTO multibytetest (value) VALUES (_utf8'$value')*/
The idea to quote it in _utf8 is from this blog from dossy 
<http://dossy.org/archives/000218.html>
I am using the MySQL query browser to see if the  character inserted 
into db is correct or not. The chinese characters gets inserted fine.

The adp also retrieves the same value back and renders it on the HTML 
page along with the tcl string which was inserted.
The Content-Type is set to "text/html; charset=utf-8" as well as adp 
mimetype.

Yet the record fetched back does not get rendered on browser correctly.

Here is a screenshot of a testrun

Screenshot of test run

Any ideas why I am not able to get the multibyte record correctly?

Couple of points :-

1. The records gets rendered fine if the column type is BLOB instead 
of VARCHAR. But I dont think we should need to convert the datatype 
of table in order to store multibyte characters .Java is able to 
insert and fetch the records correctly
2. If the INSERT sql is modifed to simple
/*     INSERT INTO multibytetest (value) VALUES ('$value')*/
both 'Inserted value' and 'Value retrieved from database' is rendered 
correctly but the record inserted in mysql is not correct. This is 
verified by viewing the record in mysql query browser as well as 
fetching it from a java program

Hoping to get any pointers here--
-- Rajesh Nair


Bas Scheffers wrote:
Some questions to help us, and maybe give you some hints as to what  
might be wrong.

How did you determine the fact that the value isn't a correct Tcl  
string?

What encoding is the database in? (UTF-8?) Are you 100% sure the 
data  in the database is actually correct?

I wouldn't think you would need to convert anything. Tcl uses UTF-8  
internally. A string from the database should be nothing more than 
an  array of bytes and when a Tcl string is created, it is assumed 
these  bytes are UTF-8.

Did you specify the correct encoding in either the server headers  
(preferred) or html tags? Unless you do that, the browser won't 
know  what character set it should use.

Regards,
Bas.

On 03/04/2008, at 4:52 AM, Rajesh nair wrote:

Hi all,

We have an existing Tcl service which provides the data from mysql  
to clients as an HTML table.
We have some records in mysql with multibye characters which are 
not  being rendered correctly.

Simple ns_db getrow does not return me the correctly encoded data  
form database.
I have the sample code snippet to replicate this

       set db [ns_db gethandle]

       set sql1 "use mydb";
       ns_db exec $db $sql1;
             set row [ns_db select $db "select title from channel"]
       set numcols [ns_set size $row]
           while {[ns_db getrow $db $row]} {
          for {set i 0} {$i < $numcols} {incr i} {
             ns_puts " :[ns_set value $row $i]"
         }
         ns_puts "<br>"        }

I understand that I need to specify the encoding to get the 
correct  tcl strings in ns_set but cannot find the way to do so. 
Any pointers?

Thanks in advance--
-- Rajesh Nair


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED] > with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave 
the  Subject: field of your email blank.


--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the 
Subject: field of your email blank.



-- AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to 
<[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in 
the email message. You can leave the Subject: field of your email blank.



--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> 
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: 
field of your email blank.