Re: [sqlite] Default Encoding In Sqlite

2007-12-16 Thread Daniel Önnerby

Sorry.. meant English is NOT my primary language :)

Daniel Önnerby wrote:

I figure I'll keep it short since it's only for the FAQ.
English is my primary language, but here my suggestion:

Question: Does SQLite handle unicode?

Short answer: Yes!

Answer:
SQLite handles unicode very well. SQLite stores texts in either UTF-16 
or UTF-8 format depending on how the database is created (sqlite3_open 
or sqlite3_open16). SQLite will also seamlessly convert between  the 
different formats depending on how you retrieve the texts 
(sqlite3_column_text or sqlite_column_text16) regardless on what 
format it has been saved as.


There are some cases -like using case insensitive LIKE- where SQLite 
needs to be extended with the ICU extension to fully work with unicode 
strings.





[EMAIL PROTECTED] wrote:

=?ISO-8859-1?Q?Daniel_=D6nnerby?= <[EMAIL PROTECTED]> wrote:
 
Unicode questions seems to come up at least once a week on the 
mailinglist. Maybe there should be something about this in the FAQ 
or the features page?





I will happily accept suggested text for such entries.

--
D. Richard Hipp <[EMAIL PROTECTED]>


- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 



  


- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 





-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Default Encoding In Sqlite

2007-12-15 Thread Daniel Önnerby

I figure I'll keep it short since it's only for the FAQ.
English is my primary language, but here my suggestion:

Question: Does SQLite handle unicode?

Short answer: Yes!

Answer:
SQLite handles unicode very well. SQLite stores texts in either UTF-16 
or UTF-8 format depending on how the database is created (sqlite3_open 
or sqlite3_open16). SQLite will also seamlessly convert between  the 
different formats depending on how you retrieve the texts 
(sqlite3_column_text or sqlite_column_text16) regardless on what format 
it has been saved as.


There are some cases -like using case insensitive LIKE- where SQLite 
needs to be extended with the ICU extension to fully work with unicode 
strings.





[EMAIL PROTECTED] wrote:

=?ISO-8859-1?Q?Daniel_=D6nnerby?= <[EMAIL PROTECTED]> wrote:
  
Unicode questions seems to come up at least once a week on the 
mailinglist. Maybe there should be something about this in the FAQ or 
the features page?





I will happily accept suggested text for such entries.

--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

  


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Default Encoding In Sqlite

2007-12-15 Thread drh
=?ISO-8859-1?Q?Daniel_=D6nnerby?= <[EMAIL PROTECTED]> wrote:
> Unicode questions seems to come up at least once a week on the 
> mailinglist. Maybe there should be something about this in the FAQ or 
> the features page?
> 

I will happily accept suggested text for such entries.

--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Default Encoding In Sqlite

2007-12-15 Thread Daniel Önnerby
Unicode questions seems to come up at least once a week on the 
mailinglist. Maybe there should be something about this in the FAQ or 
the features page?


Trevor Talbot wrote:

I wrote:

  

The default storage encoding on disk is UTF-8, but it can be changed
to UTF-16 with a PRAGMA.



As Igor reminds me, if you create the database file using
sqlite3_open16() the default will be UTF-16 instead. You can still set
it explicitly via a PRAGMA if you wish.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

  


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Default Encoding In Sqlite

2007-12-14 Thread Trevor Talbot
I wrote:

> The default storage encoding on disk is UTF-8, but it can be changed
> to UTF-16 with a PRAGMA.

As Igor reminds me, if you create the database file using
sqlite3_open16() the default will be UTF-16 instead. You can still set
it explicitly via a PRAGMA if you wish.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Default Encoding In Sqlite

2007-12-14 Thread Trevor Talbot
On 12/14/07, Kalyani Phadke <[EMAIL PROTECTED]> wrote:

> Whats default encoding in SQLite3 ? How does SQLite3 handles storing
> Japanese/Chinese text in database?  I know in SQL express/sql server  I
> have to use nVarchar/nchar/ntext datatypes to store Japanese/Chinese
> text in database. It seems that in SQLite3 column having text datatype
> can also store chinese characters..

SQLite assumes TEXT data is Unicode. You can work with it in either
UTF-8 by using the *_text() APIs, or UTF-16 using the *_text16()
calls. It will convert between the two encodings as necessary. The
default storage encoding on disk is UTF-8, but it can be changed to
UTF-16 with a PRAGMA.

Note that I said it "assumes" the data is in that form. SQLite does
not validate the encoding, so it is possible to store text data in
some other encoding, like SJIS. You will just get strange results when
asking SQLite to convert the data, such as when storing it with
*_text(), but retrieving it with *_text16().

The sqlite3 shell is intended to work with UTF-8, but because of the
way different platforms handle the terminal/console encodings, it can
be difficult to use properly.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-