Re: [sqlite] hex and char functions

2017-08-09 Thread x
017 03:29 To: SQLite mailing list<mailto:sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] hex and char functions On 8 August 2017 at 18:32, x wrote: > Why does it always turn out to be my own fault ☹ > Welcome to programming ;) It's very rare in practice to find an

Re: [sqlite] hex and char functions

2017-08-08 Thread Rowan Worth
On 8 August 2017 at 18:32, x wrote: > Why does it always turn out to be my own fault ☹ > Welcome to programming ;) It's very rare in practice to find an actual bug in a library or language, especially one as widely deployed and tested as sqlite. Glad you're sorted, I just wanted to have a quick

Re: [sqlite] hex and char functions

2017-08-08 Thread x
; Subject: Re: [sqlite] hex and char functions I accidentally deleted the latest 3 replies to this thread in my mail so apologies for not including any content from those mails. Ryan, the error happens immediately in c++ builder. E.g. UnicodeString S=”\u0085”; Int Code=S[1]; Code returns 63 (

Re: [sqlite] hex and char functions

2017-08-08 Thread x
I accidentally deleted the latest 3 replies to this thread in my mail so apologies for not including any content from those mails. Ryan, the error happens immediately in c++ builder. E.g. UnicodeString S=”\u0085”; Int Code=S[1]; Code returns 63 (x3F) which is code for ‘?’ and the question mark

Re: [sqlite] hex and char functions

2017-08-07 Thread Darko Volaric
Actually a maximum of 4 bytes are required to encode a single valid code-point in UTF-8. > On Aug 8, 2017, at 2:44 AM, Jens Alfke wrote: > > >> On Aug 7, 2017, at 8:29 AM, x wrote: >> >> I thought I had learned enough about this string lunacy to get by but >> finding out that the UTF8 code

Re: [sqlite] hex and char functions

2017-08-07 Thread Jens Alfke
> On Aug 7, 2017, at 8:29 AM, x wrote: > > I thought I had learned enough about this string lunacy to get by but finding > out that the UTF8 code for the UTF16 code \u0085 is in fact \uc285 has tipped > me over the edge. I assumed they both used the same codes but UTF16 allowed > some charact

Re: [sqlite] hex and char functions

2017-08-07 Thread Simon Slavin
On 7 Aug 2017, at 8:45pm, R Smith wrote: > If it ends up as something else in that database, follow the call stack > chain, see how those look in the debugger initially (which may be different > to what you expect, depending on the debugger display encoding) Note this point very hard. It is

Re: [sqlite] hex and char functions

2017-08-07 Thread R Smith
On 2017/08/07 9:01 PM, x wrote: Thanks Ryan for going to the trouble of typing that out. Hope you’re not a one fingered typewriter like myself. The borland related stuff is welcome but I still can’t say I’m any less confused by it all. I’m having a bad day today. I’ve spent most of it trying

Re: [sqlite] hex and char functions

2017-08-07 Thread x
:sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] hex and char functions On 2017/08/07 5:29 PM, x wrote: > Apologies, I should have said I was using c++ builder Berlin on windows 10 > and that UnicodeString was UTF16. > > I thought I had learned enough about this string

Re: [sqlite] hex and char functions

2017-08-07 Thread R Smith
On 2017/08/07 5:29 PM, x wrote: Apologies, I should have said I was using c++ builder Berlin on windows 10 and that UnicodeString was UTF16. I thought I had learned enough about this string lunacy to get by but finding out that the UTF8 code for the UTF16 code \u0085 is in fact \uc285 has tip

Re: [sqlite] hex and char functions

2017-08-07 Thread Nico Williams
On Mon, Aug 07, 2017 at 11:45:50AM -0400, Richard Hipp wrote: > On 8/7/17, Nico Williams wrote: > > Internally SQLite3 uses UTF-8. The SQLite3 API lets you deal with > > UTF-16, but this just transcodes to/from UTF-8 internally. > > That is not quite correct. > > SQL statements are always conve

Re: [sqlite] hex and char functions

2017-08-07 Thread Richard Hipp
On 8/7/17, Nico Williams wrote: > > Internally SQLite3 uses UTF-8. The SQLite3 API lets you deal with > UTF-16, but this just transcodes to/from UTF-8 internally. > That is not quite correct. SQL statements are always converted into UTF8 for parsing and code generation. But data can be stored

Re: [sqlite] hex and char functions

2017-08-07 Thread Nico Williams
On Mon, Aug 07, 2017 at 03:29:41PM +, x wrote: > Apologies, I should have said I was using c++ builder Berlin on > windows 10 and that UnicodeString was UTF16. > > I thought I had learned enough about this string lunacy to get by but > finding out that the UTF8 code for the UTF16 code \u0085 i

Re: [sqlite] hex and char functions

2017-08-07 Thread x
TStringList etc are utf16. From: Igor Tandetnik<mailto:i...@tandetnik.org> Sent: 07 August 2017 15:49 To: sqlite-users@mailinglists.sqlite.org<mailto:sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] hex and char functions On 8/7/2017 9:38 AM, x wrote: > Related >

Re: [sqlite] hex and char functions

2017-08-07 Thread Igor Tandetnik
On 8/7/2017 9:38 AM, x wrote: Related Select hex(char(65,133,66)); returns ‘41C28542’ whereas I expected ‘418542’. What is the ‘C2’ about? Two-byte sequence C2 85 is the UTF-8 encoding of the Unicode codepoint U+0085. -- Igor Tandetnik ___ sqlite-u

Re: [sqlite] hex and char functions

2017-08-07 Thread Hick Gunter
The sqlite char() function returns unicode. Apparently, the encoding for code point 133 is two characters, namely c2 85. You seem to be expecting char() to return ISO characters, which it does not do. Calling sqlite3_value_text16 instructs SQLite to convert the contents of the field into utf16

Re: [sqlite] hex and char functions

2017-08-07 Thread Igor Tandetnik
On 8/7/2017 9:38 AM, x wrote: In c++ I tried to call a sqlite udf using the following sql UnicodeString SQL=“select udf(‘5\u00856’);” You are using a narrow string literal to try and hold a Unicode character. You are at the whim of your compiler as to how it represents the latter in the form