Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-12 Thread Serhiy Storchaka
10.11.19 19:54, Richard Hipp пише: On 11/8/19, Serhiy Storchaka wrote: JSON_EXTRACT [1] produces gibberish for encoded non-BMP characters. Thanks for the bug report and test case. I checked in candidate fix here: https://www.sqlite.org/src/timeline?c=51027f08c0478f1b I need to do additional

Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Andreas Kupries
> On 11/10/19, Richard Hipp wrote: > > It seems there are also problems in generating valid JSON for large > > unicode characters. For example: > > > > SELECT json_quote(char(1114111)); > > > > I'm working on the problem > I take that back. It seems that the spec allow arbitrary UTF-8

Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Richard Hipp
I now believe that check-in https://www.sqlite.org/src/timeline?c=51027f08c0478f1b is the complete and correct fix for the reported problem with UTF16 surrogate pairs in JSON. I have checked in new test cases (to TH3, where, unfortunately, you cannot see them). If anybody sees anything that I hav

Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Richard Hipp
On 11/10/19, Richard Hipp wrote: > It seems there are also problems in generating valid JSON for large > unicode characters. For example: > > SELECT json_quote(char(1114111)); > > I'm working on the problem I take that back. It seems that the spec allow arbitrary UTF-8 characters in a J

Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Richard Hipp
It seems there are also problems in generating valid JSON for large unicode characters. For example: SELECT json_quote(char(1114111)); I'm working on the problem -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@mai

Re: [sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Richard Hipp
On 11/8/19, Serhiy Storchaka wrote: > JSON_EXTRACT [1] produces gibberish for encoded non-BMP characters. Thanks for the bug report and test case. I checked in candidate fix here: https://www.sqlite.org/src/timeline?c=51027f08c0478f1b I need to do additional testing, and due to conflicting obli

[sqlite] JSON_EXTRACT does not work with non-BMP characters

2019-11-10 Thread Serhiy Storchaka
JSON_EXTRACT [1] produces gibberish for encoded non-BMP characters. Example:     sqlite> SELECT JSON_EXTRACT('"t\u00e8\u015b\ud835\udd99"', '$');     tèś�� Expected result is "tèś𝖙". Since character "𝖙" (U+1D599) is not in the Basic Multilingual Plane, it is encoded as the UTF-16 surrogate