Re: Unicode escapes with any backend encoding

2020-03-06 Thread Chapman Flack
On 3/6/20 2:19 PM, Tom Lane wrote: >> Maybe Chapman has a use case in mind he can test with? Barring that, >> the patch seems ready for commit. > > I went ahead and pushed this, just to get it out of my queue. > Chapman's certainly welcome to kibitz some more of course. Sorry, yeah, I don't think

Re: Unicode escapes with any backend encoding

2020-03-06 Thread Tom Lane
John Naylor writes: > Not this patch's job perhaps, but now that check_unicode_value() only > depends on the input, maybe it can be put into pgwchar.h with other > static inline helper functions? That test is duplicated in > addunicode() and pg_unicode_to_server(). Maybe: > static inline bool > c

Re: Unicode escapes with any backend encoding

2020-03-03 Thread John Naylor
On Tue, Feb 25, 2020 at 1:49 AM Tom Lane wrote: > > I wrote: > > [ unicode-escapes-with-other-server-encodings-2.patch ] > > I see this patch got sideswiped by the recent refactoring of JSON > lexing. Here's an attempt at fixing it up. Since the frontend > code isn't going to have access to enco

Re: Unicode escapes with any backend encoding

2020-02-24 Thread Robert Haas
On Mon, Feb 24, 2020 at 11:19 PM Tom Lane wrote: > I see this patch got sideswiped by the recent refactoring of JSON > lexing. Here's an attempt at fixing it up. Since the frontend > code isn't going to have access to encoding conversion facilities, > this creates a difference between frontend a

Re: Unicode escapes with any backend encoding

2020-02-24 Thread Tom Lane
I wrote: > [ unicode-escapes-with-other-server-encodings-2.patch ] I see this patch got sideswiped by the recent refactoring of JSON lexing. Here's an attempt at fixing it up. Since the frontend code isn't going to have access to encoding conversion facilities, this creates a difference between

Re: Unicode escapes with any backend encoding

2020-01-15 Thread Tom Lane
I wrote: > Andrew Dunstan writes: >> Perhaps I expressed myself badly. What I meant was that we should keep >> the json and text escape rules in sync, as they are now. Since we're >> changing the text rules to allow resolvable non-ascii unicode escapes >> in non-utf8 locales, we should do the same

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Tom Lane
Andrew Dunstan writes: > Perhaps I expressed myself badly. What I meant was that we should keep > the json and text escape rules in sync, as they are now. Since we're > changing the text rules to allow resolvable non-ascii unicode escapes > in non-utf8 locales, we should do the same for json. Got

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Andrew Dunstan
On Wed, Jan 15, 2020 at 7:55 AM Tom Lane wrote: > > Andrew Dunstan writes: > > On Wed, Jan 15, 2020 at 4:25 AM Chapman Flack wrote: > >> On 1/14/20 10:10 AM, Tom Lane wrote: > >>> to me that this error is just useless pedantry. As long as the DB > >>> encoding can represent the desired characte

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Chapman Flack
On 1/14/20 4:25 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On Wed, Jan 15, 2020 at 4:25 AM Chapman Flack wrote: >>> On 1/14/20 10:10 AM, Tom Lane wrote: to me that this error is just useless pedantry. As long as the DB encoding can represent the desired character, it should be tra

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Tom Lane
Andrew Dunstan writes: > On Wed, Jan 15, 2020 at 4:25 AM Chapman Flack wrote: >> On 1/14/20 10:10 AM, Tom Lane wrote: >>> to me that this error is just useless pedantry. As long as the DB >>> encoding can represent the desired character, it should be transparent >>> to users. >> That's my posit

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Andrew Dunstan
On Wed, Jan 15, 2020 at 4:25 AM Chapman Flack wrote: > > On 1/14/20 10:10 AM, Tom Lane wrote: > > to me that this error is just useless pedantry. As long as the DB > > encoding can represent the desired character, it should be transparent > > to users. > > That's my position too. > and mine. c

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Chapman Flack
On 1/14/20 10:10 AM, Tom Lane wrote: > to me that this error is just useless pedantry. As long as the DB > encoding can represent the desired character, it should be transparent > to users. That's my position too. Regards, -Chap

Re: Unicode escapes with any backend encoding

2020-01-14 Thread Tom Lane
I wrote: > Andrew Dunstan writes: >> On Tue, Jan 14, 2020 at 10:02 AM Tom Lane wrote: >>> Grepping for other direct uses of unicode_to_utf8(), I notice that >>> there are a couple of places in the JSON code where we have a similar >>> restriction that you can only write a Unicode escape in UTF8 s

Re: Unicode escapes with any backend encoding

2020-01-13 Thread Tom Lane
Andrew Dunstan writes: > On Tue, Jan 14, 2020 at 10:02 AM Tom Lane wrote: >> Grepping for other direct uses of unicode_to_utf8(), I notice that >> there are a couple of places in the JSON code where we have a similar >> restriction that you can only write a Unicode escape in UTF8 server >> encodi

Re: Unicode escapes with any backend encoding

2020-01-13 Thread Andrew Dunstan
On Tue, Jan 14, 2020 at 10:02 AM Tom Lane wrote: > > > Grepping for other direct uses of unicode_to_utf8(), I notice that > there are a couple of places in the JSON code where we have a similar > restriction that you can only write a Unicode escape in UTF8 server > encoding. I'm not sure whether