According to RFC4627 [1] "JSON text SHALL be encoded in Unicode. The default encoding is UTF-8.". It says the following about escaping of unicode characters in string litterals: "All Unicode characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F).", it also says that "Any character may be escaped.". This means that it is ok for the encoding library you are using to escape any char that it wishes (which apparently it does), but it must also be able to parse all unescaped chars (unless they are in the range mentioned above).
The example you mentioned: "\u2018Hello world\u2018", is probably properly escaped since 2018 is LEFT SINGLE QUOTATION MARK [2], and quotation marks should be escaped, but is this the sequence you are experiencing problems with? If that gets returned unescaped, that could be a problem, but if it's another char, like \u00f6, then it should be allowed to be represented unescaped as a proper unicode char. It might still be a bug though, so if you could provide some more details that would be great. Also could you check if the truncation of messages occurs in your client due to decoding issues, on the server or on the wire. Cheers, Tobias [1] http://www.ietf.org/rfc/rfc4627.txt?number=4627 [2] http://www.fileformat.info/info/unicode/char/2018/index.htm On Tue, Apr 13, 2010 at 12:07 AM, Alastair James <[email protected]> wrote: > Arh... notice that the json returned from the API has the UTF-8 data > directly as UTF-8. Should it not also return encoded as \u1234 codes? > > Al > > On 12 April 2010 23:03, Alastair James <[email protected]> wrote: > > > Hi there... > > > > Was going to submit this as a bug, but I am not sure it is a bug (well, > its > > undesirable behaviour, but might need more discussion). > > > > Playing with the REST API and PHP. All going well, however one problem. > > > > The data I am trying to save is encoded in UTF-8. When I try to save data > > in this format, the response from the sever is truncated (e.g. cut off in > > mid response). The PHP json_encode function converts the utf-8 data to > > unicode codepoitnts (eg. \u1234). > > > > E.g. > > > > $node = array( > > 'title' => 'Some text with UTF-8 chars' > > ); > > json_encode($node); > > > > returns something like '\u2018Hello world\u2018' > > > > However if you post data like this to the REST API the response comes > back > > cut off. This is an example response: > > > > {"incoming typed relationships":" > > http://localhost:9999/node/80/relationships/in/{-list|&|types}< > http://localhost:9999/node/80/relationships/in/%7B-list%7C&%7Ctypes%7D > >","incoming > > relationships":"http://localhost:9999/node/80/relationships/in","all > > relationships":"http://localhost:9999/node/80/relationships/all","create > > relationship":"http://localhost:9999/node/80/relationships > ","data":{"title":"Intihuatana > > Stone, the ‘Hitching Post of the Sun,‘"},"traverse":" > > http://localhost:9999/node/80/traverse/{returnType}< > http://localhost:9999/node/80/traverse/%7BreturnType%7D> > > ","property":"http://localhost:9999/node/80/properties/{key}< > http://localhost:9999/node/80/properties/%7Bkey%7D> > > ","self":"http://localhost:9999/node/80","properties":"; > > http://localhost:9999/node/80/properties","all typed relationships":" > > http://localhost:9999/node/80/relationships/all/{-list|&|types}< > http://localhost:9999/node/80/relationships/all/%7B-list%7C&%7Ctypes%7D > >","outgoing > > typed relationships":" > > http://localhost:9999/node/80/relationships/out/{-list|&|types}< > http://localhost:9999/node/80/relationships/out/%7B-list%7C&%7Ctypes%7D > >","outgoing > > relationships":"http://localhost:9999/node/80/relationships/o > > > > Note that it cuts off half way through the "outgoing relationships" > value. > > > > So, whats the correct way to do this? Is this a bug, or is it that I need > > to pass in a HTTP header to signal its unicode? Is the problem that (as I > > beleive java uses UTF-16 internally) its outputting the wrong character > > encoding? Is it possible to tell it what you want? > > > > Al > > > > > > > -- > Dr Alastair James > CTO James Publishing Ltd. > http://www.linkedin.com/pub/3/914/163 > > www.worldreviewer.com > > WINNER Travolution Awards Best Travel Information Website 2009 > WINNER IRHAS Awards, Los Angeles, Best Travel Website 2008 > WINNER Travolution Awards Best New Online Travel Company 2008 > WINNER Travel Weekly Magellan Award 2008 > WINNER Yahoo! Finds of the Year 2007 > > "Noli nothis permittere te terere!" > _______________________________________________ > Neo mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Tobias Ivarsson <[email protected]> Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
