Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Needs Fixing http://json.org/ says that not only '' and '\' need to be escaped with a backslash but also a solidus '/'. And, some control characters have special backspace escapes: backspace (x8 - \b), formfeed (xC - \f), newline (xA - \n), carriage return (xD - \r) and HTab (x9 - \t). Currently, they are serialized as hex. I would propose to add tests for those as well. -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
json.org says that '/' *MAY* be escaped, but it's not *REQUIRED* for it to be so. As for the rest, I've added code to emit them. (The original code didn't emit them either.) Note that it wasn't wrong, just less pretty. I added a test (within the limits of what Zorba accepts in characters). -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Needs Fixing cannot test it on windows as it doesn't currently build on windows (this doesn't have anything to do with this branch). on linux: The following tests FAILED: 1104 - test/rbkt/zorba/jsoniq/coll_dyn_03 (Failed) -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
I think that query is wrong since there are newline literals in strings rather than \n. -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Actually, the query may be OK, but the expected result is wrong. I've fixed it. -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Approve -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Approve -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
The proposal to merge lp:~paul-lucas/zorba/bug-1025622 into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Validation queue job bug-1025622-2012-07-19T18-07-03.448Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
The proposal to merge lp:~paul-lucas/zorba/bug-1025622 into lp:zorba has been updated. Status: Approved = Merged For more details, see: https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Approve -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Paul J. Lucas has proposed merging lp:~paul-lucas/zorba/bug-1025622 into
lp:zorba.
Requested reviews:
Dennis Knochenwefel (dennis-knochenwefel)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #1025622 in Zorba: incorrect JSON serialization of supplementory plane
code points
https://bugs.launchpad.net/zorba/+bug/1025622
For more details, see:
https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636
Now doing proper JSON serialization.
--
https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'src/api/serialization/serializer.cpp'
--- src/api/serialization/serializer.cpp 2012-07-13 06:50:40 +
+++ src/api/serialization/serializer.cpp 2012-07-18 23:45:24 +
@@ -31,8 +31,10 @@
#include api/unmarshaller.h
#include util/ascii_util.h
+#include util/string_util.h
+#include util/unicode_util.h
+#include util/utf8_string.h
#include util/utf8_util.h
-#include util/string_util.h
#include util/xml_util.h
#include system/globalenv.h
@@ -1203,29 +1205,38 @@
/***
/
-void serializer::json_emitter::emit_json_string(zstring string)
+void serializer::json_emitter::emit_json_string(zstring const string)
{
tr '';
- zstring::const_iterator i = string.begin();
- zstring::const_iterator end = string.end();
- for (; i end; i++)
- {
-if (*i 0x20)
-{
- // Escape control sequences
- std::stringstream hex;
- hex \\u std::setw(4) std::setfill('0')
- std::hex static_castint(*i);
- tr hex.str();
- continue;
-}
-if (*i == '\\' || *i == '')
-{
- // Output escape char for \ or
- tr '\\';
- // Fall through to output original character
-}
-tr *i;
+ utf8_stringzstring const const u( string );
+ FOR_EACH( utf8_stringzstring const, i, u ) {
+unicode::code_point const cp = *i;
+if ( ascii::is_cntrl( cp ) ) {
+ std::ostringstream oss;
+ oss std::hex std::setfill('0') \\u std::setw(4) cp;
+ tr oss.str();
+ continue;
+}
+if ( unicode::is_supplementary_plane( cp ) ) {
+ unsigned high, low;
+ unicode::convert_surrogate( cp, high, low );
+ std::ostringstream oss;
+ oss std::hex std::setfill('0')
+ \\u std::setw(4) high
+ \\u std::setw(4) low;
+ tr oss.str();
+ continue;
+}
+switch ( cp ) {
+ case '\\':
+ case '':
+tr '\\';
+// no break;
+ default: {
+utf8::encoded_char_type ec;
+tr.write( ec, utf8::encode( cp, ec ) );
+ }
+}
}
tr '';
}
=== modified file 'src/api/serialization/serializer.h'
--- src/api/serialization/serializer.h 2012-07-13 06:50:40 +
+++ src/api/serialization/serializer.h 2012-07-18 23:45:24 +
@@ -402,7 +402,7 @@
void emit_jsoniq_xdm_node(store::Item *item, int depth);
-void emit_json_string(zstring string);
+void emit_json_string(zstring const string);
store::Item_t theJSONiqValueName;
store::Item_t theTypeName;
=== modified file 'src/util/ascii_util.h'
--- src/util/ascii_util.h 2012-07-12 17:29:55 +
+++ src/util/ascii_util.h 2012-07-18 23:45:24 +
@@ -141,6 +141,25 @@
}
/**
+ * Checks whether the given character is a control character. This function
+ * exists to make a proper function out of the standard iscntrl(3) that may be
+ * implemented as a macro.
+ *
+ * @param CharType The character type.
+ * @param c The character to check.
+ * @return Returns \c true only if the character is a control character.
+ */
+templatetypename CharType inline
+bool is_cntrl( CharType c ) {
+#ifdef WIN32
+ // Windows' iscntrl() implementation crashes for non-ASCII characters.
+ return __isascii( c ) iscntrl( c );
+#else
+ return iscntrl( c );
+#endif
+}
+
+/**
* Checks whether the given character is a decimal digit. This function exists
* to make a proper function out of the standard isdigit(3) that may be
* implemented as a macro.
@@ -160,6 +179,25 @@
}
/**
+ * Checks whether the given character is a printing character. This function
+ * exists to make a proper function out of the standard isprint(3) that may be
+ * implemented as a macro.
+ *
+ * @param CharType The character type.
+ * @param c The character to check.
+ * @return Returns \c true only if the character is a printing character.
+ */
+templatetypename CharType inline
+bool is_print( CharType c ) {
+#ifdef WIN32
+ // Windows' isprint() implementation crashes for non-ASCII characters.
+ return __isascii( c ) isprint( c );
+#else
+ return isprint( c );
+#endif
+}
+
+/**
* Checks whether the given character is a punctuation character. This function
* exists to make a proper function out of the standard ispunct(3) that may be
* implemented as a macro.
--
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
The proposal to merge lp:~paul-lucas/zorba/bug-1025622 into lp:zorba has been updated. Status: Needs review = Approved For more details, see: https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/bug-1025622-2012-07-18T23-51-03.584Z/log.html -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Review: Needs Fixing Cool, thanks. Please add a couple test cases including the problem query described in the bug report; also, add a note to the Changelog mentioning the bug fix. -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Validation queue job bug-1025622-2012-07-18T23-51-03.584Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
Voting does not meet specified criteria. Required: Approve 1, Disapprove 1, Needs Fixing 1, Pending 1. Got: 1 Approve, 1 Needs Fixing, 1 Pending. -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
The proposal to merge lp:~paul-lucas/zorba/bug-1025622 into lp:zorba has been updated. Status: Approved = Needs review For more details, see: https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
I added a mention in the change log and also the one test (can't think of others). -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~paul-lucas/zorba/bug-1025622 into lp:zorba
s/supplementory/supplementary/ -- https://code.launchpad.net/~paul-lucas/zorba/bug-1025622/+merge/115636 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
