cloud-fan commented on code in PR #59097:
URL: https://github.com/apache/spark/pull/59097#discussion_r4332529373
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionSuite.scala:
##########
@@ -737,6 +737,41 @@ class VariantExpressionSuite extends SparkFunSuite with
ExpressionEvalHelper {
// scalastyle:on nonascii
testVariantGet("[1, 2, 3]", "$[2147483647]", IntegerType, null)
+ testVariantGet("""{"a'b": 1}""", """$['a\'b']""", IntegerType, 1)
+ testVariantGet("""{"a']b": 1}""", """$['a\']b']""", IntegerType, 1)
+ testVariantGet("""{"a\nb":1,"a\\nb":2}""", """$.a\nb""", IntegerType, 2)
+ testVariantGet("""{"a\"b": 2}""", """$['a"b']""", IntegerType, 2)
+ testVariantGet("""{"a\\b": 4}""", """$['a\\b']""", IntegerType, 4)
+ testVariantGet("""{"a\tb": 5}""", """$['a\tb']""", IntegerType, 5)
+ testVariantGet("""{"a\nb": 6}""", """$['a\nb']""", IntegerType, 6)
+ testVariantGet("""{"a\bb": 11}""", """$['a\bb']""", IntegerType, 11)
+ testVariantGet("""{"a\fb": 12}""", """$['a\fb']""", IntegerType, 12)
+ testVariantGet("""{"a\rb": 13}""", """$['a\rb']""", IntegerType, 13)
+ testVariantGet("""{"a/b": 14}""", """$['a\/b']""", IntegerType, 14)
+ // Split the Unicode escape so Scala does not decode it.
+ testVariantGet("{\"a\\" + "u000bb\": 7}", "$['a\\" + "u000bb']",
IntegerType, 7)
+ testVariantGet("""{"a\"b": 8}""", """$["a\"b"]""", IntegerType, 8)
+ testVariantGet("""{"a'b": 9}""", """$["a'b"]""", IntegerType, 9)
+ testVariantGet("""{"a\\xb": 15}""", """$['a\xb']""", IntegerType, 15)
+ testVariantGet("""{"a\\u12G4b": 17}""", "$['a\\" + "u12G4b']",
IntegerType, 17)
+ testVariantGet("""{"a\\\"b": 18}""", """$['a\"b']""", IntegerType, 18)
+ testVariantGet("""{"a\\'b": 19}""", """$["a\'b"]""", IntegerType, 19)
+ testVariantGet(
+ "{\"a\\" + "uD83D\\" + "uDE00b\": 20}",
+ "$['a\\" + "uD83D\\" + "uDE00b']",
Review Comment:
**Nit (P3):** The pair branch of `escapeSeq`
(`u[dD][89abAB][0-9a-fA-F]{2}\\u[dD][c-fC-F][0-9a-fA-F]{2}`) accepts either hex
case and any high surrogate from D800 to DBFF, but the pair cases here use only
upper-case escapes with a D8 high surrogate (`\uD83D\uDE00`). If the pattern
were narrowed to upper-case hex, or to D8 high surrogates, every assertion
would still pass. In the default mode, a path such as `$['\ud83d\ude00']` (the
lower-case form that serializers such as Python's `json.dumps` emit) or
`$['\uDBFF\uDFFF']` would then decode as two unpaired escapes kept as literal
text, so `variant_get` and the manipulation functions would silently address a
different key. One lower-case pair case and one DBxx pair case next to these
tests would close the gap. Legacy mode decodes no escapes and is not affected.
**Verification:**
- **Regression:** Restricting the surrogate-pair pattern to upper-case hex
digits makes the lower-case pair assertion fail.
- **Regression:** Restricting the high-surrogate range to D8xx makes the
DB00-DBFF pair assertion fail.
- **Behavior:** With the current decoder, both new pair cases resolve the
decoded supplementary-character key in default mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
