sdf-jkl opened a new issue, #11627:
URL: https://github.com/apache/arrow-rs/issues/11627

   ### Is your feature request related to a problem or challenge?
   
   We've referred to both Spark/Databricks paths and JSONPath while building
   VariantPath, but haven't settled which behavior we're targeting when they 
differ.
   
   Escaping is one example: Arrow reads `['a\nb']` as the key `anb`. Spark 4.2 
and 4.3 keep the backslash and read `a\nb`. Spark master, targeting 4.4, reads 
`\n` as a newline (apache/spark#59097, merged as SPARK-59218), and 
`spark.sql.legacy.variantPathBackslashAsLiteral` restores the older behavior. 
Arrow also accepts `\]`, which RFC 9535 rejects.
   
   ### Describe the solution you'd like
   
   Agree on the target and document the supported subset:
   
   - Should VariantPath follow Spark/Databricks, RFC 9535, or its existing 
behavior?
   - How should we handle accepted paths whose meaning would change?
   - Which extensions belong to specific consumers, such as `[*]` for shredding
     schemas versus singular paths for `variant_get`?
   
   Capture the decision in a small compatibility table and tests, starting with
   quotes and backslash escapes.
   
   ### Describe alternatives you've considered
   
   Keep the current grammar and document its differences. That's also a valid
   choice; the goal is to make future parsing decisions consistent.
   
   ### Additional context
   
   - [#9012 
discussion](https://github.com/apache/arrow-rs/pull/9012#discussion_r2640273680)
     proposed a Databricks-compatible subset.
   - #9276 introduced arbitrary backslash escaping for unquoted bracket fields;
     #9479 later required quotes while retaining that behavior.
   - #11231 fixed bracket handling inside quotes and preserved `\]` for 
compatibility.
   - datafusion-contrib/datafusion-variant#143 parses rooted paths per Spark 
4.2. datafusion-contrib/datafusion-variant#166 tracks dropping its rootless 
paths.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to