sdf-jkl opened a new issue, #11228:
URL: https://github.com/apache/arrow-rs/issues/11228
### Describe the bug
`VariantPath::try_from` accepts quoted bracket fields, but an unescaped `]`
terminates the segment even when it is inside the field's quotes. For example,
`["a]b"]` cannot select the literal key `a]b`.
This is a request to improve the quoted-field grammar. The current parser
explicitly documents `\]` as an escape, so the rejected spelling is not a
regression against that documented rule. However, quoting a field does not
currently protect its bracket boundaries, and some inputs are accepted as a
different, multi-field path rather than rejected.
### To Reproduce
```rust
use parquet_variant::{VariantPath, VariantPathElement};
let expected = VariantPath::from_iter([VariantPathElement::field("a]b")]);
// Current workaround succeeds.
assert_eq!(VariantPath::try_from(r#"["a\]b"]"#).unwrap(), expected);
// Requested behavior: these quoted fields should identify the same key.
// Currently both return ParseError.
assert_eq!(VariantPath::try_from(r#"["a]b"]"#).unwrap(), expected);
assert_eq!(VariantPath::try_from("['a]b']").unwrap(), expected);
```
`["[0]"]` similarly fails to select the literal key `[0]`.
A quoted field can also be split silently:
A generated test reduced to the four-character key `"]["`. Escaping its
quotes, but leaving its bracket literal, produces this path:
```text
["\"][\""]
```
Current behavior:
```rust
use parquet_variant::{VariantPath, VariantPathElement};
let key = r#""][""#;
let parsed = VariantPath::try_from(r#"["\"][\""]"#).unwrap();
// Actual: two empty field names.
assert_eq!(parsed, VariantPath::default().join("").join(""));
// Requested under a quote-aware grammar: one literal field name.
let expected = VariantPath::from_iter([VariantPathElement::field(key)]);
assert_ne!(parsed, expected);
```
With an object containing both that literal key mapped to `42` and nested
empty keys mapped to `99`, `Variant::get_path` returns `99` for the parsed
path; a programmatically constructed single-field path returns `42`. This input
omits the currently required bracket escape, but accepting it as a different
path makes the limitation less obvious than the basic parse error.
### Expected behavior
Quoted fields should preserve a literal `]` until the matching closing
quote. `["a]b"]` and `['a]b']` should both select the single key `a]b`; the
reduced case above should likewise produce one literal field rather than two
empty fields.
Keep the existing `\]` spelling working. The request concerns quoted bracket
boundaries; decisions about other escape rules or previously accepted ambiguous
quote sequences can be handled separately.
### Additional context
[`parse_in_bracket`](https://github.com/apache/arrow-rs/blob/92b50428182929402f669aead6015a2c89f3be4a/parquet-variant/src/utils.rs#L230)
consumes escapes and stops at the first unescaped `]`. Only afterward does it
strip surrounding quotes. It therefore has no quote state while deciding where
the segment ends.
Spark 4.2 accepts `$["a]b"]`, `$['a]b']`, and `$["[0]"]`. The Arrow
reproductions above require no SQL parser.
A fixed-seed generated audit (`0x5eed20260926`) checked 116,189 key cases,
including exhaustive short strings, random keys, Unicode, controls, and long
keys. All 464,756 root/nested checks using Arrow's documented escaping passed.
When `]` was left literal inside quotes, failures were confined to keys
containing `]`, including 63 silently misparsed root paths. A separate
100,000-path random sweep found no panics. This is sampled testing, not proof
of exhaustive correctness.
Related: [#8954](https://github.com/apache/arrow-rs/issues/8954),
[#9479](https://github.com/apache/arrow-rs/pull/9479), and the
allocation-focused [#9493](https://github.com/apache/arrow-rs/issues/9493).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
