cetra3 opened a new pull request, #10352: URL: https://github.com/apache/arrow-rs/pull/10352
# Which issue does this PR close? There are no issues raised. I can raise them for individual findings but that feels like it's a little pedantic. Here's a list of bugs found: | # | Defect | Location | Effect | Status | |---|--------|----------|--------|--------| | 1 | Unsorted dictionary with an empty field name is rejected | `metadata.rs:317` | Rejects **valid** data | Fixed | | 2 | Unsorted dictionary offsets not checked for UTF-8 character boundaries | `metadata.rs` (same branch) | Validated, then **panics** on index/iter | Fixed | | 3 | Duplicate field names in an object are accepted | `object.rs:270` | Accepts **invalid** data | Fixed | | 4 | `decode_date` overflows on large day counts | `decoder.rs:281` | **Panics** inside `try_new` | Fixed | | 5 | `decode_uuid` indexes raw on a truncated payload | `decoder.rs:347` | **Panics** inside `try_new` | Fixed | | 6 | Validation recurses without a depth limit | `list.rs:234`, `object.rs` | **Process abort** | Fixed | | 7 | `shred_variant` double-appends on duplicate field names | `shred_variant.rs:418-429` | **Panics** in arrow-array | Open | | 8 | `PartialEq` recurses without bound on shallow variants | `list.rs:313`, `object.rs` | **Process abort** | Open | # Rationale for this change This PR was raised after some bugs caught in production caused some failing jobs. The original bug was empty field strings caused a weird `offsets not monotonically increasing`, but with the assistance of an agent, we expanded our search to use `proptest` to find some other similar cases. # What changes are included in this PR? Adds a new `proptest` harness for `parquet-variant` and fixes # Are these changes tested? Yes, most of these changes are driven by failing proptests and converted into smaller unit tests. # Are there any user-facing changes? The only one that is probably worth mentioning, and maybe something we allow to be configured, is a max depth recursion constant was added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
