amitvijapur opened a new pull request, #11181: URL: https://github.com/apache/arrow-rs/pull/11181
# Which issue does this PR close? Closes #9484. # Rationale for this change `infer_json_schema` promotes a field that holds both scalars and arrays to a list type, but `ListLikeArrayDecoder` only accepts `[` for a list field, so reading the same data with the inferred schema fails with `whilst decoding field 'a': expected [ got 1`. Inference and the reader disagree about their own output. #9485 tried the other direction, removing the promotion from inference; its author closed it, and #9494 restores the promotion to keep that PR's scope narrow. On #9494 @alamb noted that behaviour changes have a higher bar for inclusion than additive ones, and making the reader accept what inference produces is the additive direction. # What changes are included in this PR? In `ListLikeArrayDecoder::decode`, a scalar tape element in a list field is decoded as a single-element list, for both nullable and non-nullable fields. Objects are not promoted, matching inference, which reports an object next to an array as incompatible. The new arm sits after the `ignore_type_conflicts` arm, so with that option on a scalar in a list field still becomes null, as `test_type_conflict_nulls` pins. # Are these changes tested? `test_scalar_promoted_to_list` infers a schema from `1`, `[1, 2]` and `null` and reads it back; `test_scalar_promoted_to_nested_list` does the same for `[1]` next to `[[2, 3]]`, which inference also promotes. Both fail on `main` with the error above. The issue's `test/data/mixed_arrays.json` reads with this change too, but only with `coerce_primitive`, because it also mixes scalar types under one key and inference resolves those to `Utf8`; that second mismatch is separate and not addressed here. `cargo test -p arrow-json` passes. # Are there any user-facing changes? A JSON list field accepts a bare scalar as a one-element list. Inputs that read successfully before read the same way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
