CaptainAni187 opened a new pull request, #51807:
URL: https://github.com/apache/arrow/pull/51807

   ### Rationale for this change
   
   `Schema.fbs` allows `DictionaryEncoding.indexType` to be omitted: "If this 
field is null, the indices must be signed int32." The C++ IPC reader required 
it instead, so C++ and PyArrow failed to read such a stream with `IOError: 
Unexpected null field DictionaryEncoding.indexType in flatbuffer-encoded 
metadata`.
   
   ### What changes are included in this PR?
   
   `FieldFromFlatbuffer` uses `int32()` as the index type when `indexType` is 
absent, instead of failing the null check.
   
   ### Are these changes tested?
   
   Yes. `TestMessageInternal.DictionaryEncodingWithoutIndexType` builds a 
schema message whose dictionary-encoded field has no `indexType` and checks 
that `ReadSchema` returns `dictionary<values=string, indices=int32>`. On main 
it fails with the error above. The `arrow-ipc-*` tests pass locally.
   
   ### Are there any user-facing changes?
   
   Streams that omit `indexType` can now be read. Streams that include it are 
read as before.
   
   ### Was AI used for this PR?
   
   In accordance to the [AI generation 
guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code),
 please disclose below whether and how AI was used in this PR.
   
   **PR code and description written by:**
   
   - [ ] Human
   - [ ] AI
   
   **Reviewed before submission by:**
   
   - [ ] Human
   - [ ] AI
   - [ ] Not reviewed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to