OmBiradar opened a new pull request, #50158: URL: https://github.com/apache/arrow/pull/50158
### Rationale for this change Currently the parquet file structs are read sequentially, even when the `use_threads` option is used by the user. This PR aims to bridge this gap by making the struct reading truly parallel. ### What changes are included in this PR? Changes to the `LoadBatch` and `BuildArray` functions in the `StructReader` in the Parquet reader to enable multi threaded reads of structs in parquet files. ### Are these changes tested? * Yes I have tested them. Details can be found in this [testing and benchmarking repo](https://github.com/OmBiradar/pyarrow-lincc-fw-openastronomy-gsoc26) * Weather benchmarks should be added for parallel reads is a question I would ask the reviewer. ### Are there any user-facing changes? This is purely a performance related PR. No changes to the user or API is needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
