rdtr opened a new pull request, #13539: URL: https://github.com/apache/gluten/pull/13539
## What changes are proposed in this pull request? Velox failed to read some Parquet files with an array of struct (#11865). `spark.read.parquet` on Spark's `parquet-thrift-compat.snappy.parquet` failed with `Converted type VARCHAR is not allowed for requested type ROW<nestedintscolumn:ARRAY<INTEGER>,nestedstringcolumn:VARCHAR>`, and `ParquetAvroCompatibilitySuite` `various complex types` is excluded under the same issue. facebookincubator/velox#16966 (merged 2026-11-03) fixes reading an array of row in the Velox Parquet reader. This re-enables the tests: - `GlutenParquetThriftCompatibilitySuite` `Gluten - Read Parquet file generated by parquet-thrift` in spark34, spark35, spark40, spark41 and spark42 (`ignoreGluten` becomes `testGluten`) - `GlutenParquetAvroCompatibilitySuite` `various complex types` in spark40, spark41 and spark42 (exclude removed) The fix is not in the pinned Velox yet. `dft-2026_11_04` is based on upstream as of 2026-11-02, so this PR depends on the next daily Velox update. I'll rebase once it lands. Fixes #11865 ## How was this patch tested? Built Gluten against the pinned Velox (`dft-2026_11_01`) with facebookincubator/velox#16966 and facebookincubator/velox#19461 cherry-picked, and ran the two suites with `-Dspark.test.home` set: - Spark 4.2, 4.1, 4.0: `GlutenParquetThriftCompatibilitySuite` and `GlutenParquetAvroCompatibilitySuite`, 9 passed, 0 failed - Spark 3.5, 3.4: `GlutenParquetThriftCompatibilitySuite`, 2 passed, 0 failed ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
