baibaichen opened a new issue, #11914: URL: https://github.com/apache/gluten/issues/11914
## Backend VL (Velox) ## Bug description Two Parquet struct field handling issues in GlutenParquetIOSuite on Spark 4.1: **Issue 1: SPARK-53535** — Spark 4.1 added `returnNullStructIfAllFieldsMissing` parameter for missing struct field handling. Gluten has not adapted to this change. **Issue 2: Vectorized reader missing struct fields** — Pre-existing test (SPARK-34863) for vectorized reader complex type handling. SPARK-53535 modified this test, causing it to fail on Spark 4.1 with Gluten. These are two independent root causes but both only affect Spark 4.1. Parent issue: #11550 ### Impact | Suite | Exclude | spark40 | spark41 | |-------|---------|:-------:|:-------:| | GlutenParquetIOSuite | SPARK-53535 | 🟢 | 🔴 | | GlutenParquetIOSuite | vectorized reader: missing all struct fields | 🟢 | 🔴 | Note: GlutenParquetIOSuite also has a SPARK-54220 (NullType) exclude tracked separately under the Tracking issue #11910. ### References - Apache Spark JIRA: [SPARK-53535](https://issues.apache.org/jira/browse/SPARK-53535) - Apache Spark JIRA: [SPARK-34863](https://issues.apache.org/jira/browse/SPARK-34863) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
