nssalian commented on PR #14984: URL: https://github.com/apache/iceberg/pull/14984#issuecomment-5182849845
I took a look at the variant module. The variant + DSv2 side looks like a clean carry-forward from 4.1. SparkV2Filters, SparkScanBuilder, PruneColumnsWithoutReordering, and the reader/writer/shredding code are all identical to 4.1, and the Spark APIs they depend on (VariantVal, getVariant, VariantGet incl. its try_variant_get/variant_get representation, VariantUtil.Type) are unchanged from 4.1.0 to 4.2.0, so variant_get filter pushdown and reads should stay coherent. Also, I checked the spark-tests (4.2, 2.13, core) last workflow run and all six variant cases are succeeding: `TestSparkVariants, TestSparkVariantRead, TestStructInternalRowVariant, TestVariantShredding, TestFilterPushDown#testVariantExtractFiltering, and TestSelect#variantTypeInFilter`. `testVariantExtractFiltering` passing confirms the `try_variant_get(...)` residual-filter assertions hold on 4.2, so no concern. Only nit: the Verification section lists the geospatial suites but not the variant ones. Mind adding a one-liner for the record? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
