Visorgood commented on PR #19017: URL: https://github.com/apache/iceberg/pull/19017#issuecomment-6987188390
Thanks for taking a look, @singhpk234. This comes from #18953, which @manuzhang filed while verifying Spark 4.2.1 (#18294). Spark has its own test for SPARK-60403 in `WholeStageCodegenSuite`. The Iceberg-specific part is MERGE: it runs through `MergeRowsExec` for both copy-on-write and merge-on-read, and when the generated code doesn't compile, Spark falls back from whole-stage codegen without failing the query, so the MERGE just gets slower. I agree the join tests and the checks on generated call names test Spark internals. I can drop `TestOuterJoinCodegen` and the `TestBase` helper and keep only the MERGE test, which runs with `spark.sql.codegen.fallback` disabled and checks the merged rows; it fails on Spark 4.2.0 and passes on 4.2.1. Or I can close this if Spark's coverage is enough. @manuzhang, what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
