anew commented on PR #57644: URL: https://github.com/apache/spark/pull/57644#issuecomment-5271488939
Test coverage gap: the new tests cover data-only schemas, named vs implicit flows, wrong data columns, full metadata schemas, and SCD2 omission, but they do not cover the case where the AutoCDC flow’s effective case sensitivity differs from the Spark session. That is the important edge case for this change because the surrounding AutoCDC code derives resolver behavior from pipeline/flow SQL conf, not just the session. I would add coverage with session `spark.sql.caseSensitive=false` and pipeline/flow `spark.sql.caseSensitive=true` (and/or the reverse), plus a materialization-level test to verify the metadata column is appended or not appended under the effective resolver. I would also add a negative test where the user declares `__spark_autocdc_metadata` with the wrong type/nullability so validation catches it before MERGE time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
