anew opened a new pull request, #57517:
URL: https://github.com/apache/spark/pull/57517
### What changes were proposed in this pull request?
`AutoCdcAuxiliaryTable.buildScd1AuxiliaryTableSpecFor` had no focused unit
test; its behavior was only exercised indirectly via higher-level pipeline
suites. [SPARK-58320](https://issues.apache.org/jira/browse/SPARK-58320) added
`AutoCdcScd2AuxiliaryTableSpecSuite` for the SCD2 spec builder; this adds the
parallel SCD1 coverage.
`AutoCdcScd1AuxiliaryTableSpecSuite` resolves a real SCD1 AutoCDC graph and
asserts the derived auxiliary-table spec:
- the schema is exactly the key columns plus the CDC metadata column -- no
user data columns and no SCD2 framework columns, since the SCD1 auxiliary table
stores per-key tombstones rather than full rows;
- the CDC metadata column uses the SCD1 metadata struct schema and is
non-null;
- the spec records `ScdType.Type1`, the key column names (property +
`expectedKeyFields`), and the correct identifiers;
- composite keys are all carried through.
### Why are the changes needed?
Fills a test-coverage gap for the SCD1 auxiliary-table spec derivation,
matching the coverage that now exists for SCD2, so future changes to
`buildScd1AuxiliaryTableSpecFor` (schema shape, recorded drift metadata) are
guarded by a focused unit test.
### Does this PR introduce _any_ user-facing change?
No. Test-only change; no production code is modified.
### How was this patch tested?
- `build/sbt 'pipelines/testOnly *AutoCdcScd1AuxiliaryTableSpecSuite'` --
5/5 pass
- `dev/lint-scala` -- Scalastyle and Scalafmt clean
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
