cloud-fan opened a new pull request, #56113:
URL: https://github.com/apache/spark/pull/56113
### What changes were proposed in this pull request?
Follow-up cleanup of review feedback on #56042:
- Remove the now-dead
`Scd1BatchProcessor.validateCdcMetadataColumnNotPresent` validator and its call
site. It referenced the error class `AUTOCDC_RESERVED_COLUMN_NAME_CONFLICT`
which the parent PR removed from `error-conditions.json`; the new
construction-time check in
`AutoCdcMergeFlow.requireReservedPrefixAbsentInSourceColumns` is the
authoritative validator and supersedes it.
- Lift `comment: Option[String]` to the `UnresolvedFlow` trait so both
`UntypedFlow` and `AutoCdcFlow` carry it symmetrically; previously only
`AutoCdcFlow` had it.
- Reorder `AutoCdcFlow`'s constructor so defaulted params (`sqlConf`,
`comment`) trail the non-defaulted ones (`origin`, `changeArgs`), allowing
positional construction.
- Fix Scaladoc/comment text: factual wording for the keys-presence check,
the `[[ResolvedFlow.load]]` link, the `Scd1ForeachBatchHandler` reference (was
`Scd1ForeachBatchExec`, which does not exist), and several minor
grammar/typography nits.
### Why are the changes needed?
Cleanup of follow-up items identified during review of the parent PR. The
dead validator is the most material: if its code path were reached, it would
throw an internal `SparkException("Cannot find main error class ...")` instead
of a user-facing `AnalysisException`. The path is unreachable through current
call patterns (the construction-time check rejects bad schemas before this code
runs), but the references should not be left behind.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests in `AutoCdcFlowSuite`, `Scd1BatchProcessorSuite`,
`ConnectInvalidPipelineSuite`, and `ConnectValidPipelineSuite` continue to
cover the affected paths. No new tests added; the change is structural cleanup
and doc edits.
### Was this patch authored or co-authored using generative AI tooling?
Co-authored by Claude.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
