spark-test commented on PR #57644:
URL: https://github.com/apache/spark/pull/57644#issuecomment-6377019428
Thanks for the update. validateDeclaredReservedFields now catches the
unknown-column, case-duplicate, and nested-field cases at validation time. One
part of the earlier P2 is still open. VirtualTableInput.load calls
appendEngineOwnedReservedFields while downstream flows are being resolved,
before validateUserSpecifiedSchemas runs. So under case-insensitive analysis, a
declaration with both __spark_autocdc_metadata and __SPARK_AUTOCDC_METADATA
that another dataset in the same graph reads still fails with INTERNAL_ERROR on
that reader instead of AUTOCDC_INVALID_RESERVED_SCHEMA.DUPLICATE_COLUMN.
Details are in the new comment on elements.scala.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"conversation:6373631540","thread_id":"conversation:6373631540","verdict_sha256":"e87c9ad3f907d9e9882624c48ca2903fb73e416d9f07a18e4b5b15159500670d"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
