zsxwing opened a new pull request, #38553: URL: https://github.com/apache/spark/pull/38553
### What changes were proposed in this pull request? https://github.com/apache/spark/pull/36963 added a check to disallow any source setting `CatalogTable` in the batch plan. However, this check is not safe to enforce: - In a self-union query, the batch plan created by the source will be shared by multiple nodes in the plan. When we transform the plan, the batch plan will be visited multiple times. Hence, the first visit will set the `CatalogTable` and the second visit will try to set it again and fail the query. - A source built by arbitrary developers can set `CatalogTable` in the batch plan. We should not fail as it would break an existing source. This PR fixes the issue by removing the check and set `CatalogTable` only if the batch plan doesn't have one. ### Why are the changes needed? Fix a bug in master. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? The new added unit test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
