anuragmantri opened a new pull request, #57508:
URL: https://github.com/apache/spark/pull/57508
## What changes were proposed in this pull request?
This PR fixes dynamic table options being silently dropped when the same
table is referenced more than once in a single DML statement. Options are now
re-applied on a per-query relation cache hit, so each reference keeps its own
options.
## Why are the changes needed?
The per-query relation cache in RelationResolution is keyed by catalog,
namespace, name, and time travel spec, but not by options. When the same table
is referenced twice in one statement with different options, the second
reference gets a cache hit and reuses the first reference's options, so the
options the user wrote on the second reference are silently ignored. This
affects INSERT, UPDATE, and MERGE.
## Does this PR introduce any user-facing change?
No.
## How was this patch tested?
Added end-to-end tests to the following suites, each asserting that both
references keep their own options:
- `DataSourceV2OptionSuite` (INSERT selecting from the same table)
- `MergeIntoTableSuiteBase`
- `UpdateTableSuiteBase`
## Was this patch authored or co-authored using generative AI tooling?
I used Claude Code (Claude Opus 4.8) to generate the code and tests and
verified manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
