andygrove opened a new pull request, #7643:
URL: https://github.com/apache/datafusion-comet/pull/7643

   ## Which issue does this PR close?
   
   Backport of #7348 to `branch-1.2`. Closes #7312 on this branch.
   
   ## Rationale for this change
   
   A captured Comet setting may be unset on the session after a plan is cached. 
Inside a later SQL execution Spark propagates the current SQL configuration to 
the task, but if none of the captured keys remains set, 
`CapturedSQLConf.withConf` mistakes the absence of those keys for an 
unpropagated configuration and reinstalls the stale capture, so the task runs 
with settings the user has since unset. `branch-1.2` has the same `withConf` 
check (it came in with #7302, backported in #7337). See #7348 and #7312 for the 
full analysis.
   
   ## What changes are included in this PR?
   
   Cherry-pick of 8386a34ded (`git cherry-pick -x`) with no conflicts and no 
adaptations. `withConf` now skips the captured configuration when the task has 
a SQL execution id. The source PR's regression test in 
`CometSessionExtensionsSuite` is included.
   
   Other release branches: `branch-1.2` is the newest release branch, so there 
is no newer one to port to. Whether `branch-1.1` needs a backport is not part 
of this PR.
   
   ## How are these changes tested?
   
   On this branch, with JDK 17 and the default Spark profile (4.1): `./mvnw 
test-compile` passes, and `CometSessionExtensionsSuite` passes (3 tests, 0 
failures) against a native library built from `branch-1.2`. With 
`CapturedSQLConf.scala` reverted to `branch-1.2`'s version, the new test 
`captured configs do not override a SQL execution after a setting is unset` 
fails, so the branch has the bug and the test catches it. I ran the default 
Spark profile only.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to