ajithme commented on issue #27343: [WIP][SPARK-30621][SQL] Dynamic Pruning thread propagates the localProperties to executionContext URL: https://github.com/apache/spark/pull/27343#issuecomment-578493323 > Do you think we can add a test case? I investigate this and see that Step1 : actual child of `SubqueryBroadcastExec` is a `BroadcastExchangeExec`, which gets evaluated in its own execution context i.e `BroadcastExchangeExec#executionContext` (we need to have https://github.com/apache/spark/pull/27266 to fix `BroadcastExchangeExec` first) already PR is available with testcase Step 2 : Next, in `SubqueryBroadcastExec#executionContext` we just copy this hashkeys (obtained after evaluating relation in Step 1) can be used for dynamic pruning. Currently it do not use any thread local properties to do operations here, but as a safety its better to copy localProperties to `SubqueryBroadcastExec#executionContext` , this we cannot have a testcase as its not used now and more of a code review point.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
