hvanhovell commented on issue #27267: [WIP][SPARK-30556][SQL] Copy sparkContext.localproperties to child thread inSubqueryExec.executionContext URL: https://github.com/apache/spark/pull/27267#issuecomment-576333918 IMO the thread pools should not be a big issue. The subquery is guaranteed to be executed on a different thread (you can even add an assert for this). You just set some unique property on the local properties (value should also be unique), construct the something that contains a broadcast join, use an accumulator that you modify using either a UDF (easy) or a dataset operation.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
