Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21376#discussion_r189472746
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -107,7 +107,20 @@ object SQLConf {
* run tests in parallel. At the time this feature was implemented, this
was a no-op since we
* run unit tests (that does not involve SparkSession) in serial order.
*/
- def get: SQLConf = confGetter.get()()
+ def get: SQLConf = {
+ if (TaskContext.get != null) {
+ new ReadOnlySQLConf(TaskContext.get())
+ } else {
+ if (Utils.isTesting && SparkContext.getActive.isDefined) {
+ val schedulerEventLoopThread =
+
SparkContext.getActive.get.dagScheduler.eventProcessLoop.eventThread
+ if (schedulerEventLoopThread.getId ==
Thread.currentThread().getId) {
+ throw new RuntimeException("Cannot get SQLConf inside scheduler
event loop thread.")
--- End diff --
Not sure if we need to add a short comment for the reason. WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
