Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19788#discussion_r169229060
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -280,6 +281,16 @@ private[spark] abstract class MapOutputTracker(conf:
SparkConf) extends Logging
}
}
+ protected def supportsContinuousBlockBulkFetch: Boolean = {
+ // continuousBlockBulkFetch only happens in SparkSQL, it uses
UnsafeRowSerializer,
+ // which supports relocation of serialized objects, so we only
consider compression
+ val adaptiveEnabled: Boolean =
conf.getBoolean("spark.sql.adaptive.enabled", false)
--- End diff --
I don't think we can get this conf here. SQL conf is per-session and is
kind of a layer above Spark conf.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
