To clarify, we're asking about the *spark.sql.tungsten.enabled* flag, which was introduced in Spark 1.5 and enables Project Tungsten optimizations in Spark SQL. This option is set to *true* by default in Spark 1.5+ and exists primarily to allow users to disable the new code paths if they encounter bugs or performance regressions.
If anyone sets spark.sql.tungsten.enabled=*false *in their SparkConf in order to *disable* these optimizations, we'd like to hear from you in order to figure out why you disabled them and to see whether we can make improvements to allow your workload to run with Tungsten enabled. Thanks, Josh On Thu, Oct 15, 2015 at 9:33 AM, mkhaitman <mark.khait...@chango.com> wrote: > Are you referring to spark.shuffle.manager=tungsten-sort? If so, we saw the > default value as still being as the regular sort, and since it was only > first introduced in 1.5, were actually waiting a bit to see if anyone > ENABLED it as opposed to DISABLING it since - it's disabled by default! :) > > I recall enabling it during testing within our dev environment, but didn't > have a comparable workload and environment to our production cluster, so we > were going to play it safe and wait until 1.6 in case there were any major > changes / regressions that weren't seen during 1.5 testing! > > Mark. > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/If-you-use-Spark-1-5-and-disabled-Tungsten-mode-tp14604p14627.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > >
