Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21288#discussion_r187822729
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala ---
@@ -32,14 +32,14 @@ import org.apache.spark.util.{Benchmark, Utils}
*/
object FilterPushdownBenchmark {
val conf = new SparkConf()
- conf.set("orc.compression", "snappy")
- conf.set("spark.sql.parquet.compression.codec", "snappy")
+ .setMaster("local[1]")
+ .setAppName("FilterPushdownBenchmark")
+ .set("spark.driver.memory", "3g")
--- End diff --
aha, ok. Looks good to me.
I just added this along with other benchmark code, e.g.,
`TPCDSQueryBenchmark`.
If no problem, I'll fix the other places in follow-up.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
