GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22522
[SPARK-25510][TEST] Create new trait replace BenchmarkWithCodegen ## What changes were proposed in this pull request? We need create a new trait to replace `BenchmarkWithCodegen` as `BenchmarkWithCodegen` extends from `SparkFunSuite`. For example. when doing `AggregateBenchmark` refactor. Before this change, it should be: ```scala object AggregateBenchmark extends BenchmarkBase { lazy val sparkSession = SparkSession.builder .master("local[1]") .appName(this.getClass.getSimpleName) .config("spark.sql.shuffle.partitions", 1) .config("spark.sql.autoBroadcastJoinThreshold", 1) .getOrCreate() /** Runs function `f` with whole stage codegen on and off. */ def runBenchmark(name: String, cardinality: Long)(f: => Unit): Unit = { val benchmark = new Benchmark(name, cardinality, output = output) benchmark.addCase(s"$name wholestage off", numIters = 2) { iter => sparkSession.conf.set("spark.sql.codegen.wholeStage", value = false) f } benchmark.addCase(s"$name wholestage on", numIters = 5) { iter => sparkSession.conf.set("spark.sql.codegen.wholeStage", value = true) f } benchmark.run() } override def benchmark(): Unit = { runBenchmark("aggregate without grouping") { val N = 500L << 22 runBenchmark("agg w/o group", N) { sparkSession.range(N).selectExpr("sum(id)").collect() } } ... ``` After this change: ```scala object AggregateBenchmark extends BenchmarkBase with RunBenchmarkWithCodegen { override def benchmark(): Unit = { runBenchmark("aggregate without grouping") { val N = 500L << 22 runBenchmark("agg w/o group", N) { sparkSession.range(N).selectExpr("sum(id)").collect() } } ... ``` All affect benchmarks: ``` AggregateBenchmark BenchmarkWideTable JoinBenchmark MiscBenchmark ObjectHashAggregateExecBenchmark SortBenchmark UnsafeArrayDataBenchmark ``` ## How was this patch tested? manual tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/wangyum/spark SPARK-25510 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22522 ---- commit 275cc6c5f8f106eb339c7ed01734e279a223705e Author: Yuming Wang <yumwang@...> Date: 2018-09-21T17:36:57Z Create new BenchmarkWithCodegen trait doesn't extends SparkFunSuite ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org