Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21882#discussion_r205935403
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/PartitionBatchPruningSuite.scala
---
@@ -161,10 +191,20 @@ class PartitionBatchPruningSuite
query: String,
expectedReadPartitions: Int,
expectedReadBatches: Int)(
- expectedQueryResult: => Seq[Int]): Unit = {
+ expectedQueryResult: => Seq[Any]): Unit = {
+ checkBatchPruning(
+ query, sql(query), expectedReadPartitions,
expectedReadBatches)(expectedQueryResult)
+ }
+
+ def checkBatchPruning(
+ title: String,
+ actual: => DataFrame,
--- End diff --
Without this, we will hit:
```
java.lang.NullPointerException
at
org.apache.spark.sql.test.SQLTestUtilsBase$$anonfun$sql$1.apply(SQLTestUtils.scala:152)
at
org.apache.spark.sql.test.SQLTestUtilsBase$$anonfun$sql$1.apply(SQLTestUtils.scala:152)
at
org.apache.spark.sql.execution.columnar.PartitionBatchPruningSuite.checkBatchPruning(PartitionBatchPruningSuite.scala:195)
at
org.apache.spark.sql.execution.columnar.PartitionBatchPruningSuite.<init>(PartitionBatchPruningSuite.scala:101)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.scalatest.tools.Runner$.genSuiteConfig(Runner.scala:1422)
at org.scalatest.tools.Runner$$anonfun$31.apply(Runner.scala:1236)
at org.scalatest.tools.Runner$$anonfun$31.apply(Runner.scala:1235)
at scala.collection.immutable.List.map(List.scala:284)
at
org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1235)
at
org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
at
org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
at
org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1500)
at
org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
at org.scalatest.tools.Runner$.run(Runner.scala:850)
at org.scalatest.tools.Runner.run(Runner.scala)
at
org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:131)
at
org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
```
I made this evaluation lazy.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
