Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22594#discussion_r224146203
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
---
@@ -70,6 +70,8 @@ class FileScanRDD(
private val ignoreCorruptFiles =
sparkSession.sessionState.conf.ignoreCorruptFiles
private val ignoreMissingFiles =
sparkSession.sessionState.conf.ignoreMissingFiles
+ // only for test
+ private val inputMetricsTest =
sparkSession.sessionState.conf.contains("spark.inputmetrics.test")
--- End diff --
Ah you know, we have `spark.testing` for this purpose too.
I'm actually on the fence here about whether it's worth this extra
complexity for the test. It's a simple change and the overall effect is tested
by other tests of input metrics.
Hm, what do you think, just drop this? is it necessary for the test to work
correctly?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
