Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22435#discussion_r217933342
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/DataSourceScanExecRedactionSuite.scala
---
@@ -83,4 +83,20 @@ class DataSourceScanExecRedactionSuite extends QueryTest
with SharedSQLContext {
}
}
+ test("FileSourceScanExec metadata") {
+ withTempDir { dir =>
+ val basePath = dir.getCanonicalPath
+ spark.range(0, 10).toDF("a").write.parquet(new Path(basePath,
"foo=1").toString)
+ val df = spark.read.parquet(basePath).filter("a = 1")
--- End diff --
Hi, @wangyum . I know that you follow the style of the other test cases in
this suite, but could you simplify like the following? We had better keep a
single test case as simple as possible by excluding irrelevant stuffs.
```scala
withTempPath { path =>
val dir = path.getCanonicalPath
spark.range(0, 10).toDF("a").write.parquet(dir)
val df = spark.read.parquet(dir).filter("a = 1")
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
