[GitHub] spark pull request #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metasto...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22267 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metasto...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22267#discussion_r213885090
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala
---
@@ -1021,6 +1021,18 @@ class ParquetFilterSuite extends QueryTest with
ParquetTest with SharedSQLContex
}
}
}
+
+ test("SPARK-25206: wrong records are returned when Hive metastore schema
and parquet schema " +
--- End diff --
this is a end-to-end test and should not be put here. How about
`HiveParquetSuite`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metasto...
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22267 [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore schema and parquet schema are in different letter cases ## What changes were proposed in this pull request? Since https://github.com/apache/spark/pull/21696. Spark uses Parquet schema instead of Hive metastore schema to do pushdown. This change can avoid wrong records returned when Hive metastore schema and parquet schema are in different letter cases. This pr add a test case for it. More details: https://issues.apache.org/jira/browse/SPARK-25206 ## How was this patch tested? unit tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/wangyum/spark SPARK-24716-TESTS Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22267.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22267 commit f5559f40dc7d3bfd80ced7090f617998094811bf Author: Yuming Wang Date: 2018-08-29T10:03:15Z Improvement test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
