wangyum commented on a change in pull request #24371:
[SPARK-27176][FOLLOW-UP][SQL] Shade Parquet to sql/core
URL: https://github.com/apache/spark/pull/24371#discussion_r276199388
##########
File path:
sql/hive/src/test/scala/org/apache/spark/sql/sources/ParquetHadoopFsRelationSuite.scala
##########
@@ -124,7 +123,7 @@ class ParquetHadoopFsRelationSuite extends
HadoopFsRelationTest {
test("SPARK-8604: Parquet data source should write summary file while doing
appending") {
withSQLConf(
- ParquetOutputFormat.JOB_SUMMARY_LEVEL -> "ALL",
+ "parquet.summary.metadata.level" -> "ALL",
Review comment:
No, both versions have the same class name, but they are not the same. E.g:
`org.apache.parquet.hadoop.ParquetOutputFormat`. parquet 1.10.1 has
JOB_SUMMARY_LEVEL, but parquet 1.8.1 does not have JOB_SUMMARY_LEVEL:
https://github.com/apache/parquet-mr/blob/apache-parquet-1.10.1/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetOutputFormat.java#L131
https://github.com/apache/parquet-mr/blob/apache-parquet-1.8.1/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetOutputFormat.java#L106
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
