GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/21411
[SPARK-24367][SQL]Parquet: use JOB_SUMMARY_LEVEL instead of deprecated flag
ENABLE_JOB_SUMMARY
## What changes were proposed in this pull request?
In current parquet version,the conf ENABLE_JOB_SUMMARY is deprecated.
When writing to Parquet files, the warning message
```WARN org.apache.parquet.hadoop.ParquetOutputFormat: Setting
parquet.enable.summary-metadata is deprecated, please use
parquet.summary.metadata.level```
keeps showing up.
From
https://github.com/apache/parquet-mr/blame/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetOutputFormat.java#L164
we can see that we should use JOB_SUMMARY_LEVEL.
## How was this patch tested?
Unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark summaryLevel
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21411.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21411
----
commit 6fa194968893b1347e7c08ab5b9eb12716114e46
Author: Gengliang Wang <gengliang.wang@...>
Date: 2018-05-23T13:35:26Z
use JOB_SUMMARY_LEVEL instead of deprecated flag ENABLE_JOB_SUMMARY
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
