[ 
https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119132#comment-16119132
 ] 

Steve Scaffidi commented on SPARK-8118:
---------------------------------------

For anybody who might find this helpful, I found a workaround for hive, 
documented here: 
https://issues.apache.org/jira/browse/SPARK-4412?focusedCommentId=16118403 
(also posted to SO here: https://stackoverflow.com/a/45572400/14186)

And there's a workaround for spark that's similar here: 
https://stackoverflow.com/a/38344902/14186

> Turn off noisy log output produced by Parquet 1.7.0
> ---------------------------------------------------
>
>                 Key: SPARK-8118
>                 URL: https://issues.apache.org/jira/browse/SPARK-8118
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 1.4.1, 1.5.0
>            Reporter: Cheng Lian
>            Assignee: Cheng Lian
>            Priority: Minor
>
> Parquet 1.7.0 renames package name to "org.apache.parquet", need to adjust 
> {{ParquetRelation.enableLogForwarding}} accordingly to avoid noisy log output.
> A better approach than simply muting these log lines is to redirect Parquet 
> logs via SLF4J, so that we can handle them consistently. In general these 
> logs are very useful. Esp. when used to diagnosing Parquet memory issue and 
> filter push-down.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to