[
https://issues.apache.org/jira/browse/SPARK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545677#comment-14545677
]
Yana Kadiyska edited comment on SPARK-4412 at 5/16/15 5:11 PM:
---------------------------------------------------------------
I would like to reopen as I believe the issue has again regressed in Spark
1.3.0. This SO thread has a lengthy discussion
http://stackoverflow.com/questions/30052889/how-to-suppress-parquet-log-messages-in-spark
but the short summary is that log4j.rootCategory=ERROR, console setting still
leaks
{quote}
INFO: parquet.hadoop.InternalParquetRecordReader
{quote} messages
was (Author: yanakad):
I would like to reopen as I believe the issue has again regressed in Spark
1.3.0. This SO thread has a lengthy discussion
http://stackoverflow.com/questions/30052889/how-to-suppress-parquet-log-messages-in-spark
but the short summary is that log4j.rootCategory=ERROR, console setting still
leaks INFO: parquet.hadoop.InternalParquetRecordReader messages
> Parquet logger cannot be configured
> -----------------------------------
>
> Key: SPARK-4412
> URL: https://issues.apache.org/jira/browse/SPARK-4412
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Jim Carroll
>
> The Spark ParquetRelation.scala code makes the assumption that the
> parquet.Log class has already been loaded. If
> ParquetRelation.enableLogForwarding executes prior to the parquet.Log class
> being loaded then the code in enableLogForwarding has no affect.
> ParquetRelation.scala attempts to override the parquet logger but, at least
> currently (and if your application simply reads a parquet file before it does
> anything else with Parquet), the parquet.Log class hasn't been loaded yet.
> Therefore the code in ParquetRelation.enableLogForwarding has no affect. If
> you look at the code in parquet.Log there's a static initializer that needs
> to be called prior to enableLogForwarding or whatever enableLogForwarding
> does gets undone by this static initializer.
> The "fix" would be to force the static initializer to get called in
> parquet.Log as part of enableForwardLogging.
> PR will be forthcomming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
