Jim Carroll created SPARK-4412:
----------------------------------
Summary: Parquet logger cannot be configured
Key: SPARK-4412
URL: https://issues.apache.org/jira/browse/SPARK-4412
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.2.0
Reporter: Jim Carroll
The Spark ParquetRelation.scala code makes the assumption that the parquet.Log
class has already been loaded. If ParquetRelation.enableLogForwarding executes
prior to the parquet.Log class being loaded then the code in
enableLogForwarding has no affect.
ParquetRelation.scala attempts to override the parquet logger but, at least
currently (and if your application simply reads a parquet file before it does
anything else with Parquet), the parquet.Log class hasn't been loaded yet.
Therefore the code in ParquetRelation.enableLogForwarding has no affect. If you
look at the code in parquet.Log there's a static initializer that needs to be
called prior to enableLogForwarding or whatever enableLogForwarding does gets
undone by this static initializer.
The "fix" would be to force the static initializer to get called in parquet.Log
as part of enableForwardLogging.
PR will be forthcomming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
