[
https://issues.apache.org/jira/browse/PARQUET-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058769#comment-15058769
]
Ryan Blue commented on PARQUET-369:
-----------------------------------
The issue [~tu...@avast.com] raises is basically that shading both slf4j-api
and slf4j-nop disables logging from thrift entirely. (Only thrift is affected
because the Parquet code doesn't contain any logging.) Specifically, that will
suppress warnings for IOExceptions caught in
[TIOStreamTransport#close|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/transport/TIOStreamTransport.java#L104].
That's the same problem that [~k.shaposhni...@gmail.com] brought up.
I'm still okay with this change. I think that it is rare for Thrift to suppress
exceptions and log them and the right solution is to allow exceptions to
propagate. That behavior isn't going to change, but I still don't think
exceptions that happen on close are worth adding a dependency on SLF4J.
But... this has been flagged twice. In addition, Parquet MR now depends on
SLF4J for its logging (PARQUET-305), so it may not make as much sense to avoid
the dependency from parquet-format any more. As long as any user of Parquet MR
has a dependency on SLF4J, why not add it here also? The version is *old* but
we could add a direct dependency on a newer SLF4J to override the transitive
dependency.
[~julienledem], [~dweeks-netflix], [~alexlevenson], [~lian cheng], what do you
guys think?
> Shading SLF4J prevents SLF4J locating org.slf4j.impl.StaticLoggerBinder
> -----------------------------------------------------------------------
>
> Key: PARQUET-369
> URL: https://issues.apache.org/jira/browse/PARQUET-369
> Project: Parquet
> Issue Type: Bug
> Components: parquet-format
> Reporter: Cheng Lian
> Assignee: Ryan Blue
> Fix For: format-2.3.1
>
>
> Parquet-format shades SLF4J to {{parquet.org.slf4j}} (see
> [here|https://github.com/apache/parquet-format/blob/apache-parquet-format-2.3.0/pom.xml#L162]).
> This also accidentally shades [this
> line|https://github.com/qos-ch/slf4j/blob/v_1.7.2/slf4j-api/src/main/java/org/slf4j/LoggerFactory.java#L207]
> {code}
> private static String STATIC_LOGGER_BINDER_PATH =
> "org/slf4j/impl/StaticLoggerBinder.class";
> {code}
> to
> {code}
> private static String STATIC_LOGGER_BINDER_PATH =
> "parquet/org/slf4j/impl/StaticLoggerBinder.class";
> {code}
> and thus {{LoggerFactory}} can never find the correct {{StaticLoggerBinder}}
> implementation even if we provide dependencies like {{slf4j-log4j12}} on the
> classpath.
> This happens in Spark. Whenever we write a Parquet file, we see the following
> famous message and can never get rid of it:
> {noformat}
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
