[
https://issues.apache.org/jira/browse/HDFS-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107409#comment-14107409
]
Colin Patrick McCabe commented on HDFS-4486:
--------------------------------------------
It's pretty strange, all right. I'm also seeing a lot of "too many open files"
exceptions, which makes me wonder if the JVM is failing to load the jar file
because it's out of file descriptors?
org/apache/hadoop/util/PerformanceAdvisory.java is certainly in hadoop-common,
so there should be no problem depending on it in hadoop-hdfs. The
hadoop-common unit tests also pass prior to the hadoop-hdfs tests throwing this
exception, so we know that the class got compiled.
Just a random guess, but maybe try adding a reference to PerformanceAdvisory
inside MiniDFSCluster.java, to force Maven to realize that there is a
dependency? Or try adding a constructor or other method to
PerformanceAdvisory.. maybe it's getting somehow optimized out since it only
has a static member now?
> Add log category for long-running DFSClient notices
> ---------------------------------------------------
>
> Key: HDFS-4486
> URL: https://issues.apache.org/jira/browse/HDFS-4486
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Todd Lipcon
> Assignee: Zhe Zhang
> Priority: Minor
> Attachments: HDFS-4486-20140820.patch, HDFS-4486-20140821.patch,
> HDFS-4486-20140822.patch, hdfs-4486-20140821-2.patch
>
>
> There are a number of features in the DFS client which are transparent but
> can make a fairly big difference for performance -- two in particular are
> short circuit reads and native checksumming. Because we don't want log spew
> for clients like "hadoop fs -cat" we currently log only at DEBUG level when
> these features are disabled. This makes it difficult to troubleshoot/verify
> for long-running perf-sensitive clients like HBase.
> One simple solution is to add a new log category - eg
> o.a.h.h.DFSClient.PerformanceAdvisory - which long-running clients could
> enable at DEBUG level without getting the full debug spew.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
