[
https://issues.apache.org/jira/browse/HDFS-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101674#comment-14101674
]
Colin Patrick McCabe commented on HDFS-4486:
--------------------------------------------
The main functionality we want here is to be able to turn on the logging of
these perf issues selectively. It seems like all we need to do is create a
{{Logger}} instance. I like your suggestion of {{PerformanceAdvisory}}. So
really, all we need is a class with that name, with a {{Logger}} inside. Then
we replace the {{LOG.debug}} invocations with {{PerformanceAdvisory.LOG.debug}}
invocations and add a few import statements...
I don't think we need a custom log level. Maybe I'm missing something, but I
can't think of any way that it would help, and it would certainly complicate
life (if we ever switch logging frameworks, like people are suggesting on the
mailing list). And where does it fit in the continuum of TRACE = most logging,
WARN = least? etc. It just seems like way overkill for this.
The main thing to do is just to track down places where we quietly fall back to
a lower-performance solution. Things like
{{JniBasedGroupsMappingWithFallback}}, {{NativeIO}}, some of the checksum
stuff, etc. It's not much code, but it will be a lot of tracking down those
cases.
> Add log category for long-running DFSClient notices
> ---------------------------------------------------
>
> Key: HDFS-4486
> URL: https://issues.apache.org/jira/browse/HDFS-4486
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Todd Lipcon
> Assignee: Zhe Zhang
> Priority: Minor
>
> There are a number of features in the DFS client which are transparent but
> can make a fairly big difference for performance -- two in particular are
> short circuit reads and native checksumming. Because we don't want log spew
> for clients like "hadoop fs -cat" we currently log only at DEBUG level when
> these features are disabled. This makes it difficult to troubleshoot/verify
> for long-running perf-sensitive clients like HBase.
> One simple solution is to add a new log category - eg
> o.a.h.h.DFSClient.PerformanceAdvisory - which long-running clients could
> enable at DEBUG level without getting the full debug spew.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
