[
https://issues.apache.org/jira/browse/HADOOP-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12582796#action_12582796
]
dhruba borthakur commented on HADOOP-3110:
------------------------------------------
The logger should be writing log lines to its buffer, so each log message
should be very very low cost. Do you see a case when this causes observeable
difference in performance?
> NameNode does logging in critical sections just about everywhere
> ----------------------------------------------------------------
>
> Key: HADOOP-3110
> URL: https://issues.apache.org/jira/browse/HADOOP-3110
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.14.4, 0.15.0, 0.15.1,
> 0.15.2, 0.15.3, 0.16.0, 0.16.1
> Environment: All
> Reporter: Pete Wyckoff
>
> e.g., FSNameSystem.addStoredBlock (but almost every method has logging in its
> critical sections)
> This method is synchronized and it's spitting something out to Log.info every
> block stored. Normally not a big deal, but since this is in the name node and
> these are critical sections...
> We shouldn't even do any logging at all in critical sections, so even the
> info and warn are bad. But, in many places in the code, it would be hard to
> tease these out (although eventually they really should be), but the system
> could start using something like an AsyncAppender and see how it improves
> performance.
> Even though the log may have a buffer, the writing and doing the formatting
> and stuff cause a drag on performance with 100s/1000s of machines trying to
> talk to the name node.
> At a minimum, the most often triggered Log.info could be changed to
> Log.debug.
> for reference:
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/AsyncAppender.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
