[
https://issues.apache.org/jira/browse/CHUKWA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714219#action_12714219
]
Eric Yang commented on CHUKWA-267:
----------------------------------
We do not want synchronize to occur on every log statement, the performance
will be severely impacted, if we lock on every log statement. Hence, we check
for condition first, if there is already one of thread pass through, then rest
of the reads don't need to go through the same subroutine. There would be only
one thread pass in this subroutine to creates the background thread to register
the file. Second, we can't do this in the constructor because we do not know
the task attempt id until there is actually something to log, hence this was
done at subAppend level.
> Collect shuffling information from userlogs/syslog
> --------------------------------------------------
>
> Key: CHUKWA-267
> URL: https://issues.apache.org/jira/browse/CHUKWA-267
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: data collection
> Affects Versions: 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.1.2, 0.2.0
>
> Attachments: CHUKWA-267.patch
>
>
> Hadoop uses a customized version of log4j appender
> (org.apache.hadoop.mapred.TaskLogAppender) for tasks. Shuffling information
> is written to userlogs/syslog by TaskLogAppender. For chukwa to collect
> shuffling information, we need to write a new TaskLogAppender to do this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
