[
https://issues.apache.org/jira/browse/CHUKWA-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang updated CHUKWA-757:
-----------------------------
Attachment: CHUKWA-757.patch
In Hadoop-env.sh, user also needs to define:
{code}
export HADOOP_NAMENODE_OPTS="-Dhadoop.log.port=9096
-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS}
-Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender}
$HADOOP_NAMENODE_OPTS"
export HADOOP_DATANODE_OPTS="-Dhadoop.log.port=9098
-Dhadoop.security.logger=ERROR,RFAS $HADOOP_DATANODE_OPTS"
export HADOOP_SECONDARYNAMENODE_OPTS="-Dhadoop.log.port=9097
-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS}
-Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender}
$HADOOP_SECONDARYNAMENODE_OPTS"
{code}
In yarn-env.sh:
{code}
export YARN_RESOURCEMANAGER_OPTS="-Dhadoop.log.port=9099"
export YARN_NODEMANAGER_OPTS="-Dhadoop.log.port=9100"
{code}
The ports need to align with port number in initial_adaptors for this to work
correctly.
> Supply example log4j.properties to stream Hadoop logs to Chukwa
> ---------------------------------------------------------------
>
> Key: CHUKWA-757
> URL: https://issues.apache.org/jira/browse/CHUKWA-757
> Project: Chukwa
> Issue Type: Sub-task
> Components: Data Collection
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: CHUKWA-757.patch
>
>
> $CHUKWA_CONF_DIR/hadoop-log4j.properties file contains a list of appender to
> stream Hadoop logs to Chukwa. This was designed for Hadoop 0.20.x. For
> supporting Hadoop 2.x, we need to update the log4j.properties example, and
> hadoop-metrics2.properties files as examples of how to stream logs and
> metrics to Chukwa.
> On Chukwa side, the initial_adaptors file also need to register
> SocketAdaptors to stream the data to Chukwa.
> The define ports to stream data are:
> || Port || Description ||
> | 9095 | Hadoop Metrics |
> | 9096 | Hadoop NN Log |
> | 9097 | Hadoop SNN Log |
> | 9098 | Hadoop DN Log |
> | 9099 | Hadoop RM Log |
> | 9100 | Hadoop NM Log |
> | 9101 | Hadoop HS Log |
> | 9102 | Hadoop RM Summary |
> | 9103 | Hadoop TLA |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
