Repository: hadoop Updated Branches: refs/heads/trunk f250ad177 -> fc20129c5
HDFS-7623. Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace. (yliu) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/fc20129c Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/fc20129c Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/fc20129c Branch: refs/heads/trunk Commit: fc20129c56f7703f4a42ee025a5702e3458bdcc2 Parents: f250ad1 Author: yliu <[email protected]> Authored: Wed Jan 21 00:59:47 2015 +0800 Committer: yliu <[email protected]> Committed: Wed Jan 21 00:59:47 2015 +0800 ---------------------------------------------------------------------- .../src/main/resources/core-default.xml | 24 +++++++++++++++++++ .../hadoop-common/src/site/apt/Tracing.apt.vm | 25 +++++++++++++++----- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 3 +++ 3 files changed, 46 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/fc20129c/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml index e7a382d..bebc263 100644 --- a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml +++ b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml @@ -1653,4 +1653,28 @@ for ldap providers in the same way as above does. </description> </property> +<property> + <name>hadoop.htrace.sampler</name> + <value>NeverSampler</value> + <description> + Configure the samplers for HTrace, the value can be NeverSampler, + AlwaysSampler or ProbabilitySampler. NeverSampler: HTrace is OFF + for all spans; AlwaysSampler: HTrace is ON for all spans; + ProbabilitySampler: HTrace is ON for some percentage% of top-level + spans. + </description> +</property> + +<property> + <name>hadoop.htrace.spanreceiver.classes</name> + <value></value> + <description> + A comma separated list of the fully-qualified class name of classes + implementing SpanReceiver. The tracing system works by collecting + information in structs called 'Spans'. It is up to you to choose + how you want to receive this information by implementing the + SpanReceiver interface. + </description> +</property> + </configuration> http://git-wip-us.apache.org/repos/asf/hadoop/blob/fc20129c/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm b/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm index 9eda220..f04da33 100644 --- a/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm +++ b/hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm @@ -16,19 +16,32 @@ --- ${maven.build.timestamp} -Enabling Dapper-like Tracing in HDFS +Enabling Dapper-like Tracing in Hadoop %{toc|section=1|fromDepth=0} -* {Dapper-like Tracing in HDFS} +* {Dapper-like Tracing in Hadoop} ** HTrace {{{https://issues.apache.org/jira/browse/HDFS-5274}HDFS-5274}} added support for tracing requests through HDFS, - using the open source tracing library, {{{https://github.com/cloudera/htrace}HTrace}}. + using the open source tracing library, {{{https://git-wip-us.apache.org/repos/asf/incubator-htrace.git}Apache HTrace}}. Setting up tracing is quite simple, however it requires some very minor changes to your client code. +** Samplers + Configure the samplers in <<<core-site.xml>>> property: <<<hadoop.htrace.sampler>>>. + The value can be NeverSampler, AlwaysSampler or ProbabilitySampler. NeverSampler: HTrace is OFF + for all spans; AlwaysSampler: HTrace is ON for all spans; ProbabilitySampler: HTrace is ON for + some percentage% of top-level spans. + ++---- + <property> + <name>hadoop.htrace.sampler</name> + <value>NeverSampler</value> + </property> ++---- + ** SpanReceivers The tracing system works by collecting information in structs called 'Spans'. @@ -42,7 +55,7 @@ public void receiveSpan(Span span); Configure what SpanReceivers you'd like to use by putting a comma separated list of the fully-qualified class name of classes implementing SpanReceiver - in <<<hdfs-site.xml>>> property: <<<hadoop.htrace.spanreceiver.classes>>>. + in <<<core-site.xml>>> property: <<<hadoop.htrace.spanreceiver.classes>>>. +---- <property> @@ -83,11 +96,11 @@ public void receiveSpan(Span span); $ git clone https://github.com/cloudera/htrace $ cd htrace/htrace-zipkin $ mvn compile assembly:single - $ cp target/htrace-zipkin-*-jar-with-dependencies.jar $HADOOP_HOME/share/hadoop/hdfs/lib/ + $ cp target/htrace-zipkin-*-jar-with-dependencies.jar $HADOOP_HOME/share/hadoop/common/lib/ +---- The sample configuration for <<<ZipkinSpanReceiver>>> is shown below. - By adding these to <<<hdfs-site.xml>>> of NameNode and DataNodes, + By adding these to <<<core-site.xml>>> of NameNode and DataNodes, <<<ZipkinSpanReceiver>>> is initialized on the startup. You also need this configuration on the client node in addition to the servers. http://git-wip-us.apache.org/repos/asf/hadoop/blob/fc20129c/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt index 49fce24..932fee8 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt @@ -536,6 +536,9 @@ Release 2.7.0 - UNRELEASED HDFS-7640. Print NFS Client in the NFS log. (Brandon Li via wheat9) + HDFS-7623. Add htrace configuration properties to core-default.xml and + update user doc about how to enable htrace. (yliu) + OPTIMIZATIONS HDFS-7454. Reduce memory footprint for AclEntries in NameNode.
