[
https://issues.apache.org/jira/browse/HDFS-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142940#comment-14142940
]
Colin Patrick McCabe edited comment on HDFS-6881 at 9/22/14 6:29 AM:
---------------------------------------------------------------------
There are a lot of DistributedFileSystem operations that just involve making an
RPC to the NameNode or the DataNode via RPCv9. We currently never trace those,
unless there is already an active trace span. This patch changes this so that
RPCv9 calls will use the HTrace {{Sampler}} configured by the DFSClient to
decide whether to initiate a trace span when making an RPC.
This is done by adding a new method to the {{RpcInvocationHandler}} interface,
which is implemented by {{ProtobufRpcEngine#Invoker}}. This new method changes
the sampler that the invoker is using. The {{DFSClient}} uses it to tell the
proxy object it creates which HTrace sampler to use.
There a little bit of overlap between this patch and the patch in HDFS-7055
(Add tracing to DFSInputStream), since both patches rely on the
{{dfs.client.trace.sampler}} option being present. But both patches can
compile and be committed on their own.
I changed the trace span name format that we're using slightly. Formerly, we
would populate trace spans with names like
{{org.apache.hadoop.hdfs.protocol.ClientProtocol.getServerDefaults}}. This is
a rather long name, and I think {{ClientProtocol#getServerDefaults}} is just as
descriptive, and unlikely to lead to any confusion.
Requests from HTrace for a configuration key X now cause it to access
{{hadoop.htrace.X}} rather than {{hadoop.X}}. Hopefully, this will avoid
potential configuration key name conflicts for any possible HTrace
configuration key someone might want to implement. I also modified the class
search to prepend {{org.htrace.impl.}} to class names with no namespace given.
This should lead to shorter configurations, and also protect us against any
changes in HTrace in the future (like moving the span receivers to a different
namespace).
SpanReceiverHost now sets {{hadoop.htrace.local-file-span-receiver.path}} to
{{/tmp/$PID}} if it is not already set. This makes it a lot easier to test
tracing, since you can just feed all daemons the same config and get nice trace
files in {{/tmp}}.
was (Author: cmccabe):
There are a lot of DistributedFileSystem operations that just involve making an
RPC to the NameNode or the DataNode via RPCv9. We currently never trace those,
unless there is already an active trace span. This patch changes this so that
RPCv9 calls will use the HTrace {{Sampler}} configured by the DFSClient to
decide whether to initiate a trace span when making an RPC.
This is done by adding a new method to the {{RpcInvocationHandler}} interface,
which is implemented by {{ProtobufRpcEngine#Invoker}}. This new method changes
the sampler that the invoker is using. The {{DFSClient}} uses it to tell the
proxy object it creates which HTrace sampler to use.
There a little bit of overlap between this patch and the patch in HDFS-7055
(Add tracing to DFSInputStream), since both patches rely on the
{{dfs.client.trace.sampler}} option being present. But both patches can
compile and be committed on their own.
I changed the trace span name format that we're using slightly. Formerly, we
would populate trace spans with names like
{{org.apache.hadoop.hdfs.protocol.ClientProtocol.getServerDefaults}}. This is
a rather long name, and I think {{ClientProtocol#getServerDefaults}} is just as
descriptive, and unlikely to lead to any confusion.
Requests from HTrace for a configuration key X now cause it to access
{{hadoop.htrace.X}} rather than {{hadoop.X}}. Hopefully, this will avoid
potential configuration key name conflicts for any possible HTrace
configuration key someone might want to implement. I also modified the class
search to prepend {{org.htrace.impl.}} to class names with no namespace given.
This should lead to shorter configurations, and also protect us against any
changes in HTrace in the future (like moving the span receivers to a different
namespace).
SpanReceiverHost now sets {{hadoop.htrace.local-file-span-receiver.path}} to
{{/tmp/\$PID}} if it is not already set. This makes it a lot easier to test
tracing, since you can just feed all daemons the same config and get nice trace
files in {[/tmp}}.
> The DFSClient should use the sampler to determine whether to initiate trace
> spans when making RPCv9 calls to the NN and DN
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-6881
> URL: https://issues.apache.org/jira/browse/HDFS-6881
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Affects Versions: 2.6.0
> Reporter: Masatake Iwasaki
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-6881.002.patch
>
>
> The DFSClient should use the configred HTrace sampler to determine whether to
> initiate trace spans when making RPCv9 calls to the NN and DN.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
