[
https://issues.apache.org/jira/browse/HDFS-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510497#comment-14510497
]
Nick Dimiduk commented on HDFS-8213:
------------------------------------
bq. Nick Dimiduk, what if I want to trace an HBase PUT all the way through the
system? You're saying that the HBase client can't activate tracing on its own,
so I have to make code changes to the process doing the PUT (i.e. the user of
the HBase client) in order to get that info? Seems like a limitation.
I think your characterization is accurate. HBase client is a library embedded
in applications. The client doesn't enable tracing, the application does. HBase
client faithfully extends the calling context through the system by enabling
tracing on requests for which the application has initialized a trace request.
It delegates responsibility for receiver registration to that calling context
as well. For applications that exist before tracing, yes, this implies code
changes to add support for the new feature.
For instance, I have a main program and provide an option for enabling tracing
on some percentage of requests. This context is set above the hbase client,
which simply honors the context in which it is run. I imaging FsShell would
work this way, enabling a trace for a specific invocation of a command.
Phoenix's integration works this way too, with a session flag for enabling or
disabling tracing managed from the connection or user session. From screen
shots I've seen, I believe Cassandra's tracing feature works this way as well,
though I don't know if it's using HTrace.
In a way, the application that embeds the hbase client must also participate in
the cluster tracing configuration. It must configure its span receiver to send
spends to the same store as the rest of the cluster for the full context to be
available.
bq. {{hadoop.htrace.span.receiver.classes}} would be set in the NN and DN
configuration files, but not in the ones targetted towards the DFSClients. Does
Ambari do something similar?
Last I looked, Ambari appeared to drop an hdfs-site.xml file into
/etc/hbase/conf, which is identical to the one in /etc/hadoop/conf. I don't
know if this is different from the configs used by the daemon processes.
bq. There are hundreds or maybe even thousands of programs that use the HDFS
client. It's not practical to modify them all to run Trace#addSpanReceiver. In
some cases the programs that use HDFS are even proprietary or customer programs
where we don't have access to the source code.
I'm not sure where the "final responsibility" for instantiating the snap
receiver is supposed to lie; I always thought it was with process. HDFSClient
seems like an arbitrary place. Should HBase and Accumulo clients be providing
the same? Phoenix? My perspective is that there's a singleton list of instances
for the process, so it's the responsibility of the main(), not one of the
libraries it's imported. So yes, I am suggesting that it is the responsibility
of those hundreds or thousands of programs who have all written a main method
to setup span receivers for their process in order to take advantage of
tracing. The approach you describe hadn't occurred to me until just now.
> DFSClient should use hdfs.client.htrace HTrace configuration prefix rather
> than hadoop.htrace
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-8213
> URL: https://issues.apache.org/jira/browse/HDFS-8213
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.0
> Reporter: Billie Rinaldi
> Assignee: Colin Patrick McCabe
> Priority: Critical
> Attachments: HDFS-8213.001.patch
>
>
> DFSClient initializing SpanReceivers is a problem for Accumulo, which manages
> SpanReceivers through its own configuration. This results in the same
> receivers being registered multiple times and spans being delivered more than
> once. The documentation says SpanReceiverHost.getInstance should be issued
> once per process, so there is no expectation that DFSClient should do this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
