[
https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730235#comment-14730235
]
stack commented on HBASE-12911:
-------------------------------
Lets figure 'tag.' prefix. I don't think it in any other hbase bean.
bq. I'm not sure what you have in mind with this one.
(Our client has way too many threads) NVM. Its probalbly info available in the
jvm bean that is adjacent to 'hadoop'
bq. The goal is to have a bean for each host the client is sending an RPC to.
Are they aggregated?
Can we stamp the remote server beans with their ServerName? (What happens
currently when a server restarts?)
bq. My open questions are around expiring old hosts when they go away, and
about aggregating this host-level information at the connection level (or if
that's even useful, given the drastic difference between our various RPC call
durations and sizes). We could also explore other aggregations, like per region
or per table, but that requires a bit more unpacking of the IPC layer than I've
tackled just yet.
Sounds like you can't remove beans without restarting metrics (per Elliott's
previous, recent experience). Do it on a period?
The per server stuff is cool but what happens when hundreds of remote servers?
Some basic aggregation is needed I'd say.
Would suggest pick small subset of most useful metrics and lets get that in.
Its great.
> Client-side metrics
> -------------------
>
> Key: HBASE-12911
> URL: https://issues.apache.org/jira/browse/HBASE-12911
> Project: HBase
> Issue Type: Brainstorming
> Components: Client, Performance, Usability
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch, client metrics RS-Master.jpg,
> client metrics client.jpg, connection attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add
> some kind of metrics collection end up wrapping Table method invocations with
> {{System.currentTimeMillis()}}. For a crude example of this, have a look at
> what I did in {{PerformanceEvaluation}} for exposing requests latencies up to
> {{IntegrationTestRegionReplicaPerf}}. The client is quite complex, there's a
> lot going on under the hood that is impossible to see right now without a
> profiler. Being a crucial part of the performance of this distributed system,
> we should have deeper visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice
> because the client is often embedded as a library in a user's application. We
> should have integration with our metrics tools so that, i.e., a client
> embedded in a coprocessor can report metrics through the usual RS channels,
> or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out
> of the box we'd include a hadoop-metrics implementation and one other,
> possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
