[
https://issues.apache.org/jira/browse/HADOOP-10802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated HADOOP-10802:
---------------------------------
Attachment: HADOOP-10802.WIP.patch
Attaching a WIP patch.
Based on [~tucu00]'s suggestion, The {{ValueQueue}} class now allows clients to
set an implementation of {{QueueMetrics}}. The VQ will then invoke appropriate
methods on implementation.
This is done, so that on the KMS server side, we can reuse the codehale metrics
framework and on the Client (Namenode) side, we can probably use the hadoop
metrics2 framework to implement the ValueQueue.
[~andrew.wang], Thanks for the suggestions. I was just wondering though, if it
makes sense to measure the "hotness" of keys. If the intention is to monitor
the metrics and use it to dynamically tune the cache, considering the fact that
we do not have per key tunable settings, knowing that a particular key is "hot"
might not help us much. In any case, I have added a method {{markKeyReuest()}}
that takes a keyName. It then takes the {{hashcode()}} of the key and adds it
to a Histogram. This way, over time, we can know if there is a skew in the
distribution of requests.
> Add metrics for KMS client and server encrypted key caches
> ----------------------------------------------------------
>
> Key: HADOOP-10802
> URL: https://issues.apache.org/jira/browse/HADOOP-10802
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Andrew Wang
> Assignee: Arun Suresh
> Attachments: HADOOP-10802.WIP.patch
>
>
> HADOOP-10720 is adding KMS server and client caches for encrypted keys for
> performance reasons. It would be good to add metrics to make sure that the
> cache is working as expected, and to inform future dynamic cache sizing and
> refilling policies.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
