[
https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196632#comment-14196632
]
Maysam Yabandeh commented on HDFS-6982:
---------------------------------------
Thanks [~wheat9] for the comment. Let me explain how the buckets are employed
in the rolling window implementation.
The rolling window can compute the total value of the event in the past period
of time, lets say a minute. The last minute is divided to multiple buckets
where buckets are placed in a ring. The total number of the events in the last
minute the sum of the values of the buckets.
As the time rolls forward a bucket of the last time period is reused for the
current time period. Lets says that the bucket that we are writing to, was used
to accumulate the events of 67 seconds ago. Before we start adding events to
that (which will be used to compute the event of the last 60 seconds) we need
to zero the content of the bucket. Whether the bucket is stale or not is
determined by #isStaleNow method.
Considering the above explanation let me know if the current implementation of
zeroing stale buckets makes sense to you.
> nntop: top-like tool for name node users
> -----------------------------------------
>
> Key: HDFS-6982
> URL: https://issues.apache.org/jira/browse/HDFS-6982
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Maysam Yabandeh
> Assignee: Maysam Yabandeh
> Attachments: HDFS-6982.patch, HDFS-6982.v2.patch, HDFS-6982.v3.patch,
> HDFS-6982.v4.patch, HDFS-6982.v5.patch, HDFS-6982.v6.patch,
> nntop-design-v1.pdf
>
>
> In this jira we motivate the need for nntop, a tool that, similarly to what
> top does in Linux, gives the list of top users of the HDFS name node and
> gives insight about which users are sending majority of each traffic type to
> the name node. This information turns out to be the most critical when the
> name node is under pressure and the HDFS admin needs to know which user is
> hammering the name node and with what kind of requests. Here we present the
> design of nntop which has been in production at Twitter in the past 10
> months. nntop proved to have low cpu overhead (< 2% in a cluster of 4K
> nodes), low memory footprint (less than a few MB), and quite efficient for
> the write path (only two hash lookup for updating a metric).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
