[
https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338111#comment-16338111
]
Anu Engineer commented on HDFS-13055:
-------------------------------------
[~ajayydv] Thanks for the patch. It looks good overall. I have some high-level
questions and some low-level comments. I will skip the low-level comments so
that we can focus on the high-level questions first.
* The first and foremost, What issue are we trying to address here with this
patch?
* I am concerned that we are adding more load to HB processing, there are
patches like HDFS-9239 which tried to address the issue of HB processing
slowness. Aren't we adding more load to Namenode -- Memory/CPU costs to
Namenode?
* All of this information can be retained in data node and pulled in only when
needed, that is we can make this visible in Datanode JMX, and probably write a
simple page on Namenode that has a Javascript that connects to data nodes
directly and compute this info if needed. What is the rationale for aggregating
this on Namenode all the time? Why a push and not a pull when we need this
data?
> Aggregate usage statistics from datanodes
> -----------------------------------------
>
> Key: HDFS-13055
> URL: https://issues.apache.org/jira/browse/HDFS-13055
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Ajay Kumar
> Assignee: Ajay Kumar
> Priority: Major
> Attachments: HDFS-13055.001.patch
>
>
> We collect variety of statistics in DataNodes and expose them via JMX.
> Aggregating some of the high level statistics which we are already collecting
> in {{DataNodeMetrics}} (like bytesRead,bytesWritten etc) over a configurable
> time window will create a central repository accessible via JMX and UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
