[
https://issues.apache.org/jira/browse/HDFS-13347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414263#comment-16414263
]
Íñigo Goiri commented on HDFS-13347:
------------------------------------
Thanks [~linyiqun] for the review. A few comments:
* Regarding the fixed 10, I made it a configurable value
* I had to cahnge {{requireResponse}}, because the UI was getting stuck here
when we had a Namenode in safe mode (throwing an exception). Right now, it will
just collect the data from the subclusters that are running with no problem.
* Fixed the {{Gettings}} typo.
* I added a unit test for this; I wanted to add it to the metrics package but I
needed all the MiniRouterDFSCluster to query the DNs; I added it to the
{{TestRouterRpc}}.
[~shahrs87], I think there was some effort to cache this information in the
Namenode (cannot find the JIRA right now).
In our particular, we are setting this to 10 seconds (and now is configurable
so it should be fine); not sure if there is a way in JMX to pass parameters
though.
In a related note, I'm considering moving this into RouterRpcServer and cache
even the ClientProtocol calls.
In that case, we can support parameters.
My only concern is the DN[] into JSON conversion; this was what actually
triggered this effort as we found that we had lots of memory allocated by these
JSON strings.
I'll be testing this and see how the memory allocation goes.
> RBF: Cache datanode reports
> ---------------------------
>
> Key: HDFS-13347
> URL: https://issues.apache.org/jira/browse/HDFS-13347
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Íñigo Goiri
> Priority: Minor
> Attachments: HDFS-13347.000.patch, HDFS-13347.001.patch
>
>
> Getting the datanode reports is an expensive operation and can be executed
> very frequently by the UI and watchdogs. We should cache this information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
