[
https://issues.apache.org/jira/browse/HDFS-11001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609004#comment-15609004
]
Anu Engineer commented on HDFS-11001:
-------------------------------------
bq. I mean the emptyList does not need to be constructed each time getCommand()
return a non-empty list for a datanode.
Just want to make sure that we both are on the same page. The way we intend to
use this list is that some thread will come and pick up this list and process
commands using a pool of threads. At that point this list has a bunch of work
items that gets executed. While this is going on it is possible that we will be
able to come around and check for more commands. This time we again read the
list of commands that is waiting for execution.
if we have the same static list, then the list becomes shared and we would need
a lock to protect the list. In the current code where we allocate a new list,
each worker thread which picks up the list is the owner of that list and only
owner will modify the list. Hence we can get away from having to coordinate
access.
Due to this locking concern, I thought it is better to allocate the emptyList
rather than have a static list.
bq. If we don't expect any operation at the datanode side for NullCommand, can
we remove NullCommand to save a RPC roundtrip?
Right now we have this command because this will allow us build some metrics in
the Datanode which can be read by SCM later to decide on which datanode it
should allocate the containers. You are right, we could monitor other RPCs as
well as NullCommand -- But I was worried all other commands will have a payload
and marshalling/unmarshalling costs will create an unnecessary variability in
the average time calculations. I am going to file a work item to explore if we
can remove this in future. As we fine tune SCM's datanode allocation algorithm,
we might be able to remove this completely.
> Ozone:SCM: Add support for registerNode in SCM
> ----------------------------------------------
>
> Key: HDFS-11001
> URL: https://issues.apache.org/jira/browse/HDFS-11001
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Anu Engineer
> Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-11001-HDFS-7240.001.patch,
> HDFS-11001-HDFS-7240.002.patch
>
>
> Adds support for a datanode registration. Right now SCM relies on Namenode
> for the datanode registration. With this change we will be able to run SCM
> independently if needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
