[
https://issues.apache.org/jira/browse/HDFS-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543484#comment-15543484
]
Ming Ma edited comment on HDFS-10629 at 10/3/16 9:33 PM:
---------------------------------------------------------
Thanks [~jakace] for the explanation and new patch.
* Should rename method do some validation to make sure src ns != destination ns?
* Router#startTime and Router#routerId not used. Let us add them later when
they are required. There are other unused methods in other files. Can you check
if they are needed?
* It seems abandonBlock just needs to refer to one name service based on the
block id, thus can call invokeSingle?
* There are still other javadoc and code mismatch in several files.
* failIfLocked isn't used. What is for?
bq. The status report is used by the NN heartbeat service to register/discover
a NN and store it's state in the store.
Got it. How about dynamic stats such as numOfFiles which are only used by
rebalancer? We could consider adding dynamic stats later when we agree to have
rebalancer fetch such stats from store/router instead of getting them directly
from actual namenodes.
bq. Other APIs such as mkdir were modfiied to check all locations to ensure the
directory doesn't exist before permitting a create.
* Some test case to cover multiple locations will be useful.
* Wonder why mkdirs check all locations during validation, while only use the
first location for the actual mkdirs call.
* It will be useful to document the scenarios for multiple destination, what
specific mount table settings this will cover, nfly, mergeFs, etc.
* It will have extra RPC calls for all scenarios? It might be useful to check
PathLocation#hasMultipleDestinations first. If there is only one destination,
getFileInfo can be skipped.
was (Author: mingma):
Thanks [~jakace] for the explanation and new patch.
* Should rename method do some validation to make sure src ns != destination ns?
* Router#startTime and Router#routerId not used. Let us add them later when
they are required. There are other unused methods in other files. Can you check
if they are needed?
* It seems abandonBlock just needs to refer to one name service based on the
block id, thus can call invokeSingle?
* There are still other javadoc and code mismatch in several files.
* failIfLocked isn't used. What is for?
bq. The status report is used by the NN heartbeat service to register/discover
a NN and store it's state in the store.
Got it. How about dynamic stats such as numOfFiles which are only used by
rebalancer? We could consider adding dynamic stats later when we agree to have
rebalancer fetch such stats from store/router instead of getting them directly
from actual namenodes.
bq. Other APIs such as mkdir were modfiied to check all locations to ensure the
directory doesn't exist before permitting a create.
Wonder why mkdirs check all locations during validation, while only use the
first location for the actual mkdirs call.
Can you add test cases to cover this scenario?
It will be useful to document the scenarios for multiple destination, what
specific mount table settings this will cover, nfly, mergeFs, etc.
It will have extra RPC calls for all scenarios? It might be useful to check
PathLocation#hasMultipleDestinations first. If there is only one destination,
getFileInfo can be skipped.
> Federation Router
> -----------------
>
> Key: HDFS-10629
> URL: https://issues.apache.org/jira/browse/HDFS-10629
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Reporter: Inigo Goiri
> Assignee: Jason Kace
> Attachments: HDFS-10629-HDFS-10467-002.patch,
> HDFS-10629-HDFS-10467-003.patch, HDFS-10629-HDFS-10467-004.patch,
> HDFS-10629.000.patch, HDFS-10629.001.patch
>
>
> Component that routes calls from the clients to the right Namespace. It
> implements {{ClientProtocol}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
