[
https://issues.apache.org/jira/browse/HDFS-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Becker updated HDFS-14069:
--------------------------------
Description:
Currently, we only provide "minLiveReplicas" per DN that is being
decommissioned, this is not enough info because it is difficult to determine
which blocks are on their last replica. We have two design options:
# Add it to the existing report, on top of minLiveReplicas
** Advantages:
*** Initial debugging information would be more thorough
*** Easier initial implementation
** Disadvantages:
*** Add load to normal NN operation by checking every time a DN is
decommissioned
*** More difficult to add debugging information later on
# Create a new api for querying more detailed info about one DN
** Advantages:
*** We wouldnt be adding more load to the NN in normal operation
*** Much easier to extend in the future with more info
** Disadvantages:
*** Getting the info on demand for this case will be much more expensive
actually, cause we will have to find all the blocks on that DN, and then go
through all the blocks again and count how many replicas we have etc.
was:
Currently, we only provide "minLiveReplicas" per DN that is being
decommissioned, this is not enough info because it is difficult to determine
which blocks are on their last replica. We have two design options:
# Add it to the existing report, on top of minLiveReplicas
** Advantages:
*** Initial debugging information would be more thorough
** Disadvantages:
*** Add load to normal NN operation by checking every time a DN is
decommissioned
# Create a new api for querying more detailed info about one DN
** Advantages:
*** We wouldnt be adding more load to the NN in normal operation
*** Much easier to extend in the future with more info
** Disadvantages:
*** Getting the info on demand for this case will be much more expensive
actually, cause we will have to find all the blocks on that DN, and then go
through all the blocks again and count how many replicas we have etc.
> Better debuggability for datanode decommissioning
> -------------------------------------------------
>
> Key: HDFS-14069
> URL: https://issues.apache.org/jira/browse/HDFS-14069
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs, namenode
> Reporter: Danny Becker
> Priority: Major
> Attachments: HDFS-14069.000.patch
>
>
> Currently, we only provide "minLiveReplicas" per DN that is being
> decommissioned, this is not enough info because it is difficult to determine
> which blocks are on their last replica. We have two design options:
> # Add it to the existing report, on top of minLiveReplicas
> ** Advantages:
> *** Initial debugging information would be more thorough
> *** Easier initial implementation
> ** Disadvantages:
> *** Add load to normal NN operation by checking every time a DN is
> decommissioned
> *** More difficult to add debugging information later on
> # Create a new api for querying more detailed info about one DN
> ** Advantages:
> *** We wouldnt be adding more load to the NN in normal operation
> *** Much easier to extend in the future with more info
> ** Disadvantages:
> *** Getting the info on demand for this case will be much more expensive
> actually, cause we will have to find all the blocks on that DN, and then go
> through all the blocks again and count how many replicas we have etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
