[
https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927238#comment-15927238
]
Virajith Jalaparti edited comment on HDFS-11190 at 3/16/17 12:16 AM:
---------------------------------------------------------------------
Posting a patch that enables this functionality on the NN. The main changes
include:
a) Adding a {{LocatedBlockBuilder}} that is used to build {{LocatedBlocks}}.
b) Adding a {{ProvidedBlocksBuilder}} as a subclass of {{LocatedBlockBuilder}},
which is used to build {{LocatedBlocks}} for provided blocks -- the
{{ProvidedBlocksBuilder}} assigns a randomly chosen Datanode (which reports as
provided volume) as the location of the provided blocks.
c) Adding a {{ProvidedStorageMap}} which is used to keep track of datanodes
with provided volumes
d) Modifications to the {{BlockManager}} to use {{LocatedBlockBuilder}} to
build {{LocatedBlocks}} and use {{ProvidedStorageMap}} while processing reports
from datanodes.
Most of this code was contributed by [~chris.douglas].
was (Author: virajith):
Posting a patch that enables this functionality on the NN. The main changes
include:
a) Adding a {{LocatedBlockBuilder}} that is used to build {{LocatedBlocks}}.
b) Adding a {{ProvidedBlocksBuilder}} as a subclass of {{LocatedBlockBuilder}},
which is used to build {{LocatedBlocks}} for provided blocks -- the
{{ProvidedBlocksBuilder}} assigns a randomly chosen Datanode (which reports as
provided volume) as the location of the provided blocks.
c) Adding a {{ProvidedStorageMap}} which is used to keep track of datanodes
with provided volumes
d) Modifications to the {{BlockManager}} to use {{LocatedBlockBuilder}} to
build {{LocatedBlocks}} and use {{ProvidedStorageMap}} while processing reports
from datanodes.
> Namenode support for data stored in external stores.
> ----------------------------------------------------
>
> Key: HDFS-11190
> URL: https://issues.apache.org/jira/browse/HDFS-11190
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Virajith Jalaparti
> Attachments: HDFS-11190-HDFS-9806.001.patch
>
>
> The goal of this JIRA is to enable the Namenode to know about blocks that are
> in {{PROVIDED}} stores and are not necessarily stored on any Datanodes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
