[
https://issues.apache.org/jira/browse/HDFS-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537720#comment-16537720
]
Virajith Jalaparti edited comment on HDFS-13421 at 7/9/18 10:35 PM:
--------------------------------------------------------------------
bq. So we chose a design where the NN is quite explicit in what needs to happen
the DN is a dumb worker.
Sure, I am fine with concatenating multiple blocks on the DN. However, my
concern with the current implementation is that it reads all blocks into
in-memory buffers and writes them to the remote. This should be done without
having to read all the data into memory.
bq. If you think it's best, I can try to add some of the NN side things.
I meant to connect {{SyncServiceSatisfierDatanodeWorker#processSyncTasks}} with
{{BPOfferService#processCommandFromActive}} and complete the DN side of the
implementation in this JIRA. The NN functionality can be handled as a separate
sub-task.
was (Author: virajith):
bq. So we chose a design where the NN is quite explicit in what needs to happen
the DN is a dumb worker.
Sure, I am fine with concatenating multiple blocks on the DN. However, my
concern with the current implementation is that it reads all blocks into
in-memory buffers and writes them to the remote. This should be done without
having to read all the data into memory.
bq. If you think it's best, I can try to add some of the NN side things.
I meant to connect {{SyncServiceSatisfierDatanodeWorker#processSyncTasks}} with
{{BPOfferService#processCommandFromActor}} and complete the DN side of the
implementation in this JIRA. The NN functionality can be handled as a separate
sub-task.
> [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode
> -----------------------------------------------------------
>
> Key: HDFS-13421
> URL: https://issues.apache.org/jira/browse/HDFS-13421
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ewan Higgs
> Assignee: Ewan Higgs
> Priority: Major
> Attachments: HDFS-13421-HDFS-12090.001.patch,
> HDFS-13421-HDFS-12090.002.patch, HDFS-13421-HDFS-12090.003.patch
>
>
> HDFS-13310 introduces an API for DNA_BACKUP. Here, we implement DNA_BACKUP
> command in Datanode.
> These have been broken up to make reviewing it easier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
