[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) Thanks [~anu] for all the review, I've committed this to the feature branch. Once HDFS-12235 is done, we can start end-to-end testing for key deletes :) > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Fix For: HDFS-7240 > > Attachments: Block delete via HB between SCM and DN.png, Block delete > via HB between SCM and DN v2.png, HDFS-12282-HDFS-7240.001.patch, > HDFS-12282-HDFS-7240.002.patch, HDFS-12282-HDFS-7240.003.patch, > HDFS-12282-HDFS-7240.004.patch, HDFS-12282-HDFS-7240.005.patch, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: HDFS-12282-HDFS-7240.005.patch > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, Block delete > via HB between SCM and DN v2.png, HDFS-12282-HDFS-7240.001.patch, > HDFS-12282-HDFS-7240.002.patch, HDFS-12282-HDFS-7240.003.patch, > HDFS-12282-HDFS-7240.004.patch, HDFS-12282-HDFS-7240.005.patch, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[
https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12282:
---
Attachment: Block delete via HB between SCM and DN v2.png
Per discussion with [~anu], uploaded v4 patch to move block deletion logic out
of HB process. Created a new background service {{SCMBlockDeletingService}}
scans delTX log periodically and caches commands in the {{CommandQueue}}, this
way we keep HB lightweight as before. Please see more from [^Block delete via
HB between SCM and DN v2.png]. Thanks
> Ozone: DeleteKey-4: Block delete via HB between SCM and DN
> --
>
> Key: HDFS-12282
> URL: https://issues.apache.org/jira/browse/HDFS-12282
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone, scm
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: Block delete via HB between SCM and DN.png, Block delete
> via HB between SCM and DN v2.png, HDFS-12282-HDFS-7240.001.patch,
> HDFS-12282-HDFS-7240.002.patch, HDFS-12282-HDFS-7240.003.patch,
> HDFS-12282-HDFS-7240.004.patch, HDFS-12282.WIP.patch
>
>
> This is the task 3 in the design doc, implements the SCM to datanode
> interactions. Including
> # SCM sends block deletion message via HB to datanode
> # datanode changes block state to deleting when processes the HB response
> # datanode sends deletion ACKs back to SCM
> # SCM handles ACKs and removes blocks in DB
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: HDFS-12282-HDFS-7240.004.patch > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282-HDFS-7240.001.patch, HDFS-12282-HDFS-7240.002.patch, > HDFS-12282-HDFS-7240.003.patch, HDFS-12282-HDFS-7240.004.patch, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: HDFS-12282-HDFS-7240.003.patch > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282-HDFS-7240.001.patch, HDFS-12282-HDFS-7240.002.patch, > HDFS-12282-HDFS-7240.003.patch, HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: HDFS-12282-HDFS-7240.002.patch > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282-HDFS-7240.001.patch, HDFS-12282-HDFS-7240.002.patch, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: HDFS-12282-HDFS-7240.001.patch > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282-HDFS-7240.001.patch, HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Status: Patch Available (was: In Progress) > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282-HDFS-7240.001.patch, HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Summary: Ozone: DeleteKey-4: Block delete via HB between SCM and DN (was: Ozone: DeleteKey-4: SCM periodically sends block deletion message to datanode via HB and handles response) > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
[ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12282: --- Attachment: Block delete via HB between SCM and DN.png > Ozone: DeleteKey-4: Block delete via HB between SCM and DN > -- > > Key: HDFS-12282 > URL: https://issues.apache.org/jira/browse/HDFS-12282 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode, ozone, scm >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: Block delete via HB between SCM and DN.png, > HDFS-12282.WIP.patch > > > This is the task 3 in the design doc, implements the SCM to datanode > interactions. Including > # SCM sends block deletion message via HB to datanode > # datanode changes block state to deleting when processes the HB response > # datanode sends deletion ACKs back to SCM > # SCM handles ACKs and removes blocks in DB -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
