[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12354:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: HDFS-7240
Status: Resolved (was: Patch Available)
I've committed this to the feature branch, thanks [~linyiqun] for your
contribution.
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Fix For: HDFS-7240
>
> Attachments: HDFS-12354-HDFS-7240.001.patch,
> HDFS-12354-HDFS-7240.002.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12354:
-
Attachment: HDFS-12354-HDFS-7240.002.patch
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Attachments: HDFS-12354-HDFS-7240.001.patch,
> HDFS-12354-HDFS-7240.002.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12354:
-
Attachment: (was: HDFS-12354-HDFS-7240.002.patch)
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Attachments: HDFS-12354-HDFS-7240.001.patch,
> HDFS-12354-HDFS-7240.002.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12354:
-
Attachment: HDFS-12354-HDFS-7240.002.patch
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Attachments: HDFS-12354-HDFS-7240.001.patch,
> HDFS-12354-HDFS-7240.002.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12354:
-
Attachment: HDFS-12354-HDFS-7240.001.patch
Attach the initial patch. Additionally seperating the class {{ContainerStatus}}
into the independent class so that it can be used by other class.
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Attachments: HDFS-12354-HDFS-7240.001.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12354:
-
Status: Patch Available (was: Open)
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Yiqun Lin
> Attachments: HDFS-12354-HDFS-7240.001.patch
>
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12354) Ozone: Shuffle container list for datanode BlockDeletingService
[
https://issues.apache.org/jira/browse/HDFS-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12354:
---
Summary: Ozone: Shuffle container list for datanode BlockDeletingService
(was: Shuffle container list for datanode BlockDeletingService)
> Ozone: Shuffle container list for datanode BlockDeletingService
> ---
>
> Key: HDFS-12354
> URL: https://issues.apache.org/jira/browse/HDFS-12354
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ozone
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>
> {{BlockDeletingService}} is a per-datanode container block deleting service
> takes in charge of the "real" deletion of ozone blocks. It spawns a worker
> thread per container and delete blocks/chunks from disk as background
> threads. The number of threads currently is throttled by
> {{ozone.block.deleting.container.limit.per.interval}}, but there is a
> potential problem. Containers are sorted so it always fetch same of
> containers, we need to fix this by creating an API in
> {{ContainerManagerImpl}} to get a shuffled list of containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
