[
https://issues.apache.org/jira/browse/HDFS-12423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162656#comment-16162656
]
Weiwei Yang commented on HDFS-12423:
------------------------------------
Hi [~linyiqun]
Thanks for the patch, it looks good to me. Just one suggestion, since
{{ContainerStatus}} are sorted, can we stop looking for next entry if we met a
container has 0 number of pending deletion blocks? So topN policy returns us at
most N containers, but it could be any size <N. Please let me know if this
makes sense. Thanks
> Ozone: TopN container choosing policy should ignore unnecessary containers
> --------------------------------------------------------------------------
>
> Key: HDFS-12423
> URL: https://issues.apache.org/jira/browse/HDFS-12423
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Labels: ozoneMerge
> Attachments: HDFS-12423-HDFS-7240.001.patch
>
>
> TopN container choosing policy should ignore unnecessary containers.
> Currently TopN policy selects specified count of containers but not check the
> number of pending deletion blocks. So there is a chance there will be some
> unnecessary containers being chosen. That is say we will choose some
> containers that don't include any pending deletion blocks. The related output:
> {noformat}
> 17/09/11 02:58:30 INFO impl.TopNOrderedContainerDeletionChoosingPolicy:
> Select container c7a85e6e-3528-45d1-8063-cd7fec114545 for block deletion,
> pending deletion blocks num: 1.
> 17/09/11 02:58:30 INFO impl.TopNOrderedContainerDeletionChoosingPolicy:
> Select container 1d163265-8d47-4ed3-845f-f7d3eb569b83 for block deletion,
> pending deletion blocks num: 0.
> 17/09/11 02:58:30 INFO impl.TopNOrderedContainerDeletionChoosingPolicy:
> Select container 21017018-be32-44e6-9f62-e7fc9f6a6021 for block deletion,
> pending deletion blocks num: 0.
> 17/09/11 02:58:30 INFO impl.TopNOrderedContainerDeletionChoosingPolicy:
> Select container 4cccadc8-ef5e-466d-bd9e-5b9705f8748c for block deletion,
> pending deletion blocks num: 0.
> 17/09/11 02:58:30 INFO impl.TopNOrderedContainerDeletionChoosingPolicy:
> Select container 55b11be9-f16f-4620-a310-07c9be3bbfee for block deletion,
> pending deletion blocks num: 0.
> {noformat}
> We should ignore these containers which pending deletion blocks num is 0,
> this can reduce some unnecessary block-deletion iterations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
