[
https://issues.apache.org/jira/browse/HDFS-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237350#comment-16237350
]
Weiwei Yang commented on HDFS-12443:
------------------------------------
Thanks [~linyiqun], the patch looks really good to me, it addressed most of the
problems. There are just a few comments,
*SCMBlockDeletingService*
(1) line 90: the calculation of the throttling limit in the patch might not
work well. {{OZONE_KEY_DELETING_LIMIT_PER_TASK}} is used by KSM to determine
how many keys to delete per interval, we cannot use it to estimate block
number. Depending on the size of an object, number of blocks of a key might be
vary. Second thought, I think we can use
{{OZONE_BLOCK_DELETING_CONTAINER_LIMIT_PER_INTERVAL}} for calculation. This
property is used in DN to decide how many containers to do concurrent delete in
each interval, with default value 10. I think it is good to use this value
multiply by a factor 2 (to ensure DN not wait for orders) for
{{DatanodeDeletedBlockTransactions}}'s entry limit. That means it allows at
most 10 containers per DN. *Note*, TXs in DelLog may have dup containers,
{code}
dn0 : {txID:1, containerName: c1, [Blk...]}
dn0 : {txID:2, containerName: c2, [Blk...]}
dn0 : {txID:3, containerName: c1, [Blk...]} // same container to txid=1
...
{code}
when checking the limit, we need to make sure we are checking the number of
containers of all TXs. Sorry I did not figure this out in our last discussion.
(2) line 133: maybe we should replace getAllNodes with get all healthy nodes by
getNodes(NodeState.HEALTHY).
*DatanodeDeletedBlockTransactions*
line 46: consider to replace hashmap to {{Multimap}}?
*TestStorageContainerManager*
line 282, it sleeps a fixed length of time (pretty long), this might be flaky
in jenkins. Can we verify the deletion throttling by capture the
{{DeleteBlocksCommand}} SCM sent to DN and verify its size is expected?
Thanks!
> Ozone: Improve SCM block deletion throttling algorithm
> -------------------------------------------------------
>
> Key: HDFS-12443
> URL: https://issues.apache.org/jira/browse/HDFS-12443
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, scm
> Reporter: Weiwei Yang
> Assignee: Yiqun Lin
> Priority: Major
> Labels: OzonePostMerge
> Attachments: HDFS-12443-HDFS-7240.001.patch,
> HDFS-12443-HDFS-7240.002.patch, HDFS-12443-HDFS-7240.002.patch,
> HDFS-12443-HDFS-7240.003.patch, HDFS-12443-SCM-blockdeletion-throttle.pdf
>
>
> Currently SCM scans delLog to send deletion transactions to datanode
> periodically, the throttling algorithm is simple, it scans at most
> {{BLOCK_DELETE_TX_PER_REQUEST_LIMIT}} (by default 50) at a time. This is
> non-optimal, worst case it might cache 50 TXs for 50 different DNs so each DN
> will only get 1 TX to proceed in an interval, this will make the deletion
> slow. An improvement to this is to make this throttling by datanode, e.g 50
> TXs per datanode per interval.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
