[
https://issues.apache.org/jira/browse/HDFS-16163?focusedWorklogId=637254&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-637254
]
ASF GitHub Bot logged work on HDFS-16163:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Aug/21 06:28
Start Date: 12/Aug/21 06:28
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3296:
URL: https://github.com/apache/hadoop/pull/3296#issuecomment-897382966
Thanks for taking look @ferhui. Yes this is perf optimization however I just
came across this while going through an non-relevant issue in mover because of
which I was comparing all the diff b/ Hadoop 2.10 and latest 3.3 release. That
original issue is still under investigation but while looking into all
differences, I came across HDFS-11164 and realized that just to update/add one
single key->value pair, we are locking entire map and hence I thought of fixing
this. I just tested this locally for it's sanity and correctness but
unfortunately I don't have perf results because it was simple test.
The other way to look into this is with simplicity: Unless we are updating
multiple entries in single batch, we don't need to lock entire map and for
single entry update, we can rather use fine-grained ConcurrentHashMap provided
utilities.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 637254)
Time Spent: 0.5h (was: 20m)
> Avoid locking entire blockPinningFailures map
> ---------------------------------------------
>
> Key: HDFS-16163
> URL: https://issues.apache.org/jira/browse/HDFS-16163
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In order for mover to exclude pinned blocks in subsequent iteration, we try
> to put pinned blocks in a map of blockIds to set of Datanode sources.
> However, while updating an entry of this map, we don't need to lock the
> entire map. We can use fine-grained concurrency here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
