Stephen O'Donnell created HDFS-15150:
----------------------------------------
Summary: Introduce read write lock to Datanode
Key: HDFS-15150
URL: https://issues.apache.org/jira/browse/HDFS-15150
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 3.3.0
Reporter: Stephen O'Donnell
Assignee: Stephen O'Donnell
HDFS-9668 pointed out the issues around the DN lock being a point of contention
some time ago, but that Jira went in a direction of creating a new FSDataset
implementation which is very risky, and activity on the Jira has stalled for a
few years now.
I feel there could be significant gains by moving to a ReentrantReadWrite lock
within the DN. The current implementation is simply a ReentrantLock so any
locker blocks all others.
Once place I think a read lock would benefit us significantly, is when the DN
is serving a lot of small blocks and there are jobs which perform a lot of
reads. The start of reading any blocks right now takes the lock, but if we
moved this to a read lock, many reads could do this at the same time.
The first conservative step, would be to change the current lock and then make
all accesses to it obtain the write lock. That way, we should keep the current
behaviour and then we can selectively move some lock accesses to the readlock
in separate Jiras.
I would appreciate any thoughts on this, and also if anyone has attempted it
before and found any blockers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
