[
https://issues.apache.org/jira/browse/HDFS-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016054#comment-13016054
]
Todd Lipcon commented on HDFS-1148:
-----------------------------------
bq. Todd, I want to understand what methods have lot contention
I actually don't remember anymore - this was a while back that I saw this, and
only once I added HDFS-941. Since it was a read workload, it makes sense that
it would be getBlockInputStream, metaFileExists, getVisibleLength, and
getMetaDataInputStream.
bq. I am not sure what you mean mostly uncontended, because I understand the
problem description as there are lot of contentions
Sorry, what I meant here is that, once it's converted to read-write lock, there
is very little contention for the exclusive (write) lock. It's very rare to
write small blocks, whereas small frequent reads come often from applications
like HBase. So, we mostly see lots of readers and only the occasional writer.
> Convert FSDataset to ReadWriteLock
> ----------------------------------
>
> Key: HDFS-1148
> URL: https://issues.apache.org/jira/browse/HDFS-1148
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-1148-old.txt, patch-HDFS-1148-rel0.20.2.txt
>
>
> In benchmarking HDFS-941 I noticed that for the random read workload, the
> FSDataset lock is highly contended. After converting it to a
> ReentrantReadWriteLock, I saw a ~25% improvement on both latency and
> ops/second.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
