[
https://issues.apache.org/jira/browse/HDFS-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John George updated HDFS-2081:
------------------------------
Attachment: HDFS-2081.patch
Thanks for the pointer Tod. My approach to this is slightly different.
Attaching a patch without any unit tests for prelim review. I know this patch
works because of the yarn test I ran. I do want to write a unit test for this,
so that we know if it breaks in the future. I dont have a clear cut idea as to
how to write one.
> trying to read configuration in hdfs write path might result in deadlock
> ------------------------------------------------------------------------
>
> Key: HDFS-2081
> URL: https://issues.apache.org/jira/browse/HDFS-2081
> Project: Hadoop HDFS
> Issue Type: Bug
> Environment: MR-279 branch. This is specific to yarn since yarn tries
> to write the configuration file to hdfs while holding the lock of the
> configuration file.
> Reporter: John George
> Assignee: John George
> Attachments: HDFS-2081.patch
>
>
> While trying to run yarn with 1024 block size, it looks as though the
> dfsclients are hung waiting to write to DN. The reason is because yarn golds
> the configuration file lock and tries to write to hdfs which in turn tries to
> read from the configuration which requires the same lock.
> The temporary fix is to make sure that dfsclient threads dont read the config
> in its write path. There might be better long term solutions like read/write
> locks and this needs to be investigated in another JIRA.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
