[
https://issues.apache.org/jira/browse/HDFS-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707061#comment-17707061
]
ASF GitHub Bot commented on HDFS-16967:
---------------------------------------
virajjasani opened a new pull request, #5523:
URL: https://github.com/apache/hadoop/pull/5523
File based state store implementations (StateStoreFileImpl and
StateStoreFileSystemImpl) should allow updating as well as reading of the state
store records concurrently rather than serially. Concurrent access to the
record files on the hdfs based store seems to be improving the state store
cache loading performance by more than 10x.
For instance, in order to maintain data integrity, when any mount table
record(s) is updated, the cache is reloaded. This reload operation seems to be
able to gain significant performance improvement by the concurrent access of
the mount table records.
> RBF: File based state stores should allow concurrent access to the records
> --------------------------------------------------------------------------
>
> Key: HDFS-16967
> URL: https://issues.apache.org/jira/browse/HDFS-16967
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
>
> File based state store implementations (StateStoreFileImpl and
> StateStoreFileSystemImpl) should allow updating as well as reading of the
> state store records concurrently rather than serially. Concurrent access to
> the record files on the hdfs based store seems to be improving the state
> store cache loading performance by more than 10x.
> For instance, in order to maintain data integrity, when any mount table
> record(s) is updated, the cache is reloaded. This reload operation seems to
> be able to gain significant performance improvement by the concurrent access
> of the mount table records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
