[jira] [Commented] (HDFS-16967) RBF: File based state stores should allow concurrent access to the records

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/HDFS-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707128#comment-17707128
 ] 

ASF GitHub Bot commented on HDFS-16967:
---------------------------------------

virajjasani commented on PR #5523:
URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1491325194

   Default mode:
   
   <img width="522" alt="Screenshot 2023-03-29 at 11 00 00 PM" 
src="https://user-images.githubusercontent.com/34790606/229032556-3603807e-94de-48f8-87bc-831f4519ed11.png";>
   
   <img width="375" alt="Screenshot 2023-03-29 at 11 08 22 PM" 
src="https://user-images.githubusercontent.com/34790606/229032560-24d586f9-4814-4192-afc3-09bb524846ea.png";>
   
   
   Concurrent access mode:
   
   <img width="374" alt="Screenshot 2023-03-29 at 11 06 13 PM" 
src="https://user-images.githubusercontent.com/34790606/229032661-9f131979-d329-4bf3-8933-ae2009d94301.png";>
   
   <img width="429" alt="Screenshot 2023-03-29 at 11 15 03 PM" 
src="https://user-images.githubusercontent.com/34790606/229032665-f846e183-bf89-45a6-a68b-55d57fb8a6e6.png";>
   
   




> RBF: File based state stores should allow concurrent access to the records
> --------------------------------------------------------------------------
>
>                 Key: HDFS-16967
>                 URL: https://issues.apache.org/jira/browse/HDFS-16967
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Viraj Jasani
>            Assignee: Viraj Jasani
>            Priority: Major
>              Labels: pull-request-available
>
> File based state store implementations (StateStoreFileImpl and 
> StateStoreFileSystemImpl) should allow updating as well as reading of the 
> state store records concurrently rather than serially. Concurrent access to 
> the record files on the hdfs based store seems to be improving the state 
> store cache loading performance by more than 10x.
> For instance, in order to maintain data integrity, when any mount table 
> record(s) is updated, the cache is reloaded. This reload operation seems to 
> be able to gain significant performance improvement by the concurrent access 
> of the mount table records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org