[
https://issues.apache.org/jira/browse/HDDS-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-2477:
---------------------------------
Labels: pull-request-available (was: )
> TableCache cleanup issue for OM non-HA
> --------------------------------------
>
> Key: HDDS-2477
> URL: https://issues.apache.org/jira/browse/HDDS-2477
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
>
> In OM in non-HA case, the ratisTransactionLogIndex is generated by
> OmProtocolServersideTranslatorPB.java. And in OM non-HA
> validateAndUpdateCache is called from multipleHandler threads. So think of a
> case where one thread which has an index - 10 has added to doubleBuffer. (0-9
> still have not added). DoubleBuffer flush thread flushes and call cleanup.
> (So, now cleanup will go and cleanup all cache entries with less than 10
> epoch) This should not have cleanup those which might have put in to cache
> later and which are in process of flush to DB. This will cause inconsitency
> for few OM requests.
>
>
> Example:
> 4 threads Committing 4 parts.
> 1st thread - part 1 - ratis Index - 3
> 2nd thread - part 2 - ratis index - 2
> 3rd thread - part3 - ratis index - 1
>
> First thread got lock, and put in to doubleBuffer and cache with
> OmMultipartInfo (with part1). And cleanup is called to cleanup all entries in
> cache with less than 3. In the mean time 2nd thread and 1st thread put 2,3
> parts in to OmMultipartInfo in to Cache and doubleBuffer. But first thread
> might cleanup those entries, as it is called with index 3 for cleanup.
>
> Now when the 4th part upload came -> when it is commit Multipart upload when
> it gets multipartinfo it get Only part1 in OmMultipartInfo, as the
> OmMultipartInfo (with 1,2,3 is still in process of committing to DB). So now
> after 4th part upload is complete in DB and Cache we will have 1,4 parts
> only. We will miss part2,3 information.
>
> So for non-HA case cleanup will be called with list of epochs that need to be
> cleanedup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
