[
https://issues.apache.org/jira/browse/HDDS-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee reassigned HDDS-2542:
-----------------------------------------
Assignee: Shashikant Banerjee
> Race condition between read and write stateMachineData
> ------------------------------------------------------
>
> Key: HDDS-2542
> URL: https://issues.apache.org/jira/browse/HDDS-2542
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Marton Elek
> Assignee: Shashikant Banerjee
> Priority: Critical
>
> The write payload (the chunk itself) is sent to the Ratis as an external,
> binary byte array. It's not part of the LogEntry and saved from an async
> thread with calling ContainerStateMachine.writeStateMachineData
>
> As it's an async thread it's possible that the stateMachineData is not yet
> written when the data should be sent to the followers in the next heartbeat.
> By design a cache is used to avoid this issue but there are multiple problems
> with the cache.
> First, the current cache size is chunkExecutor.getCorePoolSize() which is not
> enough. By default it means 60 executor threads and a cache with size 60. But
> in case of one very slow and 59 very fast writer the cache entries can be
> invalidated before the write.
> In my tests (freon datanode-chunk-writer-generator) I have seen missed cache
> hits even with cache size 5000.
> Second: as the readStateMachineData and writeStateMachien data are called
> from two different thread there is a race condition independent from the the
> cache size. It's possible that the write thread has not yet added the data to
> the cache but the read thread needs it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
