[
https://issues.apache.org/jira/browse/HDDS-9960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17799291#comment-17799291
]
Sumit Agrawal commented on HDDS-9960:
-------------------------------------
Upgrade PrepareRequest is blocking executorService thread and waiting for
update with commitIndex from ratis (
notifyTermIndexUpdated). So can not use same exectuorService to update this
index.
Further its noticed that,
* If any changes to stateMachine, apply transaction is called, and that needs
to be applied using double buffer
* other changes for ratis configuration and other, it notified using
notifyTermIndexUpdated
*
So index are notified with increasing order and unique for each case. So
ordering as done to update LastAppliedIndex in OMStateMachine need to be
followed.
Fix is done handling missing of index and continue with next index to update
lastAppliedIndex, and avoid memory buildup for ratisTransactionMap.
> OM State machine lastIndexUpdate consume memory and incorrect
> -------------------------------------------------------------
>
> Key: HDDS-9960
> URL: https://issues.apache.org/jira/browse/HDDS-9960
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sumit Agrawal
> Assignee: Sumit Agrawal
> Priority: Major
> Labels: pull-request-available
>
> update lastTransactionIndex is also done from another flow,
> ratis --> notifyTermIndexUpdated(). --> computeAndUpdateLastAppliedIndex()
> --> else part (for flow from ratis)
> {code:java}
> } else {
> if (getLastAppliedTermIndex().getIndex() + 1 == lastFlushedIndex) {
> updateLastAppliedTermIndex(currentTerm, lastFlushedIndex);
> if (LOG.isDebugEnabled()) {
> LOG.debug("ComputeAndUpdateLastAppliedIndex due to notifyIndex {}",
> getLastAppliedTermIndex());
> }
> } else {
> ratisTransactionMap.put(lastFlushedIndex, currentTerm); {code}
> If lastTransactionAppliedIndex and ratis transaction index difference is only
> "1", it update the last transaction index. This can cause db flush
> transaction to be behind the ratis transaction. So replay of those
> tranasaction will not happen in case of OM crash at this point.
>
> So there is a two different behavior exist, that when to update
> lastAppliedTransaction, creating confusion about expectation for this
> perspective.
>
> Also its observed updating last applied transaction stuck when there is
> difference in transaction index due to exception. This was causing
> ratisTransactionMap holding large number of transaction and not applied.
> https://issues.apache.org/jira/browse/HDDS-9342.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
