Sarveksha Yeshavantha Raju created HDDS-16666:
-------------------------------------------------
Summary: Flush SCM transaction in memory during apply transaction
Key: HDDS-16666
URL: https://issues.apache.org/jira/browse/HDDS-16666
Project: Apache Ozone
Issue Type: Improvement
Components: SCM HA
Reporter: Sarveksha Yeshavantha Raju
Assignee: Sarveksha Yeshavantha Raju
As observed,
If a follower node starts after long duration, there are huge number of
transactions accumulated at leader which needs to be synchronized to the
follower node.
The Sync of raft log to follower node is done at high speed and flushed to
local directory. But speed of applyTransaction may not match and hence,
* at some point of time, commitIndex is very high compared to appliedIndex (>
5million or more)
* as ratis logic, it will try to apply all committed transaction to be applied
in loop (before taking snapshot) and thus build up huge memory to be flushed to
db.
So as solution to avoid memory buildup (or memory pressure), applyTransaction
can flush the db update to DB when pending count reaches like 10K.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
