[ 
https://issues.apache.org/jira/browse/IGNITE-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy resolved IGNITE-23595.
----------------------------------------
    Resolution: Duplicate

> Flush Metastorage underlying storage to disk after taking snapshot
> ------------------------------------------------------------------
>
>                 Key: IGNITE-23595
>                 URL: https://issues.apache.org/jira/browse/IGNITE-23595
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Roman Puchkovskiy
>            Priority: Major
>              Labels: ignite-3
>
> After taking a snapshot, JRaft truncates log to remove indices which are 
> included in the snapshot. Snapshot metadata is persisted to the underlying 
> RocksDB storage, and later, when a snapshot is restored, its metadata is read 
> from the same storage.
> Writes to the storage are not flushed to disk forcefully (hence snapshot meta 
> is not durable), but log truncates are saved durably. As a result, if a node 
> crashes (or even gets stopped gracefully), it might happen that snapshot 
> metadata write is lost. If, after that, a snapshot is tried to be restored 
> (which will happen on each node start), this will fail because an index in 
> the snapshot meta lags behind the index in the log:
> {quote}2024-11-02 19:01:05:384 +0000 
> [ERROR][%node1%JRaft-FSMCaller-Disruptormetastorage_stripe_0-0][StripedDisruptor]
>  Handle disruptor event error 
> [name=%node1%JRaft-FSMCaller-Disruptormetastorage-, 
> event=org.apache.ignite.raft.jraft.core.FSMCallerImpl$ApplyTask@e69e5ae, 
> hasHandler=false]
> java.lang.IllegalArgumentException: Try to truncate logs before 262622, but 
> the firstLogIndex is 262661
> {quote}
> We need to flush the contents of the RocksDB storage to disk after taking a 
> snapshot (but before doing log truncation) and do it durably (that is, with 
> fsync).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to