[ https://issues.apache.org/jira/browse/IGNITE-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Puchkovskiy reassigned IGNITE-17254: ------------------------------------------ Assignee: Roman Puchkovskiy > Storage API for RAFT snapshot streaming > --------------------------------------- > > Key: IGNITE-17254 > URL: https://issues.apache.org/jira/browse/IGNITE-17254 > Project: Ignite > Issue Type: Improvement > Reporter: Roman Puchkovskiy > Assignee: Roman Puchkovskiy > Priority: Major > Labels: ignite-3 > > Current PartitionStorage has a method for taking snapshots (and a method for > restoring from them). TheĀ methods work with files. In IGNITE-17253, we are > going to switch to streaming protocol (which does not use files at all). So > we need streaming snapshot-related methods on MvPartitionStorage. > h3. UPDATE > From my understanding, we basically need two more methods in the MV partition > storage to manipulate data: > {code:java} > // This one represents a combination of "addWrite" and "commitWrite", but a > faster one. > // It will be used in incoming snapshot copier to write data. > void addWriteCommitted(RowId rowId, BinaryRow row, HybridTimestamp > commitTimestamp); > // This one will be used to get all versions of a particular row and form a > SnapshotMvDataResponse message. > Cursor<ReadResult> scanVersionsEx(RowId rowId);{code} > Both of them are very simple and, for the most part, match their counterparts > in every storage. > What's not as simple is the fail-over scenario. We shouldn't treat > partially-downloaded partition as a valid set of data. So, the easiest was > (in My opinion) to achieve this is to: > * set "lastAppliedIndex" value to "-1" before writing any the data > * drop old data (these two steps may be swapped, it's not that important) > * download and write everything > * set valid value for "lastAppliedIndex" > This is a crude solution, but should work well, especially if properly > documented. -- This message was sent by Atlassian Jira (v8.20.10#820010)