[
https://issues.apache.org/jira/browse/IGNITE-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-17254:
-----------------------------------
Description:
Current PartitionStorage has a method for taking snapshots (and a method for
restoring from them). The methods work with files. In IGNITE-17253, we are
going to switch to streaming protocol (which does not use files at all). So we
need streaming snapshot-related methods on MvPartitionStorage.
h3. UPDATE
>From my understanding, we basically need two more methods in the MV partition
>storage to manipulate data:
{code:java}
// This one represents a combination of "addWrite" and "commitWrite", but a
faster one.
// It will be used in incoming snapshot copier to write data.
void addWriteCommitted(RowId rowId, BinaryRow row, HybridTimestamp
commitTimestamp);
// This one will be used to get all versions of a particular row and form a
SnapshotMvDataResponse message.
Cursor<ReadResult> scanVersionsEx(RowId rowId);{code}
Both of them are very simple and, for the most part, match their counterparts
in every storage.
What's not as simple is the fail-over scenario. We shouldn't treat
partially-downloaded partition as a valid set of data. So, the easiest was (in
My opinion) to achieve this is to:
* set "lastAppliedIndex" value to "-1" before writing any the data
* drop old data (these two steps may be swapped, it's not that important)
* download and write everything
* set valid value for "lastAppliedIndex"
This is a crude solution, but should work well, especially if properly
documented.
was:Current PartitionStorage has a method for taking snapshots (and a method
for restoring from them). The methods work with files. In IGNITE-17253, we are
going to switch to streaming protocol (which does not use files at all). So we
need streaming snapshot-related methods on MvPartitionStorage.
> Storage API for RAFT snapshot streaming
> ---------------------------------------
>
> Key: IGNITE-17254
> URL: https://issues.apache.org/jira/browse/IGNITE-17254
> Project: Ignite
> Issue Type: Improvement
> Reporter: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
>
> Current PartitionStorage has a method for taking snapshots (and a method for
> restoring from them). The methods work with files. In IGNITE-17253, we are
> going to switch to streaming protocol (which does not use files at all). So
> we need streaming snapshot-related methods on MvPartitionStorage.
> h3. UPDATE
> From my understanding, we basically need two more methods in the MV partition
> storage to manipulate data:
> {code:java}
> // This one represents a combination of "addWrite" and "commitWrite", but a
> faster one.
> // It will be used in incoming snapshot copier to write data.
> void addWriteCommitted(RowId rowId, BinaryRow row, HybridTimestamp
> commitTimestamp);
> // This one will be used to get all versions of a particular row and form a
> SnapshotMvDataResponse message.
> Cursor<ReadResult> scanVersionsEx(RowId rowId);{code}
> Both of them are very simple and, for the most part, match their counterparts
> in every storage.
> What's not as simple is the fail-over scenario. We shouldn't treat
> partially-downloaded partition as a valid set of data. So, the easiest was
> (in My opinion) to achieve this is to:
> * set "lastAppliedIndex" value to "-1" before writing any the data
> * drop old data (these two steps may be swapped, it's not that important)
> * download and write everything
> * set valid value for "lastAppliedIndex"
> This is a crude solution, but should work well, especially if properly
> documented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
