[
https://issues.apache.org/jira/browse/IGNITE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Chudov updated IGNITE-18502:
----------------------------------
Description:
*Motivation*
All tables have a transactional nature, even if we don't use transactions
explicitly. That means all entries are stored as write intents (pending
entries) before they are committed. Those entries are required for
transactional protocol and stored in in-memory collection
(_PartitionListener#txsPendingRowIds_). The collection can be lost due to
restart of the node where the data located. Recovery of the collection happens
through scan of all entries in the partition and can take long time.
There are several possible ways to solve this problem: to persist pending rows
collection, or resolve write intents when they are met by another transaction,
including RW transactions. After some discussions the latter seems to be better
decision, as it doesn't require one more persistent storage/structure, and RW
transactions should be able to resolve write intents anyway.
*Definition of Done*
No full partition scan on start is needed.
*Implementation notes*
The chosen decision implies that unresolved write intents in partition can be
present in partition, even if such write intents belong to transaction that was
already committed or aborted. When aobther transaction (no matter read-only or
read-write) meets such write-intent it tries to resolve it. If it belongs to
transaction that was already finished, then local cleanup must be run for this
write-intent. This means that the write intent should be committed or aborted
as like it was processed by _TxCleanupCommand_.
was:
*Motivation*
All tables have a transactional nature, even if we don't use transactions
explicitly. That means all entry are storied as write intents (pending entries)
before they will be committed. Those entries are required for transactional
protocol and storied in memory collection
(_PartitionListener#txsPendingRowIds_). The collection can be lost due to
restart the node where the data located. Recovery of the collection happens
through scan all entries in the partition and can require huge time.
*Definition of Done*
After storage will be implemented, the pending entries' collection should be
replaced to it.
The storage should recover itself in order to do not scan partition on start.
> Excessive full partition scan on node start
> -------------------------------------------
>
> Key: IGNITE-18502
> URL: https://issues.apache.org/jira/browse/IGNITE-18502
> Project: Ignite
> Issue Type: Improvement
> Reporter: Vladislav Pyatkov
> Priority: Major
> Labels: ignite-3
>
> *Motivation*
> All tables have a transactional nature, even if we don't use transactions
> explicitly. That means all entries are stored as write intents (pending
> entries) before they are committed. Those entries are required for
> transactional protocol and stored in in-memory collection
> (_PartitionListener#txsPendingRowIds_). The collection can be lost due to
> restart of the node where the data located. Recovery of the collection
> happens through scan of all entries in the partition and can take long time.
> There are several possible ways to solve this problem: to persist pending
> rows collection, or resolve write intents when they are met by another
> transaction, including RW transactions. After some discussions the latter
> seems to be better decision, as it doesn't require one more persistent
> storage/structure, and RW transactions should be able to resolve write
> intents anyway.
> *Definition of Done*
> No full partition scan on start is needed.
> *Implementation notes*
> The chosen decision implies that unresolved write intents in partition can be
> present in partition, even if such write intents belong to transaction that
> was already committed or aborted. When aobther transaction (no matter
> read-only or read-write) meets such write-intent it tries to resolve it. If
> it belongs to transaction that was already finished, then local cleanup must
> be run for this write-intent. This means that the write intent should be
> committed or aborted as like it was processed by _TxCleanupCommand_.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
