Ivan Bessonov created IGNITE-20975:
--------------------------------------
Summary: Store secondary indexes in separate files
Key: IGNITE-20975
URL: https://issues.apache.org/jira/browse/IGNITE-20975
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
In current implementation, secondary indexes are stored in the same file as the
partition, which makes the truncation inefficient. It also leaves a bunch of
empty pages, which may not be desired.
The proposal is to store every index in its own set of files, independent* from
partitions.
As a "groupId" we will use the "indexId", they differ from table identifiers,
so that's fine.
We have two options for a place to store these files:
* same directory as row data
* other directory
Honestly, I think managing a single directory would be easier.
The tricky part is the checkpoint:
* compactor shouldn't start compacting files until both row data and index
data are checkpointed
* upon recovery, we should delete both the incomplete row storage files and
index files, to preserve consistency
* basically, this means the re-introduction of checkpoint markers. A
per-partition marker is too much IO, so I propose per-table markers. Single
marker for everything is an overkill, it leads to unnecessary slow recovery
* anyway, per-table markers also mean slower node recovery in cases when node
has been stopped during the checkpoint
* _be aware of data races, it's very easy to mess this up_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
