[ https://issues.apache.org/jira/browse/IGNITE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kirill Tkalenko updated IGNITE-17230: ------------------------------------- Summary: Support split-file page store (was: Support splt-file page store) > Support split-file page store > ----------------------------- > > Key: IGNITE-17230 > URL: https://issues.apache.org/jira/browse/IGNITE-17230 > Project: Ignite > Issue Type: Task > Reporter: Kirill Tkalenko > Assignee: Kirill Tkalenko > Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 10m > Remaining Estimate: 0h > > *Notes* > Description may not be complete. > *Goal* > To implement a new checkpoint (described in IGNITE-15818), we will introduce > a new entity *DeltaFilePageStore*, which will be created for each partition > at each checkpoint and removed after merging with the *FilePageStore* (the > main partition file) using the compacter. > *DeltaFilePageStore* will consist of: > * Header (maybe updated in the course of implementation): > ** Allocation *pageIdx* - *pageIdx* of the last created page; > * Sorted list of *pageIdx* - allows a binary search to find the file offset > for an *pageId -> pageIdx*; > * Page content - sorted by *pageIdx*. > What will change for *FilePageStore*: > * List of class *DeltaFilePageStore* will be added (from the newest to the > oldest by the time of creation); > * Allocation index (pageIdx of the last created page) - it will be logical > and contained in the header of *FilePageStore*. At node start, it will be > read from the header of *FilePageStore* or obtained from the first > *DeltaFilePageStore* (the newest one). > How pages will be read by *pageId -> pageIdx*: > * Interrogates the class *DeltaFilePageStore* in order from the newest to the > oldest; > * If not found, then we read page from the *FilePageStore* itself. > *Some implementation notes* > * Format of the file name for the *DeltaFilePageStore* is > *part-%d-delta-%d.bin* for example *part-1-delta-3.bin* where the first digit > is the partition identifier, and the second is the serial number of the delta > file for this partition; > * Before creating *part-1-delta-3.bin*, a temporary file > *part-1-delta-3.bin.tmp* will be created at the checkpoint first, then > filled, then renamed to *part-1-delta-3.bin*; > * In each partition and delta file we will store the > *org.apache.ignite.internal.storage.pagememory.io.PartitionMetaIo*, which > will be the first page in this file, and it will be special; > ** We will not store it in *PageMemory* (we will not let it be dirty) for > optimization and will write it immediately to the delta file at the > checkpoint and at the merge with the partition file by the compactor; > ** Instead of the allocation index, we will store the logical size of the > partition, which we will store in the *PartitionMetaIo*; > * We will store the list of *pageIdx* in the header of the delta file. -- This message was sent by Atlassian Jira (v8.20.10#820010)