[
https://issues.apache.org/jira/browse/IGNITE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-17230:
-------------------------------------
Description:
*Notes*
Description may not be complete.
*Goal*
To implement a new checkpoint (described in IGNITE-15818), we will introduce a
new entity *DelataFilePageStore*, which will be created for each partition at
each checkpoint and removed after merging with the *FilePageStore* (the main
partition file) using the compacter.
*DelataFilePageStore* will consist of:
* Header (maybe updated in the course of implementation):
** Allocation *pageIdx* - *pageIdx* of the last created page;
* Sorted list of *pageIdx* - allows a binary search to find the file offset for
an *pageId -> pageIdx*;
* Page content - sorted by *pageIdx*.
What will change for *FilePageStore*:
* List of class *DelataFilePageStore* will be added (from the newest to the
oldest by the time of creation);
* Allocation index (pageIdx of the last created page) - it will be logical and
contained in the header of *FilePageStore*. At node start, it will be read from
the header of *FilePageStore* or obtained from the first *DelataFilePageStore*
(the newest one).
How pages will be read by *pageId -> pageIdx*:
* Interrogates the class *DelataFilePageStore* in order from the newest to the
oldest;
* If not found, then we read page from the *FilePageStore* itself.
*Some implementation notes*
* Format of the file name for the *DelataFilePageStore* is
*part-%d-delta-%d.bin* for example *part-1-delta-3.bin* where the first digit
is the partition identifier, and the second is the serial number of the delta
file for this partition;
* Before creating *part-1-delta-3.bin*, a temporary file
*part-1-delta-3.bin.tmp* will be created at the checkpoint first, then filled,
then renamed to *part-1-delta-3.bin*;
* Since the indexes will be stored in partitions, we can get rid of the code
associated with the index partition file.
was:
*Notes*
Description may not be complete.
*Goal*
To implement a new checkpoint (described in IGNITE-15818), we will introduce a
new entity {*}DelataFilePageStore{*}, which will be created for each partition
at each checkpoint and removed after merging with the *FilePageStore* (the main
partition file) using the compacter.
*DelataFilePageStore* will consist of:
* Header (maybe updated in the course of implementation):
** Allocation *pageIdx* - *pageIdx* of the last created page;
* Sorted list of *pageIdx* - allows a binary search to find the file offset
for an {*}pageId -> pageIdx{*};
* Page content - sorted by {*}pageIdx{*}.
What will change for {*}FilePageStore{*}:
* List of class *DelataFilePageStore* will be added (from the newest to the
oldest by the time of creation);
* Allocation index (pageIdx of the last created page) - it will be logical and
contained in the header of {*}FilePageStore{*}. At node start, it will be read
from the header of *FilePageStore* or obtained from the first
*DelataFilePageStore* (the newest one).
How pages will be read by {*}pageId -> pageIdx{*}:
* Interrogates the class *DelataFilePageStore* in order from the newest to the
oldest;
* If not found, then we read page from the *FilePageStore* itself.
*Some implementation notes*
* Format of the file name for the *DelataFilePageStore* is
*part-%d-delta-%d.bin* for example *part-1-delta-3.bin* where the first digit
is the partition identifier, and the second is the serial number of the delta
file for this partition;
* Before creating {*}part-1-delta-3.bin{*}, a temporary file
*part-1-delta-3.bin.tmp* will be created at the checkpoint first, then filled,
then renamed to {*}part-1-delta-3.bin{*}.
> Support splt-file page store
> ----------------------------
>
> Key: IGNITE-17230
> URL: https://issues.apache.org/jira/browse/IGNITE-17230
> Project: Ignite
> Issue Type: Task
> Reporter: Kirill Tkalenko
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> *Notes*
> Description may not be complete.
> *Goal*
> To implement a new checkpoint (described in IGNITE-15818), we will introduce
> a new entity *DelataFilePageStore*, which will be created for each partition
> at each checkpoint and removed after merging with the *FilePageStore* (the
> main partition file) using the compacter.
> *DelataFilePageStore* will consist of:
> * Header (maybe updated in the course of implementation):
> ** Allocation *pageIdx* - *pageIdx* of the last created page;
> * Sorted list of *pageIdx* - allows a binary search to find the file offset
> for an *pageId -> pageIdx*;
> * Page content - sorted by *pageIdx*.
> What will change for *FilePageStore*:
> * List of class *DelataFilePageStore* will be added (from the newest to the
> oldest by the time of creation);
> * Allocation index (pageIdx of the last created page) - it will be logical
> and contained in the header of *FilePageStore*. At node start, it will be
> read from the header of *FilePageStore* or obtained from the first
> *DelataFilePageStore* (the newest one).
> How pages will be read by *pageId -> pageIdx*:
> * Interrogates the class *DelataFilePageStore* in order from the newest to
> the oldest;
> * If not found, then we read page from the *FilePageStore* itself.
> *Some implementation notes*
> * Format of the file name for the *DelataFilePageStore* is
> *part-%d-delta-%d.bin* for example *part-1-delta-3.bin* where the first digit
> is the partition identifier, and the second is the serial number of the delta
> file for this partition;
> * Before creating *part-1-delta-3.bin*, a temporary file
> *part-1-delta-3.bin.tmp* will be created at the checkpoint first, then
> filled, then renamed to *part-1-delta-3.bin*;
> * Since the indexes will be stored in partitions, we can get rid of the code
> associated with the index partition file.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
