[
https://issues.apache.org/jira/browse/IGNITE-22664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philipp Shergalis updated IGNITE-22664:
---------------------------------------
Description:
Every page in persistent page memory delta file uses ~15-20 kilobytes (default
settings).
Examples. Inserting via storageUpdateHandler. No indexes:
- Inserting 1 row at a time, doing checkpoint after every insertion.
Checkpoint saves 7 pages, pageSize is 1024, but delta file created uses up 144
KB.
- Inserting million rows, every checkpoint with ~11000 pages creates delta
files of ~180 megabytes
Also, after compaction one row uses ~300 bytes, while the same row in rocksDB
uses ~100 bytes
Checkpoints / compaction are hardcoded to use 4 threads
was:
Every page in persistent page memory delta file uses ~15-20 kilobytes.
Examples. Inserting via storageUpdateHandler. No indexes:
- Inserting 1 row at a time, doing checkpoint after every insertion.
Checkpoint saves 7 pages, pageSize is 1024, but delta file created uses up 144
KB.
- Inserting million rows, every checkpoint with ~11000 pages creates delta
files of ~180 megabytes
Also, after compaction one row uses ~300 bytes, while similar row in rocksDB
uses ~100 bytes
> Insanely memory consuming checkpoints in aipersist
> --------------------------------------------------
>
> Key: IGNITE-22664
> URL: https://issues.apache.org/jira/browse/IGNITE-22664
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
> Reporter: Philipp Shergalis
> Priority: Major
> Labels: ignite-3
>
> Every page in persistent page memory delta file uses ~15-20 kilobytes
> (default settings).
>
> Examples. Inserting via storageUpdateHandler. No indexes:
> - Inserting 1 row at a time, doing checkpoint after every insertion.
> Checkpoint saves 7 pages, pageSize is 1024, but delta file created uses up
> 144 KB.
> - Inserting million rows, every checkpoint with ~11000 pages creates delta
> files of ~180 megabytes
>
> Also, after compaction one row uses ~300 bytes, while the same row in rocksDB
> uses ~100 bytes
>
> Checkpoints / compaction are hardcoded to use 4 threads
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
