Sergey Chugunov created IGNITE-13143:
----------------------------------------
Summary: Persistent store defragmentation
Key: IGNITE-13143
URL: https://issues.apache.org/jira/browse/IGNITE-13143
Project: Ignite
Issue Type: New Feature
Reporter: Sergey Chugunov
Persistent store enables users to store data of their caches in a durable
fashion on disk still benefiting from in-memory nature of Apache Ignite. Data
of caches is stored in files created for every primary or backup partition
assigned to that node and in an additional file for all user indexes.
Files in filesystem are allocated lazily (only if some data is actually stored
to particular partition) and grow automatically when more data is added to the
cache. But the problem is that files cannot shrink even if all data is removed.
This umbrella ticket covers all other tasks needed to implement simple yet
effective approach to defragmentation. Detailed discussion could be found in
[IEP-47|https://cwiki.apache.org/confluence/display/IGNITE/IEP-47%3A+Native+persistence+defragmentation]
and in corresponding [dev-list
discussion|http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-IEP-47-Native-persistence-defragmentation-td47717.html]
but core ideas are as follows:
# Defragmentation is performed in a special _maintenance_ mode when node
starts, provides access to some APIs like metrics or JMX management but doesn't
join the cluster.
# It is performed by copying all data from all partitions on node to new files
with automatic compaction. After successful copy old partition files are
deleted.
# Metrics on progress of the operation are provided to the user.
# Operation is fault-tolerant and in case of node failure proceeds after node
restart.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
