[
https://issues.apache.org/jira/browse/IGNITE-26378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Chudov updated IGNITE-26378:
----------------------------------
Description:
Data nodes history introduced in IGNITE-23466 is not compacted, the compaction
was postponed during the implementation. It needs to be done to avoid the
infinite growth.
Criteria for the history entries to become obsolete:
* they are not latest (latest entry is always preserved);
* and their timestamp less than now - dataAvailabilityTime - clockSkew
(dataAvailabilityTime is LowWatermarkConfiguration#dataAvailabilityTimeMillis)
* and their timestamp is less than timestamp of the oldest non-compacted
catalog version.
So we should calculate minimal of two values (now - dataAvailabilityTime -
clockSkew and timestamp of the oldest non-compacted catalog version) and remove
data nodes history entries that are not latest and are older than calculated
timestamp.
They can be removed on data nodes history appending.
We also should make sure that new data nodes history entry is written when zone
descriptor in catalog is updated.
was:Data nodes history introduced in IGNITE-23466 is not compacted, the
compaction was postponed during the implementation. It needs to be done to
avoid the infinite growth.
> Data nodes history compaction
> -----------------------------
>
> Key: IGNITE-26378
> URL: https://issues.apache.org/jira/browse/IGNITE-26378
> Project: Ignite
> Issue Type: Improvement
> Reporter: Denis Chudov
> Priority: Major
> Labels: ignite-3
>
> Data nodes history introduced in IGNITE-23466 is not compacted, the
> compaction was postponed during the implementation. It needs to be done to
> avoid the infinite growth.
> Criteria for the history entries to become obsolete:
> * they are not latest (latest entry is always preserved);
> * and their timestamp less than now - dataAvailabilityTime - clockSkew
> (dataAvailabilityTime is LowWatermarkConfiguration#dataAvailabilityTimeMillis)
> * and their timestamp is less than timestamp of the oldest non-compacted
> catalog version.
> So we should calculate minimal of two values (now - dataAvailabilityTime -
> clockSkew and timestamp of the oldest non-compacted catalog version) and
> remove data nodes history entries that are not latest and are older than
> calculated timestamp.
> They can be removed on data nodes history appending.
> We also should make sure that new data nodes history entry is written when
> zone descriptor in catalog is updated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
