[
https://issues.apache.org/jira/browse/PHOENIX-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated PHOENIX-7006:
----------------------------------
Description:
Phoenix max lookback age feature preserves live or deleted row versions that
are only visible through the max lookback window, it does not preserve any
unwanted row versions that should not be visible through the max lookback
window. More details on the max lookback redesign: PHOENIX-6888
As of today, maxlookback age is only configurable at the cluster level (config
key: {_}phoenix.max.lookback.age.seconds{_}), meaning the same value is used by
all tables. This does not allow individual table level compaction scanner to be
able to retain data based on the table level maxlookback age. Setting max
lookback age at the table level can serve multiple purposes e.g.
change-data-capture (PHOENIX-7001) for individual table should have it's own
latest data retention period.
The purpose of this Jira is to allow maxlookback age as a table level property:
* New column in SYSTEM.CATALOG to preserve table level maxlookback age
* PTable object to read the value of maxlookback from SYSTEM.CATALOG
* CompactionScanner should use table level maxlookbackAge, if available, else
use cluster level config
was:
Phoenix max lookback age feature preserves live or deleted row versions that
are only visible through the max lookback window, it does not preserve any
unwanted row versions that should not be visible through the max lookback
window. More details on the max lookback redesign: PHOENIX-6888
As of today, maxlookback age is only configurable at the cluster level (config
key: {_}phoenix.max.lookback.age.seconds{_}), meaning the same value is used by
all tables. This does now allow individual table level compaction scanner to be
able to retain data based on the table level maxlookback age. Setting max
lookback age at the table level can serve multiple purposes e.g.
change-data-capture (PHOENIX-7001) for individual table should have it's own
latest data retention period.
The purpose of this Jira is to allow maxlookback age as a table level property:
* New column in SYSTEM.CATALOG to preserve table level maxlookback age
* PTable object to read the value of maxlookback from SYSTEM.CATALOG
* CompactionScanner should use table level maxlookbackAge, if available, else
use cluster level config
> Configure maxLookbackAge at table level
> ---------------------------------------
>
> Key: PHOENIX-7006
> URL: https://issues.apache.org/jira/browse/PHOENIX-7006
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Viraj Jasani
> Priority: Major
>
> Phoenix max lookback age feature preserves live or deleted row versions that
> are only visible through the max lookback window, it does not preserve any
> unwanted row versions that should not be visible through the max lookback
> window. More details on the max lookback redesign: PHOENIX-6888
> As of today, maxlookback age is only configurable at the cluster level
> (config key: {_}phoenix.max.lookback.age.seconds{_}), meaning the same value
> is used by all tables. This does not allow individual table level compaction
> scanner to be able to retain data based on the table level maxlookback age.
> Setting max lookback age at the table level can serve multiple purposes e.g.
> change-data-capture (PHOENIX-7001) for individual table should have it's own
> latest data retention period.
> The purpose of this Jira is to allow maxlookback age as a table level
> property:
> * New column in SYSTEM.CATALOG to preserve table level maxlookback age
> * PTable object to read the value of maxlookback from SYSTEM.CATALOG
> * CompactionScanner should use table level maxlookbackAge, if available,
> else use cluster level config
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
