[
https://issues.apache.org/jira/browse/PHOENIX-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kadir Ozdemir reassigned PHOENIX-7382:
--------------------------------------
Assignee: Kadir Ozdemir
> Eliminating index building and treating max lookback as TTL for CDC Index
> -------------------------------------------------------------------------
>
> Key: PHOENIX-7382
> URL: https://issues.apache.org/jira/browse/PHOENIX-7382
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Kadir Ozdemir
> Assignee: Kadir Ozdemir
> Priority: Major
>
> Phoenix-7001 introduced a Change Data Capture (CDC) feature by leveraging
> Phoenix Max Lookback and Uncovered Global Index on PHOENIX_ROW_TIMESTAMP().
> The max lookback feature retains recent changes to a table and the uncovered
> index allows efficient retrieval of the changes in the order of their arrival.
> Since the changes are retained only within the max lookback window, a CDC
> index does not need to have rows beyond the max lookback window. This means
> we can treat the max lookback age as the TTL for CDC indexes.
> When a CDC feature is enabled on a table, a CDC index is created with the
> building state, and like any other index, it is built from the data table.
> This means for every mutation on the data table, an index row mutation is
> built. However, the mutations beyond max lookback are not required for CDC.
> Since index built is an expensive background operation (it can take days for
> very large tables), eliminating index built is desirable when the CDC is
> enabled on a table. This means that the CDC feature will track changes that
> happen after it is enabled, which is the expected behavior.
> The code changes for this improvement will be for
> * Creating a CDC index with the active state
> * Treating the max lookback age as the TTL for CDC indexes during
> compaction, and index verification, repair and rebuild.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
