Poorna Chandra created PHOENIX-2993:
---------------------------------------
Summary: Tephra: Prune invalid transaction set once all data for a
given invalid transaction has been dropped
Key: PHOENIX-2993
URL: https://issues.apache.org/jira/browse/PHOENIX-2993
Project: Phoenix
Issue Type: New Feature
Reporter: Poorna Chandra
Assignee: Poorna Chandra
>From TEPHRA-35 -
In addition to dropping the data from invalid transactions we need to be able
to prune the invalid set of any transactions where data cleanup has been
completely performed. Without this, the invalid set will grow indefinitely and
become a greater and greater cost to in-progress transactions over time.
To do this correctly, the TransactionDataJanitor coprocessor will need to
maintain some bookkeeping for the transaction data that it removes, so that the
transaction manager can reason about when all of a given transaction's data has
been removed. Only at this point can the transaction manager safely drop the
transaction ID from the invalid set.
One approach would be for the TransactionDataJanitor to update a table marking
when a major compaction was performed on a region and what transaction IDs were
filtered out. Once all regions in a table containing the transaction data have
been compacted, we can remove the filtered out transaction IDs from the invalid
set. However, this will need to cope with changing region names due to splits,
etc.
Note: This will be moved to Tephra JIRA once the setup of Tephra JIRA is
complete (INFRA-11445)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
