[
https://issues.apache.org/jira/browse/PHOENIX-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697272#comment-17697272
]
ASF GitHub Bot commented on PHOENIX-6888:
-----------------------------------------
kadirozde commented on code in PR #1569:
URL: https://github.com/apache/phoenix/pull/1569#discussion_r1127467301
##########
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java:
##########
@@ -588,6 +588,49 @@ public InternalScanner
preCompact(ObserverContext<RegionCoprocessorEnvironment>
@Override
public InternalScanner run() throws Exception {
InternalScanner internalScanner = scanner;
+ boolean isDisabled = false;
+ if (request.isMajor()) {
+ final String
+ fullTableName =
+
c.getEnvironment().getRegion().getRegionInfo().getTable().getNameAsString();
+ if
(!PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME.equals(fullTableName)) {
+ try (PhoenixConnection conn =
QueryUtil.getConnectionOnServer(
+
compactionConfig).unwrap(PhoenixConnection.class)) {
+ PTable table =
PhoenixRuntime.getTableNoCache(conn, fullTableName);
+ List<PTable>
+ indexes =
+
PTableType.INDEX.equals(table.getType()) ?
+ Lists.newArrayList(table) :
+ table.getIndexes();
+ // FIXME need to handle views and indexes on
views as well
+ for (PTable index : indexes) {
+ if (index.getIndexDisableTimestamp() != 0)
{
+ LOGGER.info("Modifying major
compaction scanner to retain "
+ + "deleted cells for a table
with disabled index: "
+ + fullTableName);
+ isDisabled = true;
+ break;
+ }
+ }
+ } catch (Exception e) {
+ if (e instanceof TableNotFoundException) {
+ LOGGER.debug(
+ "Ignoring HBase table that is not
a Phoenix table: "
+ + fullTableName);
+ // non-Phoenix HBase tables won't be
found, do nothing
+ } else {
+ LOGGER.error(
+ "Unable to modify compaction
scanner to retain deleted "
+ + "cells for a table with
disabled Index; "
+ + fullTableName, e);
+ }
+ }
+ }
+ }
+ if (!isDisabled) {
+ internalScanner = new
StoreCompactionScanner(c.getEnvironment(), store, scanner,
Review Comment:
My intention was to preserve the old behavior. Please see
BaseScannerRegionObserver#setScanOptionsForFlushesAndCompactions(). When we
disable the new scanner, we preserve the deleted cells and all versions.
> Fixing TTL and Max Lookback Issues for Phoenix Tables
> -----------------------------------------------------
>
> Key: PHOENIX-6888
> URL: https://issues.apache.org/jira/browse/PHOENIX-6888
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 5.1.3
> Reporter: Kadir Ozdemir
> Assignee: Kadir Ozdemir
> Priority: Major
>
> In HBase, the unit of data is a cell and data retention rules are executed at
> the cell level. These rules are defined at the column family level. Phoenix
> leverages the data retention features of HBase and exposes them to its users
> to provide its TTL feature at the table level. However, these rules (since
> they are defined at the cell level instead of the row level) results in
> partial row retention that in turn creates data integrity issues at the
> Phoenix level.
> Similarly, Phoenix’s max lookback feature leverages HBase deleted data
> retention capabilities to preserve deleted cells within a configurable max
> lookback. This requires two data retention windows, max lookback and TTL. One
> end of these windows is the current time and the end is a moment in the past
> (i.e., current time minus the window size). Typically, the max lookback
> window is shorter than the TTL window. In the max lookback window, we would
> like to preserve the complete history of mutations regardless of how many
> cell versions these mutations generated. In the remaining TTL window outside
> the max lookback, we would like to apply the data retention rules defined
> above. However, HBase provides only one data retention window. Thus, the max
> lookback window had to be extended to become TTL window and the max lookback
> feature results in unwantedly retaining deleted data for the maximum of max
> lookback and TTL periods.
> This Jira is to fix both of these issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
