[
https://issues.apache.org/jira/browse/PHOENIX-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17819775#comment-17819775
]
ASF GitHub Bot commented on PHOENIX-7006:
-----------------------------------------
sanjeet006py commented on code in PR #1751:
URL: https://github.com/apache/phoenix/pull/1751#discussion_r1499781459
##########
phoenix-core-server/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java:
##########
@@ -527,6 +535,28 @@ public static boolean isMaxLookbackTimeEnabled(long
maxLookbackTime){
return maxLookbackTime > 0L;
}
+ public static long
getMaxLookbackAge(ObserverContext<RegionCoprocessorEnvironment> c) {
+ TableName tableName =
c.getEnvironment().getRegion().getRegionInfo().getTable();
+ String fullTableName = tableName.getNameAsString();
+ Configuration conf = c.getEnvironment().getConfiguration();
+ PTable table;
+ try(PhoenixConnection conn = QueryUtil.getConnectionOnServer(
+ conf).unwrap(PhoenixConnection.class)) {
+ table = conn.getTableNoCache(fullTableName);
+ }
+ catch (Exception e) {
+ if (e instanceof TableNotFoundException) {
+ LOGGER.debug("Ignoring HBase table that is not a Phoenix
table: "
+ + fullTableName);
+ // non-Phoenix HBase tables won't be found, do nothing
Review Comment:
> What is the alternative approach?
In `TransformClient` we derive new table name from existing table name. I
saw we are appending sequence num to eixtsing table's full name separated by
underscore. I added a logic to derive logical table name from physical table
name by checking if physical table is underscore separated and if part of table
name before underscore is valid logical table name in SYSCAT along with
checking if the logical table found has physical table as original physical
table name.
I thought this will avoid FULL SCAN along max be one RPC call but even any
READ operation on any table even if we create an index will be at least one RPC
call. Please correct me if I misunderstood something. Thanks
> Configure maxLookbackAge at table level
> ---------------------------------------
>
> Key: PHOENIX-7006
> URL: https://issues.apache.org/jira/browse/PHOENIX-7006
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Viraj Jasani
> Assignee: Sanjeet Malhotra
> Priority: Major
>
> Phoenix max lookback age feature preserves live or deleted row versions that
> are only visible through the max lookback window, it does not preserve any
> unwanted row versions that should not be visible through the max lookback
> window. More details on the max lookback redesign: PHOENIX-6888
> As of today, maxlookback age is only configurable at the cluster level
> (config key: {_}phoenix.max.lookback.age.seconds{_}), meaning the same value
> is used by all tables. This does not allow individual table level compaction
> scanner to be able to retain data based on the table level maxlookback age.
> Setting max lookback age at the table level can serve multiple purposes e.g.
> change-data-capture (PHOENIX-7001) for individual table should have it's own
> latest data retention period.
> The purpose of this Jira is to allow maxlookback age as a table level
> property:
> * New column in SYSTEM.CATALOG to preserve table level maxlookback age
> * PTable object to read the value of maxlookback from SYSTEM.CATALOG
> * Allow CREATE/ALTER TABLE DDLs to provide maxlookback attribute
> * CompactionScanner should use table level maxlookbackAge, if available,
> else use cluster level config
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
