[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818626#comment-17818626 ] ASF GitHub Bot commented on PHOENIX-7040: - lokiore merged PR #1782: URL: https://github.com/apache/phoenix/pull/1782 > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815059#comment-17815059 ] ASF GitHub Bot commented on PHOENIX-7040: - jpisaac commented on code in PR #1782: URL: https://github.com/apache/phoenix/pull/1782#discussion_r1480802465 ## phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java: ## @@ -1617,66 +1620,98 @@ private PTable getTableFromCells(List tableCellList, List> allC return builder.build(); } +/** + * Method to return TTL value defined at current level ot up the Hierarchy of the view. Review Comment: nit: typo "or"? ## phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java: ## @@ -1617,66 +1620,98 @@ private PTable getTableFromCells(List tableCellList, List> allC return builder.build(); } +/** + * Method to return TTL value defined at current level ot up the Hierarchy of the view. + * @param viewKey Key of the view for which we have to find TTL + * @param clientTimeStamp Client TimeStamp + * @return TTL value for a given view, if nothing is defined anywhere then return + * TTL_NOT_DEFINED(0). + * @throws IOException + * @throws SQLException + */ + Review Comment: nit: scanTTLFromParent should be called getTTLFromHierarchy WDT? > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803319#comment-17803319 ] ASF GitHub Bot commented on PHOENIX-7040: - lokiore commented on PR #1782: URL: https://github.com/apache/phoenix/pull/1782#issuecomment-1877735841 FYI @jpisaac > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803318#comment-17803318 ] ASF GitHub Bot commented on PHOENIX-7040: - lokiore opened a new pull request, #1782: URL: https://github.com/apache/phoenix/pull/1782 - Adding full more tests for ttl at different levels in heirarchy - Also resolving bug related to using physical link for getting ttl for views while building PTable. > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792283#comment-17792283 ] ASF GitHub Bot commented on PHOENIX-7040: - jpisaac merged PR #1710: URL: https://github.com/apache/phoenix/pull/1710 > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[
https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792236#comment-17792236
]
ASF GitHub Bot commented on PHOENIX-7040:
-
jpisaac commented on code in PR #1710:
URL: https://github.com/apache/phoenix/pull/1710#discussion_r1412637207
##
phoenix-core/src/main/java/org/apache/phoenix/util/ViewUtil.java:
##
@@ -132,20 +134,35 @@ public static Pair, List>
findAllDescendantViews(
List legitimateChildViews = new ArrayList<>();
List orphanChildViews = new ArrayList<>();
-findAllDescendantViews(sysCatOrsysChildLink, serverSideConfig,
tenantId, schemaName,
+return findAllDescendantViews(sysCatOrsysChildLink, serverSideConfig,
tenantId, schemaName,
tableOrViewName, clientTimeStamp, legitimateChildViews,
orphanChildViews,
findJustOneLegitimateChildView);
-return new Pair<>(legitimateChildViews, orphanChildViews);
}
-private static void findAllDescendantViews(Table sysCatOrsysChildLink,
-Configuration serverSideConfig, byte[] parentTenantId, byte[]
parentSchemaName,
-byte[] parentTableOrViewName, long clientTimeStamp, List
legitimateChildViews,
-List orphanChildViews, boolean
findJustOneLegitimateChildView)
+public static Pair, List>
findAllDescendantViews(Table sysCatOrsysChildLink,
+ Configuration serverSideConfig, byte[] parentTenantId, byte[]
parentSchemaName,
+ byte[] parentTableOrViewName, long clientTimeStamp, List
legitimateChildViews,
+ List orphanChildViews, boolean
findJustOneLegitimateChildView)
+ throws IOException, SQLException{
+
+ return findAllDescendantViews(sysCatOrsysChildLink, null,
serverSideConfig,
+ parentTenantId, parentSchemaName, parentTableOrViewName,
clientTimeStamp,
+ legitimateChildViews, orphanChildViews,
findJustOneLegitimateChildView,
+ new Pair<>(false, false));
+
+}
+
Review Comment:
nit: Can u add some comments for the new methods and hierarchy traversal
> Support TTL for views using the new column TTL in SYSTEM.CATALOG
>
>
> Key: PHOENIX-7040
> URL: https://issues.apache.org/jira/browse/PHOENIX-7040
> Project: Phoenix
> Issue Type: Sub-task
>Reporter: Jacob Isaac
>Assignee: Lokesh Khurana
>Priority: Major
>
> Allow views to be created with TTL specs.
> Ensure TTL is specified only once in the view hierarchy.
> Child views should inherit TTL values from their parent, when not specified
> for the given view.
> Indexes should inherit the TTL values from the base tables/views.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786107#comment-17786107 ] ASF GitHub Bot commented on PHOENIX-7040: - jpisaac commented on PR #1710: URL: https://github.com/apache/phoenix/pull/1710#issuecomment-1811542826 Setting TTL=NONE at the child level and then try to set to some value at a global view level gives the following exception. Inspecting the TTL value after setting it to NONE, shows a value of "0". Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 10956 (44A37): ERROR 10956 (44A37): TTL property is already defined in hierarchy for this entity tableName=[84, 69, 83, 84, 95, 69, 78, 84, 73, 84, 89].[71, 49, 95, 49, 48, 49] TEST_ENTITY.G1_101 at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:109) at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:83) at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:3555) at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.addColumn(MetaDataEndpointImpl.java:3641) at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:17569) at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7936) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2535) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2509) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45957) > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782801#comment-17782801 ] ASF GitHub Bot commented on PHOENIX-7040: - jpisaac commented on code in PR #1710: URL: https://github.com/apache/phoenix/pull/1710#discussion_r1382298507 ## phoenix-core/src/it/java/org/apache/phoenix/end2end/TTLAsPhoenixTTLIT.java: ## @@ -370,4 +455,62 @@ private String createViewOnViewWithTTL(Connection conn, String parentViewName, return childView; } + Review Comment: You may need to add tests for resetting TTL (TTL=NONE) at a given level and then setting (ALTER) to new TTL at a different level. Currently Alter table TTL=NONE sets it to 0 and thus cannot set a new TTL at a different level. ## phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewMetadataIT.java: ## @@ -63,6 +63,7 @@ import java.util.Map; import java.util.Properties; +import org.apache.commons.lang3.mutable.MutableBoolean; Review Comment: nit: Unused imports ## phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java: ## @@ -19,78 +19,20 @@ import static org.apache.hadoop.hbase.KeyValueUtil.createFirstOnRow; import static org.apache.phoenix.coprocessor.generated.MetaDataProtos.MutationCode.UNABLE_TO_CREATE_CHILD_LINK; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.APPEND_ONLY_SCHEMA_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ARRAY_SIZE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.AUTO_PARTITION_SEQ_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CHANGE_DETECTION_ENABLED_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CLASS_NAME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_DEF_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_QUALIFIER_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_QUALIFIER_COUNTER_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_VALUE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ENCODING_SCHEME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.EXTERNAL_SCHEMA_ID_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_DISABLE_TIMESTAMP_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_TYPE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_ARRAY_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_CONSTANT_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_NAMESPACE_MAPPED_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_ROW_TIMESTAMP_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_VIEW_REFERENCED_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.JAR_PATH_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.LAST_DDL_TIMESTAMP_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.LINK_TYPE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MAX_VALUE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MIN_VALUE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NUM_ARGS_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TTL_NOT_DEFINED; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.PHYSICAL_TABLE_NAME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.RETURN_TYPE_BYTES; -import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ROW_KEY_PREFIX_BYTES; -import static org.apache.phoenix.jdbc.Phoenix
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775862#comment-17775862 ] ASF GitHub Bot commented on PHOENIX-7040: - lokiore commented on PR #1710: URL: https://github.com/apache/phoenix/pull/1710#issuecomment-1764987409 I'll clean up the PR after tests are run. Need to add more tests as well > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (PHOENIX-7040) Support TTL for views using the new column TTL in SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775861#comment-17775861 ] ASF GitHub Bot commented on PHOENIX-7040: - lokiore opened a new pull request, #1710: URL: https://github.com/apache/phoenix/pull/1710 - Enabled TTL for Updatable Views - Store TTL at only one level where TTL is defined in hierarchy. - getTTL() should get TTL from hierarchy so while building PTable store get TTL from hierarchy and store it in PTable. - Altering should be allowed if TTL is not defined in up/down the hierarchy before. Altering TTL of a parent is not reflected directed to it's child's getTTL(), need to work on invalidate Cache for this. > Support TTL for views using the new column TTL in SYSTEM.CATALOG > > > Key: PHOENIX-7040 > URL: https://issues.apache.org/jira/browse/PHOENIX-7040 > Project: Phoenix > Issue Type: Sub-task >Reporter: Jacob Isaac >Assignee: Lokesh Khurana >Priority: Major > > Allow views to be created with TTL specs. > Ensure TTL is specified only once in the view hierarchy. > Child views should inherit TTL values from their parent, when not specified > for the given view. > Indexes should inherit the TTL values from the base tables/views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
