[
https://issues.apache.org/jira/browse/IMPALA-13607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18046070#comment-18046070
]
ASF subversion and git services commented on IMPALA-13607:
----------------------------------------------------------
Commit d4644b0381979c7be8b2c069d95a76b35894cabd in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d4644b038 ]
IMPALA-13607, IMPALA-14490: Deflake
test_cache_valid_on_nontransactional_table_ddls
When catalogd runs with --start_hms_server=true, it services all the HMS
endpoints so that any HMS compatible client would be able to use
catalogd as a metadata cache. For all the DDL/DML requests, catalogd
just delegates them to HMS APIs without reloading related metadata in
the cache. For read requests like get_table_req, catalogd serves them
from its cache which could be stale.
There is a flag, invalidate_hms_cache_on_ddls, to decide whether to
explicitly invalidate the table when catalogd delegates a DDL/DML on the
table to HMS. test_cache_valid_on_nontransactional_table_ddls is a test
verifying that when invalidate_hms_cache_on_ddls=false, the cache is not
updated so should have stale metadata.
However, there are HMS events generated from invoking the HMS APIs. Even
when invalidate_hms_cache_on_ddls=false, catalogd can still update its
cache when processing the corresponding HMS events. The test fails when
its check is done after catalogd applies the event (so the cache is
up-to-date). If the check is done before that, the test passes.
This patch deflakes the test by explicitly disabling event processing.
Also updates the description of invalidate_hms_cache_on_ddls to mention
the impact of event processing.
Tests:
- Ran the test locally 100 times.
Change-Id: Ib1ffc11a793899a0dbdb009bf2ac311117f2318e
Reviewed-on: http://gerrit.cloudera.org:8080/23792
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> test_cache_valid_on_nontransactional_table_ddls() fails with assertion error
> ----------------------------------------------------------------------------
>
> Key: IMPALA-13607
> URL: https://issues.apache.org/jira/browse/IMPALA-13607
> Project: IMPALA
> Issue Type: Bug
> Reporter: Pranav Yogi Lodha
> Assignee: Quanlong Huang
> Priority: Major
> Labels: broken-build
> Attachments:
> catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-1403.vpc.cloudera.com.jenkins.log.INFO.20251115-050206.2588537
>
>
> h2. Error Message
> {noformat}
> assert None is not None + where None = FileMetadata(data=None, version=1,
> type=1).data + where FileMetadata(data=None, version=1, type=1) =
> Partition(writeId=-1, parameters={'totalSize': '2', 'transient_lastDdlTime':
> '...csjtf.db/test_cache_valid_on_nontransactional_table_ddls_tblqchdb/part_col=2')).fileMetadata{noformat}
> h2. Stacktrace
> {noformat}
> custom_cluster/test_metastore_service.py:465: in
> test_cache_valid_on_nontransactional_table_ddls
> self.__test_non_transactional_table_cache_helper(db_name, tbl_name, False)
> custom_cluster/test_metastore_service.py:596: in
> __test_non_transactional_table_cache_helper assert part.fileMetadata.data is
> not None
> E assert None is not None
> E + where None = FileMetadata(data=None, version=1, type=1).data
> E + where FileMetadata(data=None, version=1, type=1) = Partition(writeId=-1,
> parameters={'totalSize': '2', 'transient_lastDdlTime':
> '...csjtf.db/test_cache_valid_on_nontransactional_table_ddls_tblqchdb/part_col=2')).fileMetadata{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
