[
https://issues.apache.org/jira/browse/IMPALA-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452117#comment-17452117
]
Quanlong Huang commented on IMPALA-10886:
-----------------------------------------
Start debugging on this intermittent issue. The failure happens when
coordinator only see 23 partitions on the alltypes table (after dropping a
partition and then inserting back the partition). The number should be 24,
including one partition that is missed in the local cache. If the number is 23,
coordinator will see all partitions are cached so fail the test.
{code:python}
def test_reuse_partition_meta(self, unique_database):
...
# Drop one partition. Although table version bumps, metadata cache of
existing
# partitions should be reusable.
self.execute_query(
"alter table %s.alltypes drop partition(year=2009, month=1)" %
unique_database)
self.check_missing_partitions(unique_database, 0)
# Add back one partition. The partition meta is loaded in catalogd but not
the
# coordinator. So we still miss its meta. For other partitions, we can
reuse them.
self.execute_query(
"insert into %s.alltypes partition(year=2009, month=1) "
"select 0,true,0,0,0,0,0,0,'a','a',NULL" % unique_database)
self.check_missing_partitions(unique_database, 1) # <--- Could only see 23
partitions here. Then fails.{code}
I add a patch to reveal the issue (test_local_catalog.patch). It seems a race
condition with the event processor. In my environment, the test fails once in
around 10 runs. If I turn off event processor, the test pass 100 runs. I'm
still looking into it deeper.
CC [~vihangk1], [~sourabh912], [~hsnusonic]
> TestReusePartitionMetadata.test_reuse_partition_meta fails
> ----------------------------------------------------------
>
> Key: IMPALA-10886
> URL: https://issues.apache.org/jira/browse/IMPALA-10886
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Labels: broken-build
> Attachments: test_local_catalog.patch
>
>
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/14670/testReport/junit/custom_cluster.test_local_catalog/TestReusePartitionMetadata/test_reuse_partition_meta/
> {code}
> custom_cluster/test_local_catalog.py:586: in test_reuse_partition_meta
> self.check_missing_partitions(unique_database, 1)
> custom_cluster/test_local_catalog.py:595: in check_missing_partitions
> assert match.group(1) == str(partition_misses)
> E assert '0' == '1'
> E - 0
> E + 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
