[
https://issues.apache.org/jira/browse/IMPALA-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated IMPALA-8489:
--------------------------------
Summary: TestRecoverPartitions.test_post_invalidate fails with
IllegalStateException when HMS polling is enabled (was:
TestRecoverPartitions.test_post_invalidate fails with IllegalStateException
with local catalog)
I can reproduce this by just enabling polling (and not LocalCatalog). Updated
the title appropriately.
The issue seems to be in CatalogOpExecutor.updateCatalog handling of partitions
that were touched by an insert. It comes up with a list of partition IDs that
were modified by the insert, then calls loadTableMetadata() which refreshes
those partitions. Because the partition was added by ALTER TABLE RECOVER
PARTITIONS, it got marked as "dirty" which means that the refresh ends up
dropping and reloading it with a new partition ID. Then, createInsertEvents
looks for the partitions by ID, but they've since been assigned new IDs, so
they aren't found.
Digging into this a bit more to see if I can see why this affects this code
path but not others that also use the "dirty partition" hack
> TestRecoverPartitions.test_post_invalidate fails with IllegalStateException
> when HMS polling is enabled
> -------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-8489
> URL: https://issues.apache.org/jira/browse/IMPALA-8489
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Todd Lipcon
> Priority: Critical
>
> {noformat}
> metadata/test_recover_partitions.py:279: in test_post_invalidate
> "INSERT INTO TABLE %s PARTITION(i=002, p='p2') VALUES(4)" % FQ_TBL_NAME)
> common/impala_test_suite.py:620: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:628: in execute_query_expect_success
> result = cls.__execute_query(impalad_client, query, query_options, user)
> common/impala_test_suite.py:722: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:364: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:385: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:IllegalArgumentException: no such partition id 6244
> {noformat}
> The failure is reproducible for me locally with catalog v2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
