Quanlong Huang created IMPALA-9101:
--------------------------------------
Summary: Unneccessary REFRESH due to wrong self-event detection
Key: IMPALA-9101
URL: https://issues.apache.org/jira/browse/IMPALA-9101
Project: IMPALA
Issue Type: Bug
Reporter: Quanlong Huang
In {{CatalogOpExecutor.alterTable()}}, we call
{{addVersionsForInflightEvents()}} whenever the AlterTable operation changes
anything or not. If nothing changes, no HMS RPCs are sent. The event processor
ends up waiting on a non-existed self-event. Then all self-events are treated
as outside events and unneccessary REFRESH/INVALIDATE on this table will be
performed.
Codes:
{code:java}
private void alterTable(TAlterTableParams params, TDdlExecResponse response)
throws ImpalaException {
....
tryLock(tbl);
// Get a new catalog version to assign to the table being altered.
long newCatalogVersion = catalog_.incrementAndGetCatalogVersion();
addCatalogServiceIdentifiers(tbl, catalog_.getCatalogServiceId(),
newCatalogVersion);
....
// now that HMS alter operation has succeeded, add this version to list
of inflight
// events in catalog table if event processing is enabled
catalog_.addVersionsForInflightEvents(tbl, newCatalogVersion); <----
We should check before calling this.
}
{code}
Reproduce:
{code:sql}
create table testtbl (col int) partitioned by (p1 int, p2 int);
alter table testtbl add partition (p1=2,p2=6);
alter table testtbl add if not exists partition (p1=2,p2=6);
-- After this point, can't detect self-events on this table
alter table testtbl add partition (p1=2,p2=7);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
