Maxwell Guo created IMPALA-12771:
------------------------------------
Summary: Impala catalogd events-skipped may mark the wrong number
Key: IMPALA-12771
URL: https://issues.apache.org/jira/browse/IMPALA-12771
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Maxwell Guo
Assignee: Maxwell Guo
See the description of [event-skipped
metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237]
{code:java}
// total number of events which are skipped because of the flag setting or
// in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were
ignored
// because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in the
catalogd.
{code}
As for CREATE and DROP event on Database/Table/Partition (Also AddPartition is
inclued) when we found that the table/database when the database or table is
not found in the cache then we will skip the event process and make the
event-skipped metric +1.
But I found that there is some question here for alter table and Reload event:
* For alter table if renaming a table , the events-skipped metric will also +1
,see [oldTblRemoved to be false
|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L1653]
* For Reload event that is not describe in the description of events-skipped,
but the value is +1 when is oldevent;
* Besides if the table is in blacklist the metric will also +1
In summary, I think this description is inconsistent with the actual
implementation.
So can we also mark the events-skipped metric for alter partition events and
modify the
description to be all the events skipped
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
