Karen Coppage created HIVE-25022:
------------------------------------
Summary: Metric about incomplete compactions
Key: HIVE-25022
URL: https://issues.apache.org/jira/browse/HIVE-25022
Project: Hive
Issue Type: Sub-task
Reporter: Karen Coppage
Assignee: Karen Coppage
"Compactions in a state" metrics (for example compaction_num_working) count the
sum of tables/partitions where the last compaction is in that state.
I propose introducing a new metric about incomplete compactions: i.e. the
number of tables/partitions where the last finished compaction* is unsuccessful
(failed or "did not initiate"), or where major compaction was unsuccessful then
minor compaction succeeded (compaction is not "complete" since major compaction
has not succeeded in the time since it should have run).
Example:
{code:java}
These compactions ran on a partition:
major succeeded
major working
major failed
major initiated
major working
major failed
major initiated
major working
The "compactions in a state" metrics will consider the state of this table:
working.
The "incomplete compactions" metric will consider this: incomplete, since there
have been failed compactions since the last succeeded compaction.
{code}
Another example:
{code:java}
These compactions ran on a partition:
major succeeded
major failed
minor failed
minor succeeded
The "compactions in a state" metrics will consider the state of this table:
succeeded.
The "incomplete compactions" metric will consider this: incomplete, since there
hasn't been a major succeeded since major failed.{code}
Last example:
{code:java}
These compactions ran on a partition:
major succeeded
minor did not initiate
The "compactions in a state" metrics will consider the state of this table: did
not initiate.
The "incomplete compactions" metric will consider this: incomplete, since the
last compaction was "did not initiate"{code}
*finished compaction: state in (succeeded, failed, attempted/did not initiate)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
