[
https://issues.apache.org/jira/browse/HIVE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viktor Csomor updated HIVE-25914:
---------------------------------
Description:
The Cleaner updates the Initiator metric.
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java#L175
Create some test for Initiator and Cleaner to avoid this happen.
was:
The count of the below metrics counted incorrectly upon an exception.
- {{compaction_initator_failure_counter}}
- {{compaction_cleaner_failure_counter}}
Reasoning:
In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}}
which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}.
The below code-snippet waits all cleaners to complete (Initiators does it
similarly).
{code:java}
try {
....
for (CompactionInfo compactionInfo : readyToClean) {
cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(()
->
clean(compactionInfo, cleanerWaterMark, metricsEnabled)),
cleanerExecutor));
}
CompletableFuture.allOf(cleanerList.toArray(new
CompletableFuture[0])).join();
}
} catch (Throwable t) {
// the lock timeout on AUX lock, should be ignored.
if (metricsEnabled && handle != null) {
failuresCounter.inc();
}
{code}
If the {{CompleteableFututre#join}} throws an Exception then the failure
counter is incremented.
Docs:
{code}
/**
* Returns the result value when complete, or throws an
* (unchecked) exception if completed exceptionally. To better
* conform with the use of common functional forms, if a
* computation involved in the completion of this
* CompletableFuture threw an exception, this method throws an
* (unchecked) {@link CompletionException} with the underlying
* exception as its cause.
*
* @return the result value
* @throws CancellationException if the computation was cancelled
* @throws CompletionException if this future completed
* exceptionally or a completion computation threw an exception
*/
public T join() {
Object r;
return reportJoin((r = result) == null ? waitingGet(false) : r);
}
{code}
(!) Let's suppose we have 10 cleaners and the 2nd throws an exception. The
{{catch}} block will be initiated and the {{failuresCounter}} will be
incremented. If there is any consecutive error amongst the remaining cleaners
the counter won't be incremented.
> Cleaner updates Initiator cycle metric
> --------------------------------------
>
> Key: HIVE-25914
> URL: https://issues.apache.org/jira/browse/HIVE-25914
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 4.0.0
> Reporter: Viktor Csomor
> Assignee: Viktor Csomor
> Priority: Minor
> Labels: pull-request-available
>
> The Cleaner updates the Initiator metric.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java#L175
> Create some test for Initiator and Cleaner to avoid this happen.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
