[
https://issues.apache.org/jira/browse/HIVE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789437#comment-16789437
]
Peter Vary commented on HIVE-21402:
-----------------------------------
I agree with [~ashutoshc] that this is most probably a config issue. I usually
use the same hive-site.xml for HMS and HS2. This might caused the problem.
There relevant configs are:
{code:java}
<property><name>hive.txn.manager</name><value>org.apache.hadoop.hive.ql.lockmgr.DbTxnManager</value></property>
<property><name>hive.support.concurrency</name><value>true</value></property>
<property><name>hive.compactor.initiator.on</name><value>true</value></property>
<property><name>hive.compactor.worker.threads</name><value>5</value></property>
<property><name>hive.compactor.crud.query.based</name><value>true</value></property>{code}
On the other hand this highlights the issue that we can have other Throwables
in that try-catch which might prevent setting the correct state of the
compaction. So I think this change should go in anyway. What do you think?
Also - maybe in a follow-up Jira - it would be good to store the reason for the
failure for the failed transactions. This could help troubleshooting
tremendously. [~vgumashta]: Is {{CompactionInfo.metaInfo}} is designed to store
info about the compaction - could it be used to store the exception message for
the failed compactions?
Thanks,
Peter
> Compaction state remains 'working' when major compaction fails
> --------------------------------------------------------------
>
> Key: HIVE-21402
> URL: https://issues.apache.org/jira/browse/HIVE-21402
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 4.0.0
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Attachments: HIVE-21402.patch
>
>
> When calcite is not on the HMS classpath, and query based compaction is
> enabled then the compaction fails with NoClassDefFound error. Since the catch
> block only catches Exceptions the following code block is not executed:
> {code:java}
> } catch (Exception e) {
> LOG.error("Caught exception while trying to compact " + ci +
> ". Marking failed to avoid repeated failures, " +
> StringUtils.stringifyException(e));
> msc.markFailed(CompactionInfo.compactionInfoToStruct(ci));
> msc.abortTxns(Collections.singletonList(compactorTxnId));
> }
> {code}
> So the compaction is not set to failed.
> Would be better to catch Throwable instead of Exception
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
