John Sherman created HIVE-24549:
-----------------------------------
Summary: TxnManager should not be shared across queries
Key: HIVE-24549
URL: https://issues.apache.org/jira/browse/HIVE-24549
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 4.0.0
Reporter: John Sherman
Assignee: John Sherman
There are various sections of code that assume the DbTxnManager is not shared
across concurrent queries in a session.
Such as (which gets invoked during closeOperation):
[https://github.com/apache/hive/blob/3f5e01cae5b65dde7edb3fbde8ebe70c1d02f6cf/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L868-L885]
{code:java}
// is usually called after close() to commit or rollback a query and end the
driver life cycle.
// do not understand why it is needed and wonder if it could be combined with
close.
@Override
public void destroy() {
driverState.lock();
try {
// in the cancel case where the driver state is INTERRUPTED, destroy will
be deferred to
// the query process
if (driverState.isDestroyed()) {
return;
} else {
driverState.descroyed();
}
} finally {
driverState.unlock();
}
driverTxnHandler.destroy();
}
{code}
The problematic part is the: driverTxnHandler.destroy() which looks like:
{code:java}
void destroy() {
boolean isTxnOpen =
driverContext != null &&
driverContext.getTxnManager() != null &&
driverContext.getTxnManager().isTxnOpen();
release(!hiveLocks.isEmpty() || isTxnOpen);
}
{code}
What happens is (rough sketch):
Q1 - starts operation, acquires txn, does operation, closes txn/cleans up txn
info, starts fetching data
Q2 - starts operation, acquire txn
Q1 - calls close operation which in turn calls destroy which sees the Q2s
transaction information and cleans it up.
Q2 - proceeds and fails in splitGeneration when it no longer can find its
Valid*TxnIdList information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
