Yiheng Wang created LIVY-696:
--------------------------------
Summary: Support recovery in livy thrift server
Key: LIVY-696
URL: https://issues.apache.org/jira/browse/LIVY-696
Project: Livy
Issue Type: Improvement
Components: Thriftserver
Reporter: Yiheng Wang
This JIRA is opened to discuss support recovery in Livy Thrift server.
Livy server support recovery. If user set *livy.server.recovery.mode* to
*recovery* and set the *state-store*, livy server will persist its states. Once
livy server crashes, the sessions will be restored after the service restart.
The recovery is not yet supported in thrift server. After the service restarts,
all JDBC sessions and operations will be lost.
Should we support recovery in the thrift server? There's one concern. In JDBC
binary mode, the connection will be lost if the server crashes, and looks like
hive-jdbc cannot reuse session-id when reconnecting to the server. But in JDBC
http mode, we will benefit from the recovery. As http mode use short http
connections instead of long connection.
There're also two levels of recovery need to be considered.
*Session recovery* let user can still use their JDBC connection after service
restart. It should be straight forward to implement. We just need to persist
the JDBC session/livy session mapping to the state-store and recover the
mapping when restarting.
*Operation recovery* let user can still use the JDBC statement(or fetch result)
after service restart. This needs to persist ExecuteStatementOperation state.
The concern is some states are not suitable to persist to state-store, e.g.
operationMessages, rowOffset, state, operationException, backgroundHandle,
lastAccessTime, operationComplete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
