[jira] [Updated] (LIVY-724) Support Session Lazy Recover

[Yiheng Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/LIVY-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yiheng Wang updated LIVY-724:
-
Description: 
In this JIRA, we want to support recover a session from the state store to Livy 
server memory when the session is used.

In the current single node recover, when a server restarts, it will recover the 
sessions in the state store in a batch.

In the multi-active scenario, we propose a new session recovery solution. When 
a server leaves the cluster, its sessions will be transferred to other servers. 
Instead of transferring them in a batch, each Livy session gets recovered when 
a request for it arrives. It will save the session switch time.

> Support Session Lazy Recover
> 
>
> Key: LIVY-724
> URL: https://issues.apache.org/jira/browse/LIVY-724
> Project: Livy
>  Issue Type: Sub-task
>Reporter: Yiheng Wang
>Priority: Major
>
> In this JIRA, we want to support recover a session from the state store to 
> Livy server memory when the session is used.
> In the current single node recover, when a server restarts, it will recover 
> the sessions in the state store in a batch.
> In the multi-active scenario, we propose a new session recovery solution. 
> When a server leaves the cluster, its sessions will be transferred to other 
> servers. Instead of transferring them in a batch, each Livy session gets 
> recovered when a request for it arrives. It will save the session switch time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (LIVY-723) Server Registration

[Yiheng Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/LIVY-723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yiheng Wang updated LIVY-723:
-
Description: 
In this JIRA, we want to implement a cluster manager which will monitor the 
cluster state change. Each Livy server can register itself to the cluster and 
get notified when a new node join or existing node leaves the cluster.

The cluster manager can be used in places like session allocation, service 
discovery, etc. 

The current implementation is based on zookeeper.

> Server Registration
> ---
>
> Key: LIVY-723
> URL: https://issues.apache.org/jira/browse/LIVY-723
> Project: Livy
>  Issue Type: Sub-task
>Reporter: Yiheng Wang
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In this JIRA, we want to implement a cluster manager which will monitor the 
> cluster state change. Each Livy server can register itself to the cluster and 
> get notified when a new node join or existing node leaves the cluster.
> The cluster manager can be used in places like session allocation, service 
> discovery, etc. 
> The current implementation is based on zookeeper.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)