[jira] [Updated] (LIVY-721) Distributed Session ID Generation
[ https://issues.apache.org/jira/browse/LIVY-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated LIVY-721: Description: # When generate unique session id with multiple-active HA mode. First, get the distributed lock, Second, get the session id from filesystem or zookeeper. Third, increase the session id and save it in the filesystem or zookeeper. Forth, release the distributed lock. # ZooKeeperManager provides the distributed lock to generate the distributed session id. > Distributed Session ID Generation > - > > Key: LIVY-721 > URL: https://issues.apache.org/jira/browse/LIVY-721 > Project: Livy > Issue Type: Sub-task >Reporter: Yiheng Wang >Priority: Major > > # When generate unique session id with multiple-active HA mode. First, get > the distributed lock, > Second, get the session id from filesystem or zookeeper. Third, increase the > session id and save it in the filesystem or zookeeper. Forth, release the > distributed lock. > # ZooKeeperManager provides the distributed lock to generate the distributed > session id. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (LIVY-721) Distributed Session ID Generation
[ https://issues.apache.org/jira/browse/LIVY-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated LIVY-721: Description: # When generate unique session id with multiple-active HA mode. First, get the distributed lock, Second, get the session id from filesystem or zookeeper. Third, increase the session id and save it in the filesystem or zookeeper. Forth, release the distributed lock. # ZooKeeperManager provides the distributed lock to generate the distributed session id. was: # When generate unique session id with multiple-active HA mode. First, get the distributed lock, Second, get the session id from filesystem or zookeeper. Third, increase the session id and save it in the filesystem or zookeeper. Forth, release the distributed lock. # ZooKeeperManager provides the distributed lock to generate the distributed session id. > Distributed Session ID Generation > - > > Key: LIVY-721 > URL: https://issues.apache.org/jira/browse/LIVY-721 > Project: Livy > Issue Type: Sub-task >Reporter: Yiheng Wang >Priority: Major > > # When generate unique session id with multiple-active HA mode. First, get > the distributed lock, > Second, get the session id from filesystem or zookeeper. Third, increase the > session id and save it in the filesystem or zookeeper. Forth, release the > distributed lock. > # ZooKeeperManager provides the distributed lock to generate the distributed > session id. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (LIVY-723) Server Registration
[ https://issues.apache.org/jira/browse/LIVY-723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015609#comment-17015609 ] Yiheng Wang commented on LIVY-723: -- I'm working on it. > Server Registration > --- > > Key: LIVY-723 > URL: https://issues.apache.org/jira/browse/LIVY-723 > Project: Livy > Issue Type: Sub-task >Reporter: Yiheng Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (LIVY-744) Livy Job API support multi-active HA
Yiheng Wang created LIVY-744: Summary: Livy Job API support multi-active HA Key: LIVY-744 URL: https://issues.apache.org/jira/browse/LIVY-744 Project: Livy Issue Type: Sub-task Reporter: Yiheng Wang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (LIVY-724) Support Session Lazy Recover
[ https://issues.apache.org/jira/browse/LIVY-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiheng Wang updated LIVY-724: - Summary: Support Session Lazy Recover (was: Support Server Leave/Join and Session Cross Server Recovery) > Support Session Lazy Recover > > > Key: LIVY-724 > URL: https://issues.apache.org/jira/browse/LIVY-724 > Project: Livy > Issue Type: Sub-task >Reporter: Yiheng Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (LIVY-718) Support multi-active high availability in Livy
[ https://issues.apache.org/jira/browse/LIVY-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015606#comment-17015606 ] Yiheng Wang commented on LIVY-718: -- I updated the design doc based on the recent discussions in the JIRA. The major changes are: # Refine the solution architecture section # Add a new allocateServer method to allocator interface # Add details for node-session mapping allocation method [~mgaido] # update getAllSession [~mgaido] # Refine the section of comparing client-side routing and server-side routing # Add a new section "Load Balancer", which gives example how to put livy servers behind a load balancer when using client-side routing [~bikassaha] [~meisam] # Add a new section "Session Recover", which describe we recover a session object in a lazy way(when a request for that session arrives, which can be leveraged in multi-designated server solution) [~bikassaha] # Remove the session recover when there's server failover # Add multi-designate server to non-goal and add a new section "Multi-designate Server Solution Extension" to discuss how to continue to implement a multi-designate server solution from the current one [~bikassaha] > Support multi-active high availability in Livy > -- > > Key: LIVY-718 > URL: https://issues.apache.org/jira/browse/LIVY-718 > Project: Livy > Issue Type: Epic > Components: RSC, Server >Reporter: Yiheng Wang >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In this JIRA we want to discuss how to implement multi-active high > availability in Livy. > Currently, Livy only supports single node recovery. This is not sufficient in > some production environments. In our scenario, the Livy server serves many > notebook and JDBC services. We want to make Livy service more fault-tolerant > and scalable. > There're already some proposals in the community for high availability. But > they're not so complete or just for active-standby high availability. So we > propose a multi-active high availability design to achieve the following > goals: > # One or more servers will serve the client requests at the same time. > # Sessions are allocated among different servers. > # When one node crashes, the affected sessions will be moved to other active > services. > Here's our design document, please review and comment: > https://docs.google.com/document/d/1bD3qYZpw14_NuCcSGUOfqQ0pqvSbCQsOLFuZp26Ohjc/edit?usp=sharing > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (LIVY-725) Support Route Request
[ https://issues.apache.org/jira/browse/LIVY-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiheng Wang reassigned LIVY-725: Assignee: (was: mingchao zhao) > Support Route Request > - > > Key: LIVY-725 > URL: https://issues.apache.org/jira/browse/LIVY-725 > Project: Livy > Issue Type: Sub-task >Reporter: Yiheng Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
