[ https://issues.apache.org/jira/browse/LENS-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated LENS-1538: ------------------------------------------ Resolution: Fixed Fix Version/s: 2.8 Status: Resolved (was: Patch Available) Attached patch from review board here and committed the same. Thanks [~ankitkailaswar] for the contribution ! > Lens HA changes > --------------- > > Key: LENS-1538 > URL: https://issues.apache.org/jira/browse/LENS-1538 > Project: Apache Lens > Issue Type: Improvement > Components: server > Reporter: Ankit Kailaswar > Assignee: Ankit Kailaswar > Priority: Major > Fix For: 2.8 > > Attachments: Lens-1538_8.diff, Lens-1538_9.diff > > > # > h1. Implementation > # > h2. Session > # We need to persist all client session in database. > # This involve persisting client session’s id and configuration in mysql > server. persisting entire session object is not required. We need to do this > as first thing while creating user session. > # For each request from client lens server should not validate session id > from its in memory map. It should rather validate it from persisted session > in mysql server. > if > session is present in mysql and not present in lens server’s in memory > map(session manager) then it should create a lens session with same session > id and add it to in memory map > else If > session is not present in mysql and present in server’s in memory map then it > signifies that user have initiated close session from other host and we will > need to remove this session from current host’s in memory map as well > else if > session is not present in mysql and in server’s in memory map then return > “invalid session”. > # Whenever session expired/closed then we need to remove this session from > lens server’s in memory map and mysql. > > # > h2. *Query* > # For sync queries lens server will close connection as soon as it is > stopped or failed with appropriate failure message. Client will be retrying > the query. > # For async-light queries and async-heavy queries lens server takes care of > rescheduling all queries which were in running or queued state at the time of > restart. we can use this to make query ids available with both servers. > # For async queries we need to persist all non finished async user queries > in mysql server in new table current_query. > # If user executes query on host1 and it goes down and nginx points to host2 > then user should be able to poll on query status. Lens server will first > check query id in its inmemory maps if it is not present then it will check > in “finished_query” table in mysql else in “current_query” table in mysql. In > this case we will continue showing old status of query since it is scheduled > by host1. As soon as host1 comes up it will reschedule these queries and will > change the status. Optionally we can have host2 to move these queries in > “allqueries” map of its query service which will take care of recovering, in > this case we don't need to wait for host1 to come up and reschedule. > # While moving query from ”finished_quey” map to “finished_query” table in > mysql we will need to remove it from “current_query” tables as well. > # This flow will remain same for request ids/query ids generated for query > plan/estimate. > > # > h1. Issue addressed > # Lens downtime will be zero in case of deployment and if primary fail. > # User will not have to create a new session whenever switch happens > # User will be able to get status of all async queries seamlessly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)