----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69554/#review211719 -----------------------------------------------------------
lens-server/src/main/java/org/apache/lens/server/BaseLensService.java Lines 133 (patched) <https://reviews.apache.org/r/69554/#comment297271> made it static final. lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java Lines 1176 (patched) <https://reviews.apache.org/r/69554/#comment297274> We have a counters for insert and delete. update will be intermidiate operation between these 2 calls though of not having one more extra counter. Added though. lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java Lines 91 (patched) <https://reviews.apache.org/r/69554/#comment297272> this is not required here, can use base class instance. removed as member variable. lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java Lines 397 (patched) <https://reviews.apache.org/r/69554/#comment297273> removed - Ankit Kailaswar On Jan. 8, 2019, 10:52 a.m., Ankit Kailaswar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69554/ > ----------------------------------------------------------- > > (Updated Jan. 8, 2019, 10:52 a.m.) > > > Review request for lens, Amareshwari Sriramadasu and Rajitha R. > > > Bugs: LENS-1538 > https://issues.apache.org/jira/browse/LENS-1538 > > > Repository: lens > > > Description > ------- > > Implementation > > Session > > We need to persist all client session in database. > This involve persisting client session’s id and configuration in mysql > server. persisting entire session object is not required. We need to do this > as first thing while creating user session. > For each request from client lens server should not validate session id > from its in memory map. It should rather validate it from persisted session > in mysql server. > > if > > session is present in mysql and not present in lens server’s in memory > map(session manager) then it should create a lens session with same session > id and add it to in memory map > > else If > > session is not present in mysql and present in server’s in memory map then it > signifies that user have initiated close session from other host and we will > need to remove this session from current host’s in memory map as well > > else if > > session is not present in mysql and in server’s in memory map then return > “invalid session”. > > Whenever session expired/closed then we need to remove this session from > lens server’s in memory map and mysql. > > > > Query > > For sync queries lens server will close connection as soon as it is > stopped or failed with appropriate failure message. Client will be retrying > the query. > For async-light queries and async-heavy queries lens server takes care of > rescheduling all queries which were in running or queued state at the time of > restart. we can use this to make query ids available with both servers. > For async queries we need to persist all non finished async user queries > in mysql server in new table current_query. > If user executes query on host1 and it goes down and nginx points to > host2 then user should be able to poll on query status. Lens server will > first check query id in its inmemory maps if it is not present then it will > check in “finished_query” table in mysql else in “current_query” table in > mysql. In this case we will continue showing old status of query since it is > scheduled by host1. As soon as host1 comes up it will reschedule these > queries and will change the status. Optionally we can have host2 to move > these queries in “allqueries” map of its query service which will take care > of recovering, in this case we don't need to wait for host1 to come up and > reschedule. > While moving query from ”finished_quey” map to “finished_query” table in > mysql we will need to remove it from “current_query” tables as well. > This flow will remain same for request ids/query ids generated for query > plan/estimate. > > > > Issue addressed > > Lens downtime will be zero in case of deployment and if primary fail. > User will not have to create a new session whenever switch happens > User will be able to get status of all async queries seamlessly. > > > Diffs > ----- > > > lens-server-api/src/main/java/org/apache/lens/server/api/SessionValidator.java > 65403191 > > lens-server-api/src/main/java/org/apache/lens/server/api/events/AsyncEventListener.java > 94734658 > lens-server/src/main/java/org/apache/lens/server/BaseLensService.java > 9364872d > > lens-server/src/main/java/org/apache/lens/server/error/LensServerErrorCode.java > ef43371a > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java > cc6ca7d4 > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > 07a2107a > > lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java > f6d43d70 > lens-server/src/test/java/org/apache/lens/server/TestBaseLensService.java > PRE-CREATION > lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java > 066525b7 > > > Diff: https://reviews.apache.org/r/69554/diff/7/ > > > Testing > ------- > > Unit testing > > > Thanks, > > Ankit Kailaswar > >
