----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69554/#review212235 -----------------------------------------------------------
lens-client/src/test/java/org/apache/lens/client/TestLensClient.java Line 292 (original) <https://reviews.apache.org/r/69554/#comment297958> Why is fail assert removed? No more socket exception expected? If so, whats up with catch block below? lens-server/src/main/java/org/apache/lens/server/BaseLensService.java Line 571 (original), 577 (patched) <https://reviews.apache.org/r/69554/#comment297959> why is method renamed to get, which is not returning any session here? lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java Lines 112 (patched) <https://reviews.apache.org/r/69554/#comment297960> remove commented code? - Amareshwari Sriramadasu On Jan. 22, 2019, 11:08 a.m., Ankit Kailaswar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69554/ > ----------------------------------------------------------- > > (Updated Jan. 22, 2019, 11:08 a.m.) > > > Review request for lens, Amareshwari Sriramadasu and Rajitha R. > > > Bugs: LENS-1538 > https://issues.apache.org/jira/browse/LENS-1538 > > > Repository: lens > > > Description > ------- > > Implementation > > Session > > We need to persist all client session in database. > This involve persisting client session’s id and configuration in mysql > server. persisting entire session object is not required. We need to do this > as first thing while creating user session. > For each request from client lens server should not validate session id > from its in memory map. It should rather validate it from persisted session > in mysql server. > > if > > session is present in mysql and not present in lens server’s in memory > map(session manager) then it should create a lens session with same session > id and add it to in memory map > > else If > > session is not present in mysql and present in server’s in memory map then it > signifies that user have initiated close session from other host and we will > need to remove this session from current host’s in memory map as well > > else if > > session is not present in mysql and in server’s in memory map then return > “invalid session”. > > Whenever session expired/closed then we need to remove this session from > lens server’s in memory map and mysql. > > > > Query > > For sync queries lens server will close connection as soon as it is > stopped or failed with appropriate failure message. Client will be retrying > the query. > For async-light queries and async-heavy queries lens server takes care of > rescheduling all queries which were in running or queued state at the time of > restart. we can use this to make query ids available with both servers. > For async queries we need to persist all non finished async user queries > in mysql server in new table current_query. > If user executes query on host1 and it goes down and nginx points to > host2 then user should be able to poll on query status. Lens server will > first check query id in its inmemory maps if it is not present then it will > check in “finished_query” table in mysql else in “current_query” table in > mysql. In this case we will continue showing old status of query since it is > scheduled by host1. As soon as host1 comes up it will reschedule these > queries and will change the status. Optionally we can have host2 to move > these queries in “allqueries” map of its query service which will take care > of recovering, in this case we don't need to wait for host1 to come up and > reschedule. > While moving query from ”finished_quey” map to “finished_query” table in > mysql we will need to remove it from “current_query” tables as well. > This flow will remain same for request ids/query ids generated for query > plan/estimate. > > > > Issue addressed > > Lens downtime will be zero in case of deployment and if primary fail. > User will not have to create a new session whenever switch happens > User will be able to get status of all async queries seamlessly. > > > Build Log : > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.740s] > [INFO] Lens .............................................. SUCCESS [5.788s] > [INFO] Lens API .......................................... SUCCESS [48.711s] > [INFO] Lens API for server and extensions ................ SUCCESS [33.537s] > [INFO] Lens Cube ......................................... SUCCESS [7:43.672s] > [INFO] Lens DB storage ................................... SUCCESS [29.279s] > [INFO] Lens Query Library ................................ SUCCESS [22.507s] > [INFO] Lens Hive Driver .................................. SUCCESS [1:27.344s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [1:24.416s] > [INFO] Lens Elastic Search Driver ........................ SUCCESS [44.067s] > [INFO] Lens Server ....................................... SUCCESS > [12:19.915s] > [INFO] Lens client ....................................... SUCCESS [1:19.368s] > [INFO] Lens CLI .......................................... SUCCESS [1:19.672s] > [INFO] Lens Examples ..................................... SUCCESS [7.287s] > [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.661s] > [INFO] Lens Distribution ................................. SUCCESS [10.625s] > [INFO] Lens ML Lib ....................................... SUCCESS [53.247s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.418s] > [INFO] Lens Regression ................................... SUCCESS [9.296s] > [INFO] Lens UI ........................................... SUCCESS [28.750s] > [INFO] Lens Contrib ...................................... SUCCESS [0.229s] > [INFO] Lens Contributed Clients .......................... SUCCESS [0.226s] > [INFO] Lens Python Client ................................ SUCCESS [0.231s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 30:34.060s > [INFO] Finished at: Thu Jan 17 10:47:01 UTC 2019 > [INFO] Final Memory: 300M/3097M > [INFO] > ------------------------------------------------------------------------ > [Platform_Common_Job] $ /bin/sh -xe /tmp/jenkins5693507200862759570.sh > + ruby /d0/jenkins/scripts/licenseCheck.rb > /d0/jenkins/workspace/Platform_Common_Job > 0 > > > Diffs > ----- > > lens-client/src/test/java/org/apache/lens/client/TestLensClient.java > d9e60fb7 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java > a8031098 > lens-server/src/main/java/org/apache/lens/server/BaseLensService.java > 9364872d > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java > cc6ca7d4 > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > 07a2107a > > lens-server/src/main/java/org/apache/lens/server/query/QueryServiceResource.java > 47b40a8f > > lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java > f6d43d70 > lens-server/src/test/java/org/apache/lens/server/TestBaseLensService.java > PRE-CREATION > lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java > 066525b7 > > > Diff: https://reviews.apache.org/r/69554/diff/8/ > > > Testing > ------- > > Unit testing > > > Thanks, > > Ankit Kailaswar > >
