> On Dec. 25, 2014, 2:56 p.m., Amareshwari Sriramadasu wrote: > > lens-server/src/main/java/org/apache/lens/server/LensServer.java, line 161 > > <https://reviews.apache.org/r/29416/diff/1/?file=800475#file800475line161> > > > > We are using signal handler instead of shutdown hook as we were seeing > > issues with shutdown hook. I dont remember all the issues we faced, but can > > spend some time and put them here. > > Srikanth Sundarrajan wrote: > @Amareshwari, Can you share the issues either here on the JIRA once you > recall them ? Thanks > > Amareshwari Sriramadasu wrote: > With shutdown hook, we saw scenarios where stop was not getting called in > some runs of the server. It was very random. > > After going thru- > http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread) > > Here are the issues i could recollect : > > 1. Shutdown hooks should also finish their work quickly > >> Because of this, we did not want to call stop from shutdown hook > directly as LensServer.stop is involving a file system operation to write > server state to file system. So moved stop() call to main thread after join. > > 2. Note that daemon threads will continue to run during the shutdown > sequence, as will non-daemon threads if shutdown was initiated by invoking > the exit method. > >> I couldnot understand the above from api doc. I'm not sure if calling > stop() in main thread is the reason for stop not getting called all times. > > 3. If we are going with shutdown hooks - "When the virtual machine begins > its shutdown sequence it will start all registered shutdown hooks in some > unspecified order and let them run concurrently. " > >> During server stop we are writing server state to file system, so we > need to set fs.automatic.close to false in lensserver-default.xml : > > <property> > <name>fs.automatic.close</name> > <value>true</value> > <description>By default, FileSystem instances are automatically closed > at program > exit using a JVM shutdown hook. Setting this property to false disables > this > behavior. This is an advanced option that should only be used by server > applications > requiring a more carefully orchestrated shutdown sequence. > </description> > </property>
Since we are using fs objects to persist elements and hadoop automatic close comes in the way of this, we have to disable this, but then we will have to perform filesystem close instead using FileSystem::closeAll(). More specifically on the concern relating to shutdown hook completing fast enough, it is not so much to do with what is inside the shutdown hook. but how long it takes for the cleanup to complete. I guess it doesn't help the cause if another non-daemon thread keeps the JVM running well after the SIGTERM is received. - Srikanth ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29416/#review66098 ----------------------------------------------------------- On Dec. 26, 2014, 12:49 p.m., Srikanth Sundarrajan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29416/ > ----------------------------------------------------------- > > (Updated Dec. 26, 2014, 12:49 p.m.) > > > Review request for lens. > > > Bugs: LENS-163 > https://issues.apache.org/jira/browse/LENS-163 > > > Repository: lens > > > Description > ------- > > LensServer imports the following dependencies > > import sun.misc.Signal; > import sun.misc.SignalHandler; > > and this is being used to register essentially shutdownhook. > > ... > Signal.handle(new Signal("TERM"), new SignalHandler() { > > @Override > public void handle(Signal signal) { > ... > > We should use Runtime::addShutdownHook() instead. > > > Diffs > ----- > > lens-server/pom.xml eee6123 > lens-server/src/main/java/org/apache/lens/server/LensServer.java 66abbcd > lens-server/src/main/resources/lens-build-info.properties PRE-CREATION > pom.xml 9265a77 > > Diff: https://reviews.apache.org/r/29416/diff/ > > > Testing > ------- > > No new tests added. Existing tests ran alright. > > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [0.937s] > [INFO] Lens .............................................. SUCCESS [3.139s] > [INFO] Lens API .......................................... SUCCESS [5.696s] > [INFO] Lens API for server and extensions ................ SUCCESS [3.089s] > [INFO] Lens Cube ......................................... SUCCESS [5:21.532s] > [INFO] Lens DB storage ................................... SUCCESS [28.502s] > [INFO] Lens Query Library ................................ SUCCESS [19.518s] > [INFO] Lens Hive Driver .................................. SUCCESS > [11:37.892s] > [INFO] Lens Driver for Cloudera Impala ................... SUCCESS [2.718s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [36.736s] > [INFO] Lens Server ....................................... SUCCESS > [12:09.370s] > [INFO] Lens client ....................................... SUCCESS [54.261s] > [INFO] Lens CLI .......................................... SUCCESS [3:44.160s] > [INFO] Lens Examples ..................................... SUCCESS [0.534s] > [INFO] Lens Distribution ................................. SUCCESS [3.451s] > [INFO] Lens Client Distribution .......................... SUCCESS [2.473s] > [INFO] Lens ML Lib ....................................... SUCCESS [1:36.809s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 37:11.229s > [INFO] Finished at: Thu Dec 25 12:43:50 IST 2014 > [INFO] Final Memory: 135M/801M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Srikanth Sundarrajan > >
