> On Feb. 6, 2015, 11:36 a.m., Himanshu Gahlaut wrote: > > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 281 > > <https://reviews.apache.org/r/30665/diff/3/?file=852109#file852109line281> > > > > Are we expecting stale files with older time stamps left in the > > directory due to crash cases or some other use cases ? If yes, will it be > > possible to delete and recreate the persistDir itself before starting the > > persistence ? This will help in cleaning the stale files > > > > OR > > > > if persist directory contains some other data files as well, then we > > can create a dedicated directory for state persistence which can be deleted > > and recreated every time persistence starts to clean any old stale files > > with older timestamps.
Deleting not closed files would fail, even for directory deletion. Since this number should be small, we can leave them now to be manually deleted. I have seen scenarios where namenode takes a lot time (in the order of hours) to figure out NOT CLOSED files. - Amareshwari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30665/#review71410 ----------------------------------------------------------- On Feb. 6, 2015, 11:27 a.m., Amareshwari Sriramadasu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30665/ > ----------------------------------------------------------- > > (Updated Feb. 6, 2015, 11:27 a.m.) > > > Review request for lens, Himanshu Gahlaut and Srikanth Sundarrajan. > > > Bugs: LENS-278 > https://issues.apache.org/jira/browse/LENS-278 > > > Repository: lens > > > Description > ------- > > Main issue is that rename is a no-op in HDFS when destination exists > > > Diffs > ----- > > lens-server/src/main/java/org/apache/lens/server/LensServices.java b1f087c > > Diff: https://reviews.apache.org/r/30665/diff/ > > > Testing > ------- > > Tested restart scenarios on with local file system. Couldn't get my docker > working. Will do testing with HDFS as part QA verification > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.084s] > [INFO] Lens .............................................. SUCCESS [1.761s] > [INFO] Lens API .......................................... SUCCESS [5.678s] > [INFO] Lens API for server and extensions ................ SUCCESS [5.726s] > [INFO] Lens Cube ......................................... SUCCESS [6:28.915s] > [INFO] Lens DB storage ................................... SUCCESS [10.423s] > [INFO] Lens Query Library ................................ SUCCESS [4.603s] > [INFO] Lens Hive Driver .................................. SUCCESS [2:39.451s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [25.482s] > [INFO] Lens Server ....................................... SUCCESS [4:34.883s] > [INFO] Lens client ....................................... SUCCESS [19.892s] > [INFO] Lens CLI .......................................... SUCCESS [1:33.574s] > [INFO] Lens Examples ..................................... SUCCESS [0.816s] > [INFO] Lens Distribution ................................. SUCCESS [3.792s] > [INFO] Lens Client Distribution .......................... SUCCESS [5.748s] > [INFO] Lens ML Lib ....................................... SUCCESS [43.244s] > [INFO] Lens Regression ................................... SUCCESS [0.841s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 17:27.795s > [INFO] Finished at: Thu Feb 05 13:02:53 UTC 2015 > [INFO] Final Memory: 99M/980M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Amareshwari Sriramadasu > >
