The only concern I have about leaving them is that users could get confused about which sharelib they are actually using. They’re used to /user/oozie/share/lib/pig, and might think they’re using that when they are really using /user/oozie/share/lib/lib_<timestamp>/pig.
What if we moved everything that doesn’t have a prefix into a “lost and found" folder instead of deleting it (e.g. /user/oozie/share/lib/lost+found)? This way it would be more obvious that Oozie is not using leftover directories/files but we wouldn’t be deleting them in case the user needs those files for something. thoughts? On Tue, Nov 19, 2013 at 10:08 AM, Purshotam Shah <[email protected]>wrote: > Robert, > Yes, old sharelib will not be purged by oozie, we need to delete them > manually. > > >Would it make sense to simply delete anything that doesn¹t have the ³lib_² > >or ³launcher_² prefix? > >The only issues I see with that are that (a) someone could have put other > >files/dirs in share/lib and (b) it won¹t catch files/dirs that happen to > >have those prefixes (e.g. ³share/lib/lib_mystuff²). I think those two > >cases are ok because the user shouldn¹t be putting other stuff in > >/user/oozie/share/lib anyway. > > Oozie server only maintains and purge lib_<timestamp> and > launcher_<timestamp> directory. > > > Others will be ignored. We don't know the reason of existing so I believe > it's better to ignore them. > > Puru. > > > On 11/18/13 5:44 PM, "Robert Kanter" <[email protected]> wrote: > > >Hi, > > > >I was looking at the new sharelib stuff, and I¹m wondering what the story > >is when switching from the old version to the new timestamped version. If > >I had an existing sharelib from an older Oozie, then when I run the > >oozie-setup.sh sharelib create tool, it results in this: > > > ># sudo -u oozie hadoop fs -ls share/lib > > > >Found 10 items > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/distcp > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/hcatalog > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/hive > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:35 > >share/lib/lib_20131118173502 > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 > >share/lib/mapreduce-streaming > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/oozie > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/pig > > > >-rw-r--r-- 3 oozie oozie 1365 2013-11-18 17:33 > >share/lib/sharelib.properties > > > >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/sqoop > > > >I haven¹t tested it, but I assume that the old directories (e.g. > >share/lib/pig), will never get cleaned up by Oozie because its only > >looking > >for the prefixes when purging, right? > > > >Would it make sense to simply delete anything that doesn¹t have the ³lib_² > >or ³launcher_² prefix? > >The only issues I see with that are that (a) someone could have put other > >files/dirs in share/lib and (b) it won¹t catch files/dirs that happen to > >have those prefixes (e.g. ³share/lib/lib_mystuff²). I think those two > >cases are ok because the user shouldn¹t be putting other stuff in > >/user/oozie/share/lib anyway. > >Thoughts? > > > > > >thanks > >- Robert > >
