Puru, what do you think about my “lost+found” idea above? I’m concerned that users who upgraded will be confused about what sharelib they are using.
On Tue, Nov 19, 2013 at 10:30 AM, Robert Kanter <[email protected]>wrote: > The only concern I have about leaving them is that users could get > confused about which sharelib they are actually using. They’re used to > /user/oozie/share/lib/pig, and might think they’re using that when they > are really using /user/oozie/share/lib/lib_<timestamp>/pig. > > What if we moved everything that doesn’t have a prefix into a “lost and > found" folder instead of deleting it (e.g. > /user/oozie/share/lib/lost+found)? This way it would be more obvious > that Oozie is not using leftover directories/files but we wouldn’t be > deleting them in case the user needs those files for something. > > thoughts? > > > On Tue, Nov 19, 2013 at 10:08 AM, Purshotam Shah > <[email protected]>wrote: > >> Robert, >> Yes, old sharelib will not be purged by oozie, we need to delete them >> manually. >> >> >Would it make sense to simply delete anything that doesn¹t have the >> ³lib_² >> >or ³launcher_² prefix? >> >The only issues I see with that are that (a) someone could have put other >> >files/dirs in share/lib and (b) it won¹t catch files/dirs that happen to >> >have those prefixes (e.g. ³share/lib/lib_mystuff²). I think those two >> >cases are ok because the user shouldn¹t be putting other stuff in >> >/user/oozie/share/lib anyway. >> >> Oozie server only maintains and purge lib_<timestamp> and >> launcher_<timestamp> directory. >> >> >> Others will be ignored. We don't know the reason of existing so I believe >> it's better to ignore them. >> >> Puru. >> >> >> On 11/18/13 5:44 PM, "Robert Kanter" <[email protected]> wrote: >> >> >Hi, >> > >> >I was looking at the new sharelib stuff, and I¹m wondering what the story >> >is when switching from the old version to the new timestamped version. >> If >> >I had an existing sharelib from an older Oozie, then when I run the >> >oozie-setup.sh sharelib create tool, it results in this: >> > >> ># sudo -u oozie hadoop fs -ls share/lib >> > >> >Found 10 items >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/distcp >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/hcatalog >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:32 share/lib/hive >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:35 >> >share/lib/lib_20131118173502 >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 >> >share/lib/mapreduce-streaming >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/oozie >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/pig >> > >> >-rw-r--r-- 3 oozie oozie 1365 2013-11-18 17:33 >> >share/lib/sharelib.properties >> > >> >drwxr-xr-x - oozie oozie 0 2013-11-18 17:33 share/lib/sqoop >> > >> >I haven¹t tested it, but I assume that the old directories (e.g. >> >share/lib/pig), will never get cleaned up by Oozie because its only >> >looking >> >for the prefixes when purging, right? >> > >> >Would it make sense to simply delete anything that doesn¹t have the >> ³lib_² >> >or ³launcher_² prefix? >> >The only issues I see with that are that (a) someone could have put other >> >files/dirs in share/lib and (b) it won¹t catch files/dirs that happen to >> >have those prefixes (e.g. ³share/lib/lib_mystuff²). I think those two >> >cases are ok because the user shouldn¹t be putting other stuff in >> >/user/oozie/share/lib anyway. >> >Thoughts? >> > >> > >> >thanks >> >- Robert >> >> >
