[
https://issues.apache.org/jira/browse/OOZIE-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000036#comment-14000036
]
Purshotam Shah commented on OOZIE-1783:
---------------------------------------
There might be other issue with sharelib purging in HA.
Current purging logic
------
Keep all share-lib between current timestamp and 7days old + 1 latest sharelib
older than 7 days.
https://issues.apache.org/jira/browse/OOZIE-1761
-------
Each server should maintain their own sharelib, otherwise sharelib use by one
server might be purged.
Ex. two servers are running and both using different sharelib older then 7
days, one used sharelib will be purged.
At Yahoo, each server in HA has their own sharelib ( by setting different path
for oozie.service.WorkflowAppService.system.libpath).
For this approach we need to take out isFirstServer logic. So that each server
can purge their own sharelib.
{code}
if
(Services.get().get(JobsConcurrencyService.class).isFirstServer()) {
final Date current =
Calendar.getInstance(TimeZone.getTimeZone("GMT")).getTime();
purgeLibs(fs, LAUNCHER_PREFIX, current);
purgeLibs(fs, SHARED_LIB_PREFIX, current);
}
{code}
Other solution, sharelib service itself suffix hostname to path and works on
path which has hostname ( for all operations retrieving sharelib, update and
purging).
In this case also we need to take away isFirstServer logic.
> Sharelib purging only occurs at Oozie startup
> ---------------------------------------------
>
> Key: OOZIE-1783
> URL: https://issues.apache.org/jira/browse/OOZIE-1783
> Project: Oozie
> Issue Type: Bug
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Critical
> Attachments: OOZIE-1783.patch, OOZIE-1783.patch
>
>
> The purgeLibs method only gets called on startup of the first Oozie server.
> This means that if you update the sharelib without restarting Oozie, then
> Oozie will never clean up the old sharelibs and launcherlibs until you
> restart the server. In fact, with Oozie HA, it will never clean up unless
> you restart all of the servers at the same time (in other words, a rolling
> restart or just restarting one Oozie will never cause purgeLibs to get
> called).
> This should be moved into a background thread (i.e. scheduled with
> SchedulerService) to run periodically. Once a day is probably good enough,
> but we could make it configurable if needed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
