Nothing helps. Looks like I'm missing somthing. Oozie works, but there is no purge happens.
2013/10/4 Serega Sheypak <[email protected]> > purge.executions= 11 > in counters section, but nothing happens or I don't see i > > > 2013/10/4 Serega Sheypak <[email protected]> > >> Hi, I've tried this one: >> >> <property> >> <name>oozie.services.ext</name> >> <value> >> org.apache.oozie.service.PurgeService >> </value> >> </property> >> >> <!-- Purging old materializations--> >> <!-- Keep them for three months --> >> <property> >> <name>oozie.service.PurgeService.older.than</name> >> <value>90</value> >> </property> >> >> <property> >> <name>oozie.service.PurgeService.coord.older.than</name> >> <value>90</value> >> </property> >> >> <property> >> <name>oozie.service.PurgeService.purge.limit</name> >> <value>100</value> >> </property> >> >> <property> >> <name>oozie.service.PurgeService.purge.interval</name> >> <value>100</value> >> </property> >> >> <!-- We need it because of NPE in PurgeService. It needs all Purge >> properties to be explicitly set--> >> <property> >> <name>oozie.service.PurgeService.bundle.older.than</name> >> <value>14</value> >> <description> >> Completed bundle jobs older than this value, in days, will be >> purged by the PurgeService. >> </description> >> </property> >> >> Oozie is started, cool, I forgot about oozie-default.xml. Thank you. >> The problem is here I still see last 900+ coordinator materializations. >> Nothing has been deleted. >> 100 seconds (oozie.service.PurgeService.purge.interval) did pass. >> >> There is nothing related to PurgeSerivce in log. >> >> 1. What do I do wrong? >> 2. oozie.service.PurgeService.purge.limit=100 and >> oozie.service.PurgeService.older.than=90 (days) >> What has more power? Or one of these conditions should trigger >> PurgeService to cleanup oozie DB? >> >> >> >> 2013/10/2 Robert Kanter <[email protected]> >> >>> CDH does have that property. In both CDH and Apache distributions, that >>> property (and some others) is specified in oozie-default.xml. There >>> should >>> be a copy of its somewhere. Oozie has its own internal copy of >>> oozie-default.xml, so don't bother editing the one you find; Oozie loads >>> its oozie-default.xml first and then loads oozie-site.xml with any >>> properties you set (Hadoop works the same way with core-default.xml and >>> core-site.xml). >>> >>> The NPE is because you only have the two services, PurgeService and >>> SchedulerService, >>> loaded because you overrode the oozie.services property. The NPE occurs >>> because Oozie does need the ConfigurationService for sure to work >>> properly; >>> and without many of the other services, it will likely have other issues. >>> >>> You should remove your override of oozie.services from oozie-site.xml. >>> Check oozie-default.xml and see if it lists PurgeService under >>> oozie.services. If it does not, then you can add it in your >>> oozie-site.xml >>> in the oozie.services.ext property; if it does, then the PurgeService is >>> enabled. >>> >>> - Robert >>> >>> >>> >>> >>> On Wed, Oct 2, 2013 at 10:49 AM, Serega Sheypak < >>> [email protected]>wrote: >>> >>> > Hi Robert, thanks for reply. >>> > >>> > Cloudera distirbution doesn't have property oozie.services >>> > 1. I've tried to set all props for purge service with purge period = >>> 100 >>> > (seconds???) Nothing happened for half an hour >>> > >>> > 2. I've tired to explicitly declare PurgeService and got NPE. >>> > >>> > I don't understand how it works at all... I don't see expected >>> behaviour. >>> > >>> > >>> > 2013/10/2 Robert Kanter <[email protected]> >>> > >>> > > Hi Serega, >>> > > >>> > > oozie-default.xml specifies a number of oozie services to load via >>> the >>> > > "oozie.services" property; but specifying >>> > > <property> <name>oozie.services</name> <value> >>> > > org.apache.oozie.service.PurgeService, >>> > > org.apache.oozie.service.SchedulerService </value> </property> >>> > > you've now removed all but 2 of them. You should remove that >>> property >>> > from >>> > > your oozie-site.xml. >>> > > >>> > > If you want to add additional services, you should use the >>> > > "oozie.services.ext" property instead. That said, the PurgeService >>> > should >>> > > already be listed under "oozie.services" so it should be running by >>> > > default. >>> > > >>> > > - Robert >>> > > >>> > > >>> > > On Wed, Oct 2, 2013 at 2:06 AM, Serega Sheypak < >>> [email protected] >>> > > >wrote: >>> > > >>> > > > Hi, we are using Oozie vervsion Oozie BUILD_VERSION >>> [3.3.2-cdh4.3.0] >>> > > > compiled by [jenkins] on [2013.05.28-04:29:38GMT] >>> > > > >>> > > > I want to PurgeService work. >>> > > > I did set these props: >>> > > > <!-- Purging old materializations--> <!-- Keep them for three >>> months >>> > --> >>> > > > <property> <name>oozie.service.PurgeService.older.than</name> >>> > > > <value>90</value> </property> <property> >>> > > > <name>oozie.service.PurgeService.coord.older.than</name> >>> > > <value>90</value> >>> > > > </property> <property> >>> > > <name>oozie.service.PurgeService.purge.limit</name> >>> > > > <value>100</value> </property> <property> >>> > > > <name>oozie.service.PurgeService.purge.interval</name> >>> > <value>100</value> >>> > > > </property> >>> > > > >>> > > > >>> > > > The idea is: keep actions data for 90 dayes or not more than last >>> 100 >>> > > > items. >>> > > > oozie.service.PurgeService.purge.interval is set to 100 seconds. >>> > > > Nothing happened after 100 seconds. >>> > > > >>> > > > I've tried to add this section: >>> > > > <property> <name>oozie.services</name> <value> >>> > > > org.apache.oozie.service.PurgeService, >>> > > > org.apache.oozie.service.SchedulerService </value> </property> >>> > > > >>> > > > Now Ooize is not starting: >>> > > > >>> > > > >>> > > > org.apache.oozie.service.ServiceException: E0103: Could not load >>> > > > service classes, null >>> > > > at >>> > > > org.apache.oozie.service.Services.loadServices(Services.java:291) >>> > > > at >>> org.apache.oozie.service.Services.init(Services.java:208) >>> > > > >>> > > > at >>> > > > >>> > > >>> > >>> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:39) >>> > > > at >>> > > > >>> > > >>> > >>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206) >>> > > > at >>> > > > >>> > >>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4705) >>> > > > >>> > > > at >>> > > > >>> > > >>> > >>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799) >>> > > > at >>> > > > >>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779) >>> > > > at >>> > > > >>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601) >>> > > > >>> > > > at >>> > > > >>> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943) >>> > > > at >>> > > > >>> org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778) >>> > > > at >>> > > > >>> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504) >>> > > > >>> > > > at >>> > > > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317) >>> > > > at >>> > > > >>> > > >>> > >>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324) >>> > > > at >>> > > > >>> > > >>> > >>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) >>> > > > >>> > > > at >>> > > > >>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065) >>> > > > at >>> > > > org.apache.catalina.core.StandardHost.start(StandardHost.java:840) >>> > > > at >>> > > > >>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057) >>> > > > >>> > > > at >>> > > > >>> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) >>> > > > at >>> > > > >>> > >>> org.apache.catalina.core.StandardService.start(StandardService.java:525) >>> > > > at >>> > > > >>> org.apache.catalina.core.StandardServer.start(StandardServer.java:754) >>> > > > >>> > > > at >>> > org.apache.catalina.startup.Catalina.start(Catalina.java:595) >>> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>> Method) >>> > > > at >>> > > > >>> > > >>> > >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> > > > >>> > > > at >>> > > > >>> > > >>> > >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> > > > at java.lang.reflect.Method.invoke(Method.java:597) >>> > > > at >>> > > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) >>> > > > at >>> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) >>> > > > >>> > > > Caused by: java.lang.NullPointerException >>> > > > at >>> > > org.apache.oozie.service.PurgeService.init(PurgeService.java:76) >>> > > > at >>> > > > >>> org.apache.oozie.service.Services.setServiceInternal(Services.java:368) >>> > > > at >>> > > org.apache.oozie.service.Services.setService(Services.java:354) >>> > > > >>> > > > at >>> > > > org.apache.oozie.service.Services.loadServices(Services.java:287) >>> > > > ... 26 more >>> > > > >>> > > > >>> > > > What do I do wrong? >>> > > > >>> > > >>> > >>> >> >> >
