purge.executions= 11 in counters section, but nothing happens or I don't see i
2013/10/4 Serega Sheypak <[email protected]> > Hi, I've tried this one: > > <property> > <name>oozie.services.ext</name> > <value> > org.apache.oozie.service.PurgeService > </value> > </property> > > <!-- Purging old materializations--> > <!-- Keep them for three months --> > <property> > <name>oozie.service.PurgeService.older.than</name> > <value>90</value> > </property> > > <property> > <name>oozie.service.PurgeService.coord.older.than</name> > <value>90</value> > </property> > > <property> > <name>oozie.service.PurgeService.purge.limit</name> > <value>100</value> > </property> > > <property> > <name>oozie.service.PurgeService.purge.interval</name> > <value>100</value> > </property> > > <!-- We need it because of NPE in PurgeService. It needs all Purge > properties to be explicitly set--> > <property> > <name>oozie.service.PurgeService.bundle.older.than</name> > <value>14</value> > <description> > Completed bundle jobs older than this value, in days, will be > purged by the PurgeService. > </description> > </property> > > Oozie is started, cool, I forgot about oozie-default.xml. Thank you. > The problem is here I still see last 900+ coordinator materializations. > Nothing has been deleted. > 100 seconds (oozie.service.PurgeService.purge.interval) did pass. > > There is nothing related to PurgeSerivce in log. > > 1. What do I do wrong? > 2. oozie.service.PurgeService.purge.limit=100 and > oozie.service.PurgeService.older.than=90 (days) > What has more power? Or one of these conditions should trigger > PurgeService to cleanup oozie DB? > > > > 2013/10/2 Robert Kanter <[email protected]> > >> CDH does have that property. In both CDH and Apache distributions, that >> property (and some others) is specified in oozie-default.xml. There >> should >> be a copy of its somewhere. Oozie has its own internal copy of >> oozie-default.xml, so don't bother editing the one you find; Oozie loads >> its oozie-default.xml first and then loads oozie-site.xml with any >> properties you set (Hadoop works the same way with core-default.xml and >> core-site.xml). >> >> The NPE is because you only have the two services, PurgeService and >> SchedulerService, >> loaded because you overrode the oozie.services property. The NPE occurs >> because Oozie does need the ConfigurationService for sure to work >> properly; >> and without many of the other services, it will likely have other issues. >> >> You should remove your override of oozie.services from oozie-site.xml. >> Check oozie-default.xml and see if it lists PurgeService under >> oozie.services. If it does not, then you can add it in your >> oozie-site.xml >> in the oozie.services.ext property; if it does, then the PurgeService is >> enabled. >> >> - Robert >> >> >> >> >> On Wed, Oct 2, 2013 at 10:49 AM, Serega Sheypak <[email protected] >> >wrote: >> >> > Hi Robert, thanks for reply. >> > >> > Cloudera distirbution doesn't have property oozie.services >> > 1. I've tried to set all props for purge service with purge period = 100 >> > (seconds???) Nothing happened for half an hour >> > >> > 2. I've tired to explicitly declare PurgeService and got NPE. >> > >> > I don't understand how it works at all... I don't see expected >> behaviour. >> > >> > >> > 2013/10/2 Robert Kanter <[email protected]> >> > >> > > Hi Serega, >> > > >> > > oozie-default.xml specifies a number of oozie services to load via the >> > > "oozie.services" property; but specifying >> > > <property> <name>oozie.services</name> <value> >> > > org.apache.oozie.service.PurgeService, >> > > org.apache.oozie.service.SchedulerService </value> </property> >> > > you've now removed all but 2 of them. You should remove that property >> > from >> > > your oozie-site.xml. >> > > >> > > If you want to add additional services, you should use the >> > > "oozie.services.ext" property instead. That said, the PurgeService >> > should >> > > already be listed under "oozie.services" so it should be running by >> > > default. >> > > >> > > - Robert >> > > >> > > >> > > On Wed, Oct 2, 2013 at 2:06 AM, Serega Sheypak < >> [email protected] >> > > >wrote: >> > > >> > > > Hi, we are using Oozie vervsion Oozie BUILD_VERSION >> [3.3.2-cdh4.3.0] >> > > > compiled by [jenkins] on [2013.05.28-04:29:38GMT] >> > > > >> > > > I want to PurgeService work. >> > > > I did set these props: >> > > > <!-- Purging old materializations--> <!-- Keep them for three months >> > --> >> > > > <property> <name>oozie.service.PurgeService.older.than</name> >> > > > <value>90</value> </property> <property> >> > > > <name>oozie.service.PurgeService.coord.older.than</name> >> > > <value>90</value> >> > > > </property> <property> >> > > <name>oozie.service.PurgeService.purge.limit</name> >> > > > <value>100</value> </property> <property> >> > > > <name>oozie.service.PurgeService.purge.interval</name> >> > <value>100</value> >> > > > </property> >> > > > >> > > > >> > > > The idea is: keep actions data for 90 dayes or not more than last >> 100 >> > > > items. >> > > > oozie.service.PurgeService.purge.interval is set to 100 seconds. >> > > > Nothing happened after 100 seconds. >> > > > >> > > > I've tried to add this section: >> > > > <property> <name>oozie.services</name> <value> >> > > > org.apache.oozie.service.PurgeService, >> > > > org.apache.oozie.service.SchedulerService </value> </property> >> > > > >> > > > Now Ooize is not starting: >> > > > >> > > > >> > > > org.apache.oozie.service.ServiceException: E0103: Could not load >> > > > service classes, null >> > > > at >> > > > org.apache.oozie.service.Services.loadServices(Services.java:291) >> > > > at org.apache.oozie.service.Services.init(Services.java:208) >> > > > >> > > > at >> > > > >> > > >> > >> org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:39) >> > > > at >> > > > >> > > >> > >> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206) >> > > > at >> > > > >> > >> org.apache.catalina.core.StandardContext.start(StandardContext.java:4705) >> > > > >> > > > at >> > > > >> > > >> > >> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799) >> > > > at >> > > > >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779) >> > > > at >> > > > >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601) >> > > > >> > > > at >> > > > >> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943) >> > > > at >> > > > >> org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778) >> > > > at >> > > > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504) >> > > > >> > > > at >> > > > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317) >> > > > at >> > > > >> > > >> > >> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324) >> > > > at >> > > > >> > > >> > >> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) >> > > > >> > > > at >> > > > >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065) >> > > > at >> > > > org.apache.catalina.core.StandardHost.start(StandardHost.java:840) >> > > > at >> > > > >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057) >> > > > >> > > > at >> > > > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) >> > > > at >> > > > >> > org.apache.catalina.core.StandardService.start(StandardService.java:525) >> > > > at >> > > > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:754) >> > > > >> > > > at >> > org.apache.catalina.startup.Catalina.start(Catalina.java:595) >> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> > > > at >> > > > >> > > >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> > > > >> > > > at >> > > > >> > > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> > > > at java.lang.reflect.Method.invoke(Method.java:597) >> > > > at >> > > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) >> > > > at >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) >> > > > >> > > > Caused by: java.lang.NullPointerException >> > > > at >> > > org.apache.oozie.service.PurgeService.init(PurgeService.java:76) >> > > > at >> > > > >> org.apache.oozie.service.Services.setServiceInternal(Services.java:368) >> > > > at >> > > org.apache.oozie.service.Services.setService(Services.java:354) >> > > > >> > > > at >> > > > org.apache.oozie.service.Services.loadServices(Services.java:287) >> > > > ... 26 more >> > > > >> > > > >> > > > What do I do wrong? >> > > > >> > > >> > >> > >
