Let me rephrase what I said earlier as I just realized what the term 'truncate' probably came across as :)
All I do is run two simple delete statements that delete any data older than X amount of days (our purge is set to 7 days but I usually run this for 3 day intervals due to our job data being very temporal for us atm). This syntax will work for MySQL and I would assume this is all the internal code would be doing under the covers: DELETE FROM oozie.WF_ACTIONS WHERE wf_id in (SELECT id FROM oozie.WF_JOBS WHERE end_time < DATE_SUB(NOW(), INTERVAL 7 DAY)); DELETE FROM oozie.WF_JOBS WHERE end_time < DATE_SUB(NOW(), INTERVAL 7 DAY); Obviously as I said earlier this is just a stop gap but it will do what you need it to for now while you figure out what is going on with the service. Honestly as much as I love the web console I have found we can have much more selective filtering / we can actually do direct correlation between coordinator action ids and workflow runs by just doing our own querying of the oozie schema. -- Matt On Mon, Oct 8, 2012 at 1:38 PM, Eduardo Afonso Ferreira <[email protected]>wrote: > Matt, thanks for your input. > > > So far I did not have to get into the DB schema to manipulate the Oozie > data. > I'd rather not have to do that and avoid taking the risk of corrupting it. > Plain truncates won't work for me because I want to be able to see the job > history for a certain number of days, but I don't want it to keep growing. > The Oozie Web Console becomes slower and slower as the number of jobs > increase. There's a default limit of 1000 > (oozie.coord.actions.default.length) and I increased that to 5000. That > means, the web console will retrieve only that many when you request the > list of workflows associated with a coordinator. > With time, even the 5000 max will be exceeded and the UI becomes very slow > to load that, so I don't want to increase that limit more. I'd rather have > the PurgeService work as it's supposed to, i.e. do the purge when the > number of days defined by oozie.service.PurgeService.older.than is reached. > > Anybody else experiencing this problem? > Is it a problem in Oozie 3.2.0? > > Thank you. > Eduardo. > > > > ________________________________ > From: Matt Goeke <[email protected]> > To: [email protected] > Sent: Monday, October 8, 2012 8:39 AM > Subject: Re: PurgeService not working? > > We are using cdh3u3 and I haven't seen any issues with the service yet but > most of the time I end up truncating the tables manually. I realize that > this question might be more of a concern out of a bug / functionality > issues but if you are concerned about space then it is just as easy to do > this yourself. > > -- > Matt > > On Mon, Oct 8, 2012 at 1:48 AM, Eduardo Afonso Ferreira > <[email protected]>wrote: > > > Hey, apparently PurgeService is not doing it's job in my prod > environment. > > I have the following property defined in my oozie-site.xml: > > > > <property> > > <name>oozie.service.PurgeService.older.than</name> > > <value>7</value> > > <description> > > Jobs older than this value, in days, will be purged by the > > PurgeService. > > </description> > > </property> > > > > So, it should purge jobs older than 7 days, right? I don't see it > > happening. > > Is there anything else I should set to have it remove old stuff? > > > > Oozie was restarted after I added that property about 10 days ago. > > > > Thank you. > > Eduardo.
