Hi, It works like a charm.
In my case, I changed oozie.service.CoordMaterializeTriggerService.lookup.interval to 1 minute and oozie.service.CoordMaterializeTriggerService.materialization.window to 3 hours. Thanks again, Nam On Wed, Aug 21, 2013 at 10:32 PM, Papa's Gmail <[email protected]> wrote: > Thank you Paul, much appreciated. I will try this and let you know the > result. > > Bests, > Nam > > On Aug 21, 2013, at 10:06 PM, Paul Chavez <[email protected]> > wrote: > > > Hi Nam, > > > > I had the exact same issue a few weeks ago and asked this. Here is the > resoponse I got from Mona Chitnis: > > > > === > > What you are looking for here is > > "oozie.service.CoordMaterializeTriggerService.lookup.interval". Default > value = 300 sec or 5 min. This server-level setting governs how often the > materialization service is triggered to materialize the next set of actions. > > > > This service frequency works in conjunction with another setting - > "oozie.service.CoordMaterializeTriggerService.materialization.window". > > Default value = 3600 sec or 1 hour. What this means is every 5 min, > Oozie will materialize actions falling in the next 1 hour window. > > > > For your use case, if you wish to materialize actions a) faster than > every > > 5 min and b) spanning beyond 1 hour window at a time, you should > consider changing these 2 values. > > === > > > > In my case, all I had to do was update the first value > (...lookup.interval) and set it to 60s. > > > > Hope that helps, > > Paul Chavez > > > > > > -----Original Message----- > > From: Nam Pham [mailto:[email protected]] > > Sent: Wednesday, August 21, 2013 1:53 AM > > To: [email protected] > > Subject: Job creation rate > > > > Hello, > > > > I am new to this mailing list and would like to say hi to everybody. > > > > I am using oozie 3.3.2-chd4.3.0 to import about a year of data to hive. > > Data are stored hourly, so the coordinator need to create about 8000 > jobs to import all the data. > > > > The problem is that oozie coordinator is too slow in creating jobs. It > only creates about 300 jobs per day with my setup. I tried to increase the > concurrency parameter of the coordinator job but it does not help. > > > > Is there a knob anywhere making the coordinator create jobs faster, > preferably multiple jobs at once since those can be executed in parallel ? > > > > Ideas and suggestions are much appreciated. > > > > Regards, > > Nam >
