Hello Everybody, When a coordinator is waiting for past datasets to appear we have the THROTTLE parameter to restrict the number of jobs being spawned. This works well if the upload to hdfs failed for a couple days but I would like to throttle READY jobs. I.e. I have a coordinator with high frequency ( 100 jobs a day ) and oozie is down for a weekend. What happened then was that he saw all the folders in HDFS ( 300 ) and created jobs for ALL of them putting them into READY state. I get that this is not very heavy since he doesn’t need to check HDFS anymore but it exploded the nproc ulimit of my oozie server. The oozie server has 1031 threads and the limit is 1024. We can increase the nproc limits but this is a bit unclean.
I do not get why throttle does not apply to READY jobs and would like to know if there is a parameter that does? Any help would be great! Ben http://oozie.apache.org/docs/3.3.2/CoordinatorFunctionalSpec.html#a6.1.6._Coordinator_Action_Execution_Policies
