Hi Peter, Thanks a lot for your answer, useful references to the JIRAs! I'll try to have a look at the code and see if this can be improved.
Out of curiosity, what is the process covered by 'validation of the XML'? I am asking because, when doing 'oozie validate' command, it is OK very quickly. Is there a way to "deactivate" this validation part? In my specific use-case, I could use one single fork/join, the thing is that if I take that route, I'd like to be able to limit the number of concurrent actions that can run in parallel from the fork. Is it something we can do? Thanks, Pierre. 2016-06-27 17:01 GMT+02:00 Peter Cseh <gezap...@cloudera.com>: > Hi Pierre, > > There was a bugfix around submitting fork jobs which parallelized job > submission: > https://issues.apache.org/jira/browse/OOZIE-2345 > > But the issue you've reported is known and not resolved yet: > https://issues.apache.org/jira/browse/OOZIE-1978 > > I could not find a workaround description, but one sub-workflow per fork > may help as the validation of the xml is the slow part. > Best regards, > Peter > > On Mon, Jun 27, 2016 at 4:22 PM, Pierre Villard < > pierre.villard...@gmail.com > > wrote: > > > Hi guys, > > > > I am trying to submit workflows with around 50 actions. However depending > > of how the workflow is defined and the number of actions, the time needed > > by Oozie to accept the workflow may change a lot (I am not talking about > > the execution time of actions, I’m really talking about the time needed > > between the moment I launch the command line 'job –run' and the moment I > > get back the prompt and my job ID). > > > > The submission time also seems to exponentially depend of the number of > > forks in the workflow (5 forks : few seconds, 6 forks : 1 minute, 7 > forks : > > 10 minutes, 8 forks : one hour). > > > > I was expecting to have workflows with a higher number of actions. Is it > a > > known issue? Is there some tuning to perform? are there workarounds? > should > > I use sub-workflows? > > > > Thanks for your help, > > Best regards, > > Pierre > > > > > > -- > Peter Cseh > Software Engineer > <http://www.cloudera.com> >
