Hi Chris, There isn¹t a way of dynamic parallel tasks within the same Oozie workflow XML currently. But you can do some programmatically. Using Oozie Java API, you can start a dynamic number of sub-workflows based on the number of outputs.
On 2/20/14, 7:05 AM, "Heller, Chris" <[email protected]> wrote: >Hi, > >I¹m trying to figure out the best way to implement a workflow in Oozie. > >I am creating a workflow which splits an input into multiple outputs. > >Then for each output I want to run another process over each. > >The trouble is I cannot know a-priori how many outputs I will have, and >so to post process each I don¹t see how to setup a workflow to run the >next stage. > >Ideally the next stage would be a fork/join type of scenario, since each >output can be processed independently. But there isn¹t any way I can see >to setup the fork paths without using some sort of XML generation >preprocessor. > >Does anyone have a suggestion of how to proceed? Am I stuck doing >workflow generation? Or is there another way to structure this workflow >using the existing primitives? > >Thanks, >Chris
