Hi, I’m trying to figure out the best way to implement a workflow in Oozie.
I am creating a workflow which splits an input into multiple outputs. Then for each output I want to run another process over each. The trouble is I cannot know a-priori how many outputs I will have, and so to post process each I don’t see how to setup a workflow to run the next stage. Ideally the next stage would be a fork/join type of scenario, since each output can be processed independently. But there isn’t any way I can see to setup the fork paths without using some sort of XML generation preprocessor. Does anyone have a suggestion of how to proceed? Am I stuck doing workflow generation? Or is there another way to structure this workflow using the existing primitives? Thanks, Chris
