Re: MiniOozie for local dryrun or other options for doing dryrun of oozie workflows?

2016-12-05 Thread Serega Sheypak
Yeah, I see it. I found the way to test wokflow locally but it's suuuper complex. I have to start local MR, Local HDFS and Local OOzie things. Then I do mock on the fly xml actions with my test actions and run workflow. It's super complex and fragile unfortunately... I'll try to reach dev group. I

Question about spark-bagel in oozie 4.3.0

2016-12-05 Thread Dongying Jiao
Hi: I noticed oozie 4.3.0 add spark-bagel lib to spark sharelib compared to oozie 4.2.0. I found this module is deprecated and superseded by GraphX in spark official site, why do we add this deprecated compenent since GraphX is already in sharelib? And I want to use spark 2.0 for oozie 4.3.0, as

Re: Change NN and JobTracker dynamically during runtime

2016-12-05 Thread mdk-swandha
You mean I have to set env variables for each job/workflow execution and then it will be picked up by Oozie. And I should set them in my service (the service which is finding the best cluster?). For example let say I have 3 cluster: - When a job is sent via Oozie/Hue/Zepellin/Livy etc. - they are

Re: Change NN and JobTracker dynamically during runtime

2016-12-05 Thread Andras Piros
Hi Dipesh, during workflow / job submission you can define variables inside job.properties coming e.g. from env vars that are used in workflow.xml. So much for the flexibility. Can you tell me a use case where runtime routing to different JT / NN instances via Oozie (and not e.g. coming from a

Re: MiniOozie for local dryrun or other options for doing dryrun of oozie workflows?

2016-12-05 Thread Andras Piros
Hi Serega, as per *Oozie documentation * we can see that with -dryrun option does not create nor run a job. So for the killer feature request, I think it's not possible ATM. Regards, Andras -- Andras PIROS