On 29 Mar 2016, at 14:30, Fei Hu <[email protected]<mailto:[email protected]>> wrote:
Hi Jeff, Thanks for your info! I am developing a workflow system based on Oozie, but it only supports java and mapreduce now, so I want to run spark job as in local mode by the workflow system first, then extend the workflow system to run spark job on Yarn. You can run spark code on oozie today; I'm just finishing up documenting how to do it on a secure cluster https://github.com/steveloughran/spark/blob/stevel/feature/SPARK-13148-oozie/docs/running-on-yarn.md On that topic: do start testing early if you are planning to run against a secure cluster, otherwise you don't find the problems until it's (nearly) too late to fix
