[
https://issues.apache.org/jira/browse/OODT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Starch updated OODT-700:
--------------------------------
Attachment: svn-workflow-spark-2014-11-05.txt
This patch is a working prototype. Resource manager will hold this
functionality but I will leave the patch here for anyone who comes looking.
> Create a Spark Runner for the Wengine
> -------------------------------------
>
> Key: OODT-700
> URL: https://issues.apache.org/jira/browse/OODT-700
> Project: OODT
> Issue Type: Sub-task
> Components: file manager, resource manager, workflow manager
> Reporter: Michael Starch
> Fix For: 0.8
>
> Attachments: svn-workflow-spark-2014-11-05.txt
>
>
> Spark is a useful technology for processing of streams and running map-reduce
> within the memory of a cluster. Thus it is important to fold it into the
> streaming augmentation of OODT.
> Currently, the wengine runner extension point seems to be the best place to
> put this. In order for the project to be fully backwards compatible, some
> processing point in OODT must allow multiplexing of jobs (some are run on
> mesos via resource manager, some are run in Spark, etc.).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
