Thomas Weise created APEXCORE-418:
-------------------------------------
Summary: Support for Mesos
Key: APEXCORE-418
URL: https://issues.apache.org/jira/browse/APEXCORE-418
Project: Apache Apex Core
Issue Type: New Feature
Reporter: Thomas Weise
Today Apex has two modes of execution: Embedded mode (everything running in a
single JVM) and YARN. There has been a few questions around native support for
Mesos. A cursory look suggests that Mesos support can be added by
reimplementing the YARN specific portions in the master (AppMasterService,
ContainerLauncher) and limited changes to the streaming container driver.
Mesos has a different model of resource allocation: The master offers resources
to the framework while in YARN resources are requested. Apex master needs to
implement the "framework scheduler" that is responsible to accept the resources
and control the tasks.
http://mesos.apache.org/documentation/latest/app-framework-development-guide/
Tasks are launched through executors, command line and docker executors are
provided.
Apex also requires support to deploy the dependencies to the nodes on which the
streaming containers are launched. YARN supports that through resource
localization. Mesos supports this through the fetcher, which can copy the
resources to the slave node.
http://mesos.apache.org/documentation/latest/fetcher/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
