[
https://issues.apache.org/jira/browse/SAMZA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150269#comment-14150269
]
Timothy Chen commented on SAMZA-375:
------------------------------------
[~jonbringhurst][~criccomini] thanks for the explanations, this is really
useful!
So a framework in Mesos is usually not scheduled to be placed somewhere in the
cluster by Mesos, but started directly as a single instance to schedule tasks
based on resource offers. The tasks once scheduled can either run by Mesos with
specified command or frameworks like Samza can implement a custom executor that
is responsible for starting a task, reporting the task status (started,
finished/failed) and also the framework can receive messages from the executor,
that also can contain data.
I think actually it's ok launching a framework per job as long as it all using
the same role, it is Samza's responsibilty to launch the framework itself
though, and also Samza now doesn't have the full view of how much resources it
totally has and each job is independently being resource scheduled by Mesos,
and then the number of parallel frameworks that can get resource allocated is
the number of slaves available as well.
Launching just one framework then Samza can be more intelligent in the placing
of the resources itself, and can launch multiple tasks that uses portions of it
for the entireity of the job, but does become a more monolithic type resource
controller.
Just some preliminery thoughts, I think regardless we can continue down the
path we have and have a functional prototype to try out with.
> Investigate Mesos Job Support
> -----------------------------
>
> Key: SAMZA-375
> URL: https://issues.apache.org/jira/browse/SAMZA-375
> Project: Samza
> Issue Type: Bug
> Components: hello-samza
> Reporter: Jon Bringhurst
> Assignee: Jon Bringhurst
> Labels: mesos, project
> Attachments: Screen Shot 2014-08-23 at 5.51.39 PM.png, Screen Shot
> 2014-09-22 at 8.59.12 AM.png
>
>
> It would be nice if Samza had support for Mesos (https://mesos.apache.org/).
> The current plan is to create a MesosJob and MesosJobFactory, then look into
> what it would take to allow the AM code to act as a Mesos scheduler.
> The feasibility of this landing in trunk will be better understood after a
> rough prototype has been created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
