Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/5063#issuecomment-84272183
@sryza I believe this setting is a bit unique for Mesos. I think it's
better to clarify the terminology here, of what does framework and executor
means for Mesos.
In mesos a scheduler is the process which registers with Master to receives
offers, and use these offers to launch tasks through the master and eventually
the slave. An executor is responsible for actually launching a task and
reporting task statuses that is launched on the slaves, which either is a
custom executor that fine-grain mode uses or the default mesos executor that
coarse-grain mode uses. And a framework is referring to the overall scheduler +
executor.
So in the Mesos fine-grain case, we have an custom executor that is only
launched once per slave, and that running executor takes up some cpu/mem itself.
IMO I think calling it spark.mesos.executor.cores is fine enough, I
wouldn't put framework in there as framework is not the right terminology.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
