Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/731#issuecomment-85797557
Also, while I was reviewing this I had more thoughts on the appropriate
config to use. Given the semantics that we want to support, I actually think it
may not be correct to reuse `spark.executor.cores`. The reason is that here all
we guarantee is that the executor gets *at most*, not exactly,
`spark.executor.cores`. This is inconsistent with the semantics of this config
in YARN.
Instead, maybe it makes sense to configure this in a new config:
`spark.deploy.maxCoresPerExecutor`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
