Thanks for putting a comprehensive observation about Spark on Kubernetes. In mesos Spark deployment, it has a property called spark.mesos.extra.cores. The property means: * Set the extra number of cores for an executor to advertise. This does not result in more cores allocated. It instead means that an executor will "pretend" it has more cores, so that the driver will send it more tasks. Use this to increase parallelism. This setting is only used for Mesos coarse-grained mode*
Can this be used to increase parallelism? Are there other better ways to increase parallelism in Kubernetes? -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
