Hi everyone! While we are trying to utilize Spark On Mesos cluster, we are facing an issue related to effective linux user id being used to start executors on Mesos slaves: all executors are trying to use driver's linux user id to start on Mesos slaves. Let me explain in detail: spark driver program (which is going to spawn Spark on Mesos in coarse mode) is started as unprivileged linux user, for example 'user1'. We have Spark distribution unpacked and ready-to-use on every mesos slave (placed at /opt/spark, 'spark.mesos.executor.home' is pointing to this folder). And after attempt to run every executor fails to start with error log telling user 'user1' is not available. And it is really true - there is no 'user1' present on Mesos slaves. So my question is: how can I control effective user id which will be used for start executors on Mesos? Actually I was trying to setup SPARK_USER=nobody on every slave but it wasn't useful. Thanks for advice if any.
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-executor-on-Mesos-how-to-set-effective-user-id-tp25118.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
