Merge pull request #23 from jerryshao/multi-user Add Spark multi-user support for standalone mode and Mesos
This PR add multi-user support for Spark both standalone mode and Mesos (coarse and fine grained ) mode, user can specify the user name who submit app through environment variable `SPARK_USER` or use default one. Executor will communicate with Hadoop using specified user name. Also I fixed one bug in JobLogger when different user wrote job log to specified folder which has no right file permission. I separate previous [PR750](https://github.com/mesos/spark/pull/750) into two PRs, in this PR I only solve multi-user support problem. I will try to solve security auth problem in subsequent PR because security auth is a complicated problem especially for Shark Server like long-run app (both Kerberos TGT and HDFS delegation token should be renewed or re-created through app's run time). Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/be7e8da9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/be7e8da9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/be7e8da9 Branch: refs/heads/master Commit: be7e8da98ad04d66b61cd7fc8af7ae61a649d71c Parents: aadeda5 12dc385 Author: Reynold Xin <[email protected]> Authored: Wed Nov 6 23:22:47 2013 -0800 Committer: Reynold Xin <[email protected]> Committed: Wed Nov 6 23:22:47 2013 -0800 ---------------------------------------------------------------------- .../scala/org/apache/spark/SparkContext.scala | 10 + .../apache/spark/deploy/SparkHadoopUtil.scala | 18 +- .../org/apache/spark/executor/Executor.scala | 7 +- .../org/apache/spark/scheduler/JobLogger.scala | 757 ++++++++++--------- .../apache/spark/scheduler/JobLoggerSuite.scala | 4 +- 5 files changed, 417 insertions(+), 379 deletions(-) ----------------------------------------------------------------------
