Thanks for the reply. I also found that sparkConf.setExecutorEnv works for yarn.
From: Saisai Shao <[email protected]<mailto:[email protected]>> Date: Wednesday, February 17, 2016 at 6:02 PM To: Lin Zhao <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Yarn client mode: Setting environment variables IIUC for example you want to set environment FOO=bar in executor side, you could use "spark.executor.Env.FOO=bar" in conf file, AM will pick this configuration and set as environment variable through container launching. Just list all the envs you want to set in executor side like spark.executor.xxx=xxx. Thanks Saisai On Thu, Feb 18, 2016 at 3:31 AM, Lin Zhao <[email protected]<mailto:[email protected]>> wrote: I've been trying to set some environment variables for the spark executors but haven't had much like. I tried editting conf/spark-env.sh but it doesn't get through to the executors. I'm running 1.6.0 and yarn, any pointer is appreciated. Thanks, Lin
