yup that worked Thanks for the clarification!
On Fri, Feb 10, 2017 at 9:42 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > If you place core-site.xml in $SPARK_HOME/conf, I'm pretty sure Spark > will pick it up. (Sounds like you're not running YARN, which would > require HADOOP_CONF_DIR.) > > Also this is more of a user@ question. > > On Fri, Feb 10, 2017 at 1:35 PM, Sam Elamin <hussam.ela...@gmail.com> > wrote: > > Hi All, > > > > > > really newbie question here folks, i have properties like my aws access > and > > secret keys in the core-site.xml in hadoop among other properties, but > thats > > the only reason I have hadoop installed which seems a bit of an overkill. > > > > Is there an equivalent of core-site.xml for spark so I dont have to > > reference the HADOOP_CONF_DIR in my spark env.sh? > > > > I know I can export env variables for the AWS credentials but other > > properties that my application might want to use? > > > > Regards > > Sam > > > > > > > > > > -- > Marcelo >
