Yup, this should be in Spark 0.9 and 0.8.1. Matei
On Dec 13, 2013, at 9:41 AM, Koert Kuipers <[email protected]> wrote: > thats great. didn't realize this was in master already. > > > On Thu, Dec 12, 2013 at 8:10 PM, Shao, Saisai <[email protected]> wrote: > Hi Koert, > > > > Spark with multi-user support has been merged in master branch with patch > (https://github.com/apache/incubator-spark/pull/23), you can check out the > master branch. These patch can support to access hdfs with the username you > start the Spark application, not the one who starts Spark service. > > > > Thanks > > Jerry > > From: Koert Kuipers [mailto:[email protected]] > Sent: Friday, December 13, 2013 8:39 AM > To: [email protected] > Subject: Re: writing to HDFS with a given username > > > > Hey Philip, > how do you get spark to write to hdfs with your user name? When i use spark > it writes to hdfs as the user that runs the spark services... i wish it read > and wrote as me. > > > > On Thu, Dec 12, 2013 at 6:37 PM, Philip Ogren <[email protected]> wrote: > > When I call rdd.saveAsTextFile("hdfs://...") it uses my username to write to > the HDFS drive. If I try to write to an HDFS directory that I do not have > permissions to, then I get an error like this: > > Permission denied: user=me, access=WRITE, inode="/user/you/":you:us:drwxr-xr-x > > I can obviously get around this by changing the permissions on the directory > /user/you. However, is it possible to call rdd.saveAsText with an alternate > username and password? > > Thanks, > Philip > > > >
