[openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

2014-08-28 Thread Trevor McKay
Hi folks, I've updated this etherpad with notes from an investigation of Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra repo. Following the notes there, I was able to access swift:// paths from Spark jobs on a Spark standalone cluster launched from Sahara and then

Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

2014-08-28 Thread Gil Vernik
Hi, In case this is helpful for you, this is the patch i submitted to Spark about Swift and Spark integration ( about to be merged ) https://github.com/apache/spark/pull/1010 I sent information about this patch to this mailing list about two months ago. All the best, Gil. From: Trevor

Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

2014-08-28 Thread Trevor McKay
Gil, thanks! I'll take a look. Trevor On Thu, 2014-08-28 at 19:31 +0300, Gil Vernik wrote: Hi, In case this is helpful for you, this is the patch i submitted to Spark about Swift and Spark integration ( about to be merged ) https://github.com/apache/spark/pull/1010 I sent

Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

2014-08-28 Thread Michael McCune
hi Gil, that's cool about the patch to Spark, has there been any talk about upgrading that patch to include Keystone v3 operations? - Original Message - Hi, In case this is helpful for you, this is the patch i submitted to Spark about Swift and Spark integration ( about to be

Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

2014-08-28 Thread Gil Vernik
Hi Michael, I have an update to this patch with temp auth authentication also, but it's not yet submitted. I am not aware of v3 support. All the best, Gil. From: Michael McCune mimcc...@redhat.com To: OpenStack Development Mailing List (not for usage questions)