Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths
Hi Michael, I have an update to this patch with temp auth authentication also, but it's not yet submitted. I am not aware of v3 support. All the best, Gil. From: Michael McCune To: "OpenStack Development Mailing List (not for usage questions)" Date: 28/08/2014 08:14 PM Subject: Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths hi Gil, that's cool about the patch to Spark, has there been any talk about upgrading that patch to include Keystone v3 operations? - Original Message - > Hi, > > In case this is helpful for you, this is the patch i submitted to Spark > about Swift and Spark integration ( about to be merged ) > https://github.com/apache/spark/pull/1010 > > I sent information about this patch to this mailing list about two months > ago. > > All the best, > Gil. > > > > > > From: Trevor McKay > To: OpenStack Development Mailing List > > Date: 28/08/2014 06:22 PM > Subject: [openstack-dev] [sahara] Notes on developing Sahara Spark > EDP to work with swift:// paths > > > > Hi folks, > > I've updated this etherpad with notes from an investigation of > Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra > repo. > > Following the notes there, I was able to access swift:// paths from > Spark jobs on a Spark standalone cluster launched from Sahara and then > fixed up by hand. > > Comments welcome. This is a POC at this point imho, we have work to > do to fully integrate this into Sahara. > > https://etherpad.openstack.org/p/sahara_spark_edp > > Best, > > Trevor > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths
hi Gil, that's cool about the patch to Spark, has there been any talk about upgrading that patch to include Keystone v3 operations? - Original Message - > Hi, > > In case this is helpful for you, this is the patch i submitted to Spark > about Swift and Spark integration ( about to be merged ) > https://github.com/apache/spark/pull/1010 > > I sent information about this patch to this mailing list about two months > ago. > > All the best, > Gil. > > > > > > From: Trevor McKay > To: OpenStack Development Mailing List > > Date: 28/08/2014 06:22 PM > Subject: [openstack-dev] [sahara] Notes on developing Sahara Spark > EDP to work with swift:// paths > > > > Hi folks, > > I've updated this etherpad with notes from an investigation of > Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra > repo. > > Following the notes there, I was able to access swift:// paths from > Spark jobs on a Spark standalone cluster launched from Sahara and then > fixed up by hand. > > Comments welcome. This is a POC at this point imho, we have work to > do to fully integrate this into Sahara. > > https://etherpad.openstack.org/p/sahara_spark_edp > > Best, > > Trevor > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths
Gil, thanks! I'll take a look. Trevor On Thu, 2014-08-28 at 19:31 +0300, Gil Vernik wrote: > Hi, > > In case this is helpful for you, this is the patch i submitted to > Spark about Swift and Spark integration ( about to be merged ) > https://github.com/apache/spark/pull/1010 > > I sent information about this patch to this mailing list about two > months ago. > > All the best, > Gil. > > > > > > From:Trevor McKay > To:OpenStack Development Mailing List > > Date: 28/08/2014 06:22 PM > Subject: [openstack-dev] [sahara] Notes on developing Sahara > Spark EDP to work with swift:// paths > > __ > > > > Hi folks, > > I've updated this etherpad with notes from an investigation of > Spark/Swift and the hadoop-openstack plugin carried in the > sahara-extra > repo. > > Following the notes there, I was able to access swift:// paths from > Spark jobs on a Spark standalone cluster launched from Sahara and then > fixed up by hand. > > Comments welcome. This is a POC at this point imho, we have work to > do to fully integrate this into Sahara. > > https://etherpad.openstack.org/p/sahara_spark_edp > > Best, > > Trevor > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths
Hi, In case this is helpful for you, this is the patch i submitted to Spark about Swift and Spark integration ( about to be merged ) https://github.com/apache/spark/pull/1010 I sent information about this patch to this mailing list about two months ago. All the best, Gil. From: Trevor McKay To: OpenStack Development Mailing List Date: 28/08/2014 06:22 PM Subject:[openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths Hi folks, I've updated this etherpad with notes from an investigation of Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra repo. Following the notes there, I was able to access swift:// paths from Spark jobs on a Spark standalone cluster launched from Sahara and then fixed up by hand. Comments welcome. This is a POC at this point imho, we have work to do to fully integrate this into Sahara. https://etherpad.openstack.org/p/sahara_spark_edp Best, Trevor ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths
Hi folks, I've updated this etherpad with notes from an investigation of Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra repo. Following the notes there, I was able to access swift:// paths from Spark jobs on a Spark standalone cluster launched from Sahara and then fixed up by hand. Comments welcome. This is a POC at this point imho, we have work to do to fully integrate this into Sahara. https://etherpad.openstack.org/p/sahara_spark_edp Best, Trevor ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
