Hi, In case this is helpful for you, this is the patch i submitted to Spark about Swift and Spark integration ( about to be merged ) https://github.com/apache/spark/pull/1010
I sent information about this patch to this mailing list about two months ago. All the best, Gil. From: Trevor McKay <[email protected]> To: OpenStack Development Mailing List <[email protected]> Date: 28/08/2014 06:22 PM Subject: [openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths Hi folks, I've updated this etherpad with notes from an investigation of Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra repo. Following the notes there, I was able to access swift:// paths from Spark jobs on a Spark standalone cluster launched from Sahara and then fixed up by hand. Comments welcome. This is a POC at this point imho, we have work to do to fully integrate this into Sahara. https://etherpad.openstack.org/p/sahara_spark_edp Best, Trevor _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
