Yes. There also used to be something similar called Hadoop-on-Demand. But the idea is to schedule jobs to a persistent HDFS, sending jobs to where the data is, as opposed to setting up and tearing down HDFS for every job.
I probably should have given this as background: https://blogs.oracle.com/templedf/entry/beta_testing_the_sun_grid -----Original Message----- From: "Hung-Sheng Tsao (LaoTsao) Ph.D" <[email protected]> Date: Tue, 6 Mar 2012 12:12:06 -0500 To: Todd Heywood <[email protected]> Cc: "[email protected]" <[email protected]> Subject: Re: [gridengine users] Hadoop integration >did you see this blog? >https://blogs.oracle.com/ravee/entry/creating_hadoop_pe_under_sge > >Sent from my iPad > >On Mar 6, 2012, at 11:45, "Heywood, Todd" <[email protected]> wrote: > >> Way back when SGE was still at Sun, Dan Templeton wrote a SGE-Hadoop >>integration for 6.2u5 (Sun's distribution as a value-added feature). >> >> I have been told that because of changes have been made to the Hadoop >>API since Oracle purchased Sun this integration no longer works - at >>least not in the open source versions following 6.2u5. >> >> Does anyone know if this is true? Has anyone worked with this recently? >>I do see a hadoop.tar.gz at the SoGE site >>http://arc.liv.ac.uk/downloads/SGE/releases/8.0.0d<http://arc.liv.ac.uk/d >>ownloads/SGE/releases/8.0.0d/> but it looks to me like it is probably >>the 2-3 year old code from Sun (with no documentation since it was a >>value-added feature for Sun). >> >> Thanks, >> >> Todd Heywood >> >> >> _______________________________________________ >> users mailing list >> [email protected] >> https://gridengine.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
