Hi, The integration Arun pointed to is for S4 0.5 and YARN 2.0.2-alpha. This contribution has not yet been ported to S4 0.6 or new version of YARN.
Changes in S4 were mainly: - adding an Application Master specific for S4, - implementing a mechanism to fetch application code in S4 node from HDFS. Also note that by default application deployment in S4 is dynamic, i.e. the YARN integration is only for starting an S4 cluster, app deployment is somehow independent (app code is fetched from a specified location). Packaging/deploying with BigTop sounds very appealing, it would provide the ability to easily integrate a stream processing capability to a hadoop stack. It's hard for me to volunteer at the moment for that, but I would be glad to help anyone willing to work on it. Matthieu On Jun 11, 2013, at 02:01 , Roman Shaposhnik <r...@apache.org> wrote: > On Mon, Jun 10, 2013 at 4:31 PM, Arun C Murthy <a...@hortonworks.com> wrote: >> https://cwiki.apache.org/S4/deploying-s4-applications-with-yarn.html is >> another option. >> >> I'll let Flavio and others comment on where it stands ATM. > > Great point! Didn't know it existed. This makes me want to see S4 in Bigtop > even more -- would be a great test for a more of a framework aspect of YARN. > > The best we've done so far is Giraph on YARN in Bigtop. > > Thanks, > Roman.
