Thank you for the feedback. Are you running Yarn as a part of this setup? Seems like that would collide with Mesos. We have a job or 2 written in MRV1 that from what I understand can still be run with 2.2 somehow... On Mar 25, 2014 8:49 PM, "Bernardo Gomez Palacio" < [email protected]> wrote:
> Hi Gary. > I am currently using > Mesos 0.16.0 build for Protobufs 2.5.0 > (https://github.com/Guavus/mesos/tree/releases/0.16.0-HDP2x) > Hadoop version 2.2 > Spark 0.9.0 ( > https://github.com/guavus/incubator-spark/tree/guavus/branch-0.9-hdp2.2) > > Things to know. > Hadoop 1.0.4 uses protobufs 2.4.1 > Hadoop 2.2 and 2.3 uses protobufs 2.5.0 > Mesos < 0.17.0 uses protobufs 2.4.1 > Mesos >= 0.17.0 uses protobufs 2.5.0 > > If you need Mesos 0.15.0 on protobufs 2.5.0 you could try to backport > https://github.com/Guavus/mesos/tree/releases/0.16.0-HDP2x but if you > keep CDH4 with protobufs 2.4.1 you will run into trouble. > > Let me know if this help. > Bernardo. > > > > > On Tue, Mar 25, 2014 at 4:57 PM, Gary Malouf <[email protected]> > wrote: > > For various reasons, our team needs to keep all of our projects on the > same > > protobuf version. We've now hit a point where we need to upgrade > protobuf > > from 2.4.1 to 2.5.0 across the board in our projects and dependent > > platforms. > > > > *Current stack*: Mesos 0.15, Chronos, CDH 4.2.1-MRV1, Spark > > 0.9-pre-scala-2.10 build off master > > > > *Ideal stack after upgrade*: Mesos 0.17, Chronos, CDH5 beta2, Spark 0.9.1 > > (hadoop 2.2 build) > > > > From what we understand, we need a dependency on Hadoop 2.2 to get the > > necessary protobuf upgrade. From reading Cloudera's documentation and > > multiple google searches, it is not clear to me how we can construct the > > stack to continue to work. > > > > Has anyone else requested info on getting this combination to work? From > > others we've spoken to, they've basically said that we'll be forced to > use > > Yarn for Hadoop support in the very near future anyway so we should > switch. > > Since we colocate Spark with out HDFS nodes, it's hard to see how we > would > > run both Yarn and Mesos on the same servers. >
