I've just committed HADOOP-9845 to trunk (only trunk at the moment). To build trunk now you need protoc 2.5.0 (the build will fail with a warning if you don't have it).
We'd propagate this to the 2 branches once the precommit build is back to normal and see things are OK. Thanks. On Mon, Aug 12, 2013 at 2:57 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > About to commit HADOOP-9845 to trunk, in 5 mins. This will make trunk use > protoc 2.5.0. > > thx > > > On Mon, Aug 12, 2013 at 11:47 AM, Giridharan Kesavan < > gkesa...@hortonworks.com> wrote: > >> I can take care of re-installing 2.4 and installing 2.5 in a different >> location. This would fix 2.0 branch builds as well. >> Thoughts? >> >> -Giri >> >> >> On Mon, Aug 12, 2013 at 11:37 AM, Alejandro Abdelnur <t...@cloudera.com >> >wrote: >> >> > Giri, >> > >> > first of all, thanks for installing protoc 2.5.0. >> > >> > I didn't know we were installing them as the only version and not >> driven by >> > env/path settings. >> > >> > Now we have a bit of a problem, precommit builds are broken because of >> > mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1). >> > >> > We have to options: >> > >> > 1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out any >> > follow up issues. >> > 2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and >> 2.5.0 >> > coexisting >> > >> > My take would be to commit HADOOP-9845 in trunk, iron out any issues an >> > then merge it to the other branches. >> > >> > We need to sort this out quickly as precommits are not working. >> > >> > I'll wait till 3PM today for objections to option #1, if none I'll >> commit >> > it to trunk. >> > >> > Thanks. >> > >> > Alejandro >> > >> > >> > >> > On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan < >> > gkesa...@hortonworks.com> wrote: >> > >> > > Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the default >> > path. >> > > If we still need 2.4 I may have to install it. Let me know >> > > >> > > -Giri >> > > >> > > >> > > On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur < >> t...@cloudera.com >> > > >wrote: >> > > >> > > > thanks giri, how do we set 2.4 or 2.5., what is the path to both so >> we >> > > can >> > > > use and env to set it in the jobs? >> > > > >> > > > thx >> > > > >> > > > Alejandro >> > > > (phone typing) >> > > > >> > > > On Aug 9, 2013, at 23:10, Giridharan Kesavan < >> gkesa...@hortonworks.com >> > > >> > > > wrote: >> > > > >> > > > > build slaves hadoop1-hadoop9 now has libprotoc 2.5.0 >> > > > > >> > > > > >> > > > > >> > > > > -Giri >> > > > > >> > > > > >> > > > > On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan < >> > > > > gkesa...@hortonworks.com> wrote: >> > > > > >> > > > >> Alejandro, >> > > > >> >> > > > >> I'm upgrading protobuf on slaves hadoop1-hadoop9. >> > > > >> >> > > > >> -Giri >> > > > >> >> > > > >> >> > > > >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur < >> > t...@cloudera.com >> > > > >wrote: >> > > > >> >> > > > >>> pinging again, I need help from somebody with sudo access to the >> > > hadoop >> > > > >>> jenkins boxes to do this or to get sudo access for a couple of >> > hours >> > > to >> > > > >>> set >> > > > >>> up myself. >> > > > >>> >> > > > >>> Please!!! >> > > > >>> >> > > > >>> thx >> > > > >>> >> > > > >>> >> > > > >>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur < >> > > t...@cloudera.com >> > > > >>>> wrote: >> > > > >>> >> > > > >>>> To move forward with this we need protoc 2.5.0 in the apache >> > hadoop >> > > > >>>> jenkins boxes. >> > > > >>>> >> > > > >>>> Who can help with this? I assume somebody at Y!, right? >> > > > >>>> >> > > > >>>> Thx >> > > > >>>> >> > > > >>>> >> > > > >>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark < >> ecl...@apache.org> >> > > > >>> wrote: >> > > > >>>> >> > > > >>>>> In HBase land we've pretty well discovered that we'll need to >> > have >> > > > the >> > > > >>>>> same version of protobuf that the HDFS/Yarn/MR servers are >> > running. >> > > > >>>>> That is to say there are issues with ever having 2.4.x and >> 2.5.x >> > on >> > > > >>>>> the same class path. >> > > > >>>>> >> > > > >>>>> Upgrading to 2.5.x would be great, as it brings some new >> classes >> > we >> > > > >>>>> could use. With that said HBase is getting pretty close to a >> > > rather >> > > > >>>>> large release (0.96.0 aka The Singularity) so getting this in >> > > sooner >> > > > >>>>> rather than later would be great. If we could get this into >> > 2.1.0 >> > > it >> > > > >>>>> would be great as that would allow us to have a pretty easy >> story >> > > to >> > > > >>>>> users with regards to protobuf version. >> > > > >>>>> >> > > > >>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee < >> kih...@yahoo-inc.com >> > > >> > > > >>> wrote: >> > > > >>>>>> Sorry to hijack the thread but, I also wanted to mention >> Avro. >> > See >> > > > >>>>> HADOOP-9672. >> > > > >>>>>> The version we are using has memory leak and inefficiency >> > issues. >> > > > >>> We've >> > > > >>>>> seen users running into it. >> > > > >>>>>> >> > > > >>>>>> Kihwal >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> ________________________________ >> > > > >>>>>> From: Tsuyoshi OZAWA <ozawa.tsuyo...@gmail.com> >> > > > >>>>>> To: "common-dev@hadoop.apache.org" < >> > common-dev@hadoop.apache.org> >> > > > >>>>>> Cc: "hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org >> >; >> > " >> > > > >>>>> yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org>; " >> > > > >>>>> mapreduce-...@hadoop.apache.org" < >> > mapreduce-...@hadoop.apache.org> >> > > > >>>>>> Sent: Thursday, August 8, 2013 1:59 AM >> > > > >>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, >> > > > >>>>> HADOOP-9845 >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> Hi, >> > > > >>>>>> >> > > > >>>>>> About Hadoop, Harsh is dealing with this problem in >> HADOOP-9346. >> > > > >>>>>> For more detail, please see the JIRA ticket: >> > > > >>>>>> https://issues.apache.org/jira/browse/HADOOP-9346 >> > > > >>>>>> >> > > > >>>>>> - Tsuyoshi >> > > > >>>>>> >> > > > >>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur < >> > > > >>> t...@cloudera.com> >> > > > >>>>> wrote: >> > > > >>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release. >> > > > >>>>>>> >> > > > >>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant >> > > benefits >> > > > >>> to >> > > > >>>>>>> justify the upgrade. >> > > > >>>>>>> >> > > > >>>>>>> Doing the upgrade now, with the first beta, will make things >> > > easier >> > > > >>> for >> > > > >>>>>>> downstream projects (like HBase) using protobuf and adopting >> > > Hadoop >> > > > >>> 2. >> > > > >>>>> If >> > > > >>>>>>> we do the upgrade later, downstream projects will have to >> > > support 2 >> > > > >>>>>>> different versions and they my get in nasty waters due to >> > > classpath >> > > > >>>>> issues. >> > > > >>>>>>> >> > > > >>>>>>> I've locally tested the patch in a pseudo deployment of >> > > 2.1.0-beta >> > > > >>>>> branch >> > > > >>>>>>> and it works fine (something is broken in trunk in the RPC >> > layer >> > > > >>>>> YARN-885). >> > > > >>>>>>> >> > > > >>>>>>> Now, to do this it will require a few things: >> > > > >>>>>>> >> > > > >>>>>>> * Make sure protobuf 2.5.0 is available in the jenkins box >> > > > >>>>>>> * A follow up email to dev@ aliases indicating developers >> > should >> > > > >>>>> install >> > > > >>>>>>> locally protobuf 2.5.0 >> > > > >>>>>>> >> > > > >>>>>>> Thanks. >> > > > >>>>>>> >> > > > >>>>>>> -- >> > > > >>>>>>> Alejandro >> > > > >>>> >> > > > >>>> >> > > > >>>> >> > > > >>>> -- >> > > > >>>> Alejandro >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> -- >> > > > >>> Alejandro >> > > > >> >> > > > >> >> > > > >> > > >> > >> > >> > >> > -- >> > Alejandro >> > >> > > > > -- > Alejandro > -- Alejandro
