Also, what about Hive 0.11? It's been released for a while now. It's also the first release of Hive that contains Hive Server2 (with support for concurrent queries). It also contains HCatalog as a part of Hive (since HCatalog graduated the incubator to become a part of Hive). I think this will be a great addition to Bigtop 0.7.
I understand, like Cos said, the focus of Bigtop 0.7 is to pay off our technical debt but I think Hive 0.11 is worth the investment in Bigtop 0.7 primarily because of all the goodness it brings in. If no one has any objections, can we add it to the BOM for 0.7 please? Thanks! Mark On Thu, Jul 11, 2013 at 11:24 AM, Sean Mackrory <[email protected]>wrote: > I would agree. As I've said I really don't have a strong opinion on > which one should be the "default" or whether we should do separate > releases, but I also don't object to any of the proposals so far. I'm > happy to lend a hand if needed, having already packaged these two > side-by-side successfully before, but it's not very hard from a > technical standpoint. > > > > On Wed, Jul 10, 2013 at 12:19 PM, Mark Grover <[email protected]> wrote: > > Thanks everyone for your feedback! > > > > I think we are moving towards a consensus where we should put sqoop2 as > the > > only sqoop in Bigtop 0.7 BOM. We, as a community, are very open to adding > > Sqoop1 back in Bigtop (0.6.1 or 0.7.0, whatever the decision is). We will > > just have to ensure (and it shouldn't be too hard to do so) that there > are > > no namespace/command name conflicts between sqoop1 and the sqoop already > > present in Bigtop (sqoop2). BIGTOP-1016 seems to be a good starting place > > for that. > > > > Do folks agree with the above? > > > > Mark > > > > On Wed, Jul 10, 2013 at 10:52 AM, Konstantin Boudnik <[email protected]> > wrote: > > > >> No, I meant stability of the framework itself: packaging, iTest, etc. > >> Perhaps > >> stability is too overloaded... robustness, perhaps? > >> > >> On Tue, Jul 09, 2013 at 10:57AM, Bruno Mahe wrote: > >> > Bigtop as a framework? You mean stable api of its projects? > >> > > >> > Sent from my HTC EVO 4G LTE exclusively from Sprint > >> > > >> > ----- Reply message ----- > >> > From: "Konstantin Boudnik" <[email protected]> > >> > To: <[email protected]> > >> > Cc: "Sean Mackrory" <[email protected]> > >> > Subject: [DISCUSS] BOM for release 0.7.0 of Bigtop > >> > Date: Tue, Jul 9, 2013 10:40 > >> > > >> > > >> > Bruno, > >> > > >> > just to clarify my stance of 'stability': it is more about stability > of > >> the > >> > Bigtop as a framework than a stability of the stack. > >> > > >> > I am not sure we have resources to do maintenance releases at this > >> point. May > >> > be it is just me. > >> > > >> > Cos > >> > > >> > On Tue, Jul 09, 2013 at 10:10AM, Bruno MahИ wrote: > >> > > On 07/09/2013 09:47 AM, Sean Mackrory wrote: > >> > >> Without wanting to detract from the spirit of focussing on system > >> > >> stability, I'd like to suggest a few changes I think it's time we > at > >> least > >> > >> discuss seriously: > >> > >> > >> > >> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to > add > >> > >> support for Oracle JDK 7? It's working pretty well in my > experience, > >> and > >> > >> although it's less tested upstream, the only JDK we officially > >> support is > >> > >> officially EOL, so we're not exactly in a good position now IMO. > >> > >> > >> > >> Debian 7 has also been out for a while, and I think we should do at > >> least > >> > >> one release on it. It's likely very little work but I think there's > >> value > >> > >> in certifying the stack will work well there. (On the topic of > OS's - > >> are > >> > >> we specifically talking SP3 of SLES 11?). I don't feel strongly on > >> this, > >> > >> but I'm just curious if there's a reason you're suggesting staying > >> with > >> > >> 12.10 and not 13.04 - other than wanting less change in this > release? > >> > >> Again, I hardly have an opinion on that one. > >> > >> > >> > >> Other components that have recently had releases that I don't > >> consider to > >> > >> impact the > >> > >> - Hue 2.4.0 > >> > >> - Whirr 0.8.2 > >> > >> - Flume 1.4.0 > >> > >> > >> > >> There's also been a ticket to package Avro for a long time and I'd > >> like to > >> > >> get to that soon. Perhaps Parquet as well? Although like Phoenix > and > >> > >> DataFu, I would suggest doing just the libraries for now, not all > the > >> CLI > >> > >> tools. > >> > >> > >> > >> Again - I don't mean to take away from the focus on stability, but > I > >> also > >> > >> don't think we shouldn't stretch to stay up to date either. > >> > >> > >> > >> +1 to everything else as suggested, however. > >> > >> > >> > >> > >> > >> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell < > [email protected]> > >> wrote: > >> > >>> Would you be willing to consider Phoenix, only BIGTOP-993? > >> Installing the > >> > >>> package produced by 993 only drops a library for HBase into > >> > >>> /usr/lib/phoenix, essentially the same relationship between the > >> DataFu > >> > >>> package and Pig. There is follow up work that is more ambitious, > for > >> > >>> example BIGTOP-1007, but that is not required by any means and > could > >> come > >> > >>> in if/whenever you are comfortable with it. > >> > >>> > >> > >>> > >> > >>> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik < > [email protected]> > >> wrote: > >> > >>> > >> > >>>> Guys, > >> > >>>> > >> > >>>> I wanna kick-off the discussion on the content of 0.7.0 BOM > >> > >>>> Release 0.6.0 was all about stabilization of the stack and I > think > >> we > >> > >> got a > >> > >>>> great headway on that. The following components/OS were in the > >> frame of > >> > >> the > >> > >>>> discussion: > >> > >>>> http://is.gd/H52iVe > >> > >>>> > >> > >>>> My personal take that we need to spend this release cycle working > >> on the > >> > >>>> improvements in Bigtop itself: we got enough "technical debts" in > >> the > >> > >>>> pipeline that have to be addressed. To name a few: > >> > >>>> - testability/test coverage > >> > >>>> - test framework > >> > >>>> - package improvements > >> > >>>> - build improvements (including performance) > >> > >>>> > >> > >>>> In order to be able to deliver a solid stack again yet improve > all > >> things > >> > >>>> Bigtop I'd like to focus on the latter, hence keeping the former > at > >> bay > >> > >> and > >> > >>>> limiting the component updates to the bugfix releases only (if > >> > >> warranted). > >> > >>>> E.g. > >> > >>>> > >> > >>>> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2) > >> > >>>> HBase 0.94.9 (update from the Bigtop 0.6.0) > >> > >>>> HCatalog 0.5.0 (same as 0.6.0, as well as following...) > >> > >>>> Zookeeper 3.4.5 > >> > >>>> Pig 0.11.1 > >> > >>>> Hive 0.10.0 > >> > >>>> Sqoop 2 > >> > >>>> Oozie 3.3.2 > >> > >>>> Whirr 0.8.1 > >> > >>>> Mahout 0.7 > >> > >>>> Flume 1.3.1 > >> > >>>> Giraph 0.2.0 > >> > >>>> Hue 2.2.0 > >> > >>>> Datafu 0.0.6 > >> > >>>> Solr 4.2.1 > >> > >>>> Crunch 0.5.0 > >> > >>>> Tomcat 6.0.36 > >> > >>>> Spark 0.7.3 (it has been in the queue for a long time) > >> > >>>> > >> > >>>> Also, I'd suggest to keep the same set of OSes as last time: > >> > >>>> > >> > >>>> CentOS/RHEL5 > >> > >>>> CentOS/RHEL6 > >> > >>>> SLES11 > >> > >>>> Ubuntu 12.04 (LTS) > >> > >>>> Fedora 18 > >> > >>>> OpenSUSE 12.3 > >> > >>>> Ubuntu 12.10 > >> > >>>> > >> > >>>> To reiterate, with a known stable version of the stack we can > safely > >> > >> focus > >> > >>>> on > >> > >>>> the improvements to the framework and the overall system > usability. > >> > >>>> > >> > >>>> Please jump on the discussion Also, I have opened up the > following > >> JIRA > >> > >> to > >> > >>>> track the BOM update > >> https://issues.apache.org/jira/browse/BIGTOP-1023 > >> > >>>> > >> > >>>> Thanks, > >> > >>>> Cos > >> > >>>> > >> > >>>> > >> > >>> > >> > >>> > >> > >>> -- > >> > >>> Best regards, > >> > >>> > >> > >>> - Andy > >> > >>> > >> > >>> Problems worthy of attack prove their worth by hitting back. - > Piet > >> Hein > >> > >>> (via Tom White) > >> > >> > >> > > > >> > > > >> > > +1 to what Andrew and Sean said. > >> > > I would also like to add: > >> > > * I would also like to keep the door open for Apache Gora to come > in. > >> > > Someone is already working on it > >> > > * Fedora 19 is out and maybe 20 will be out by the time we release > >> 0.7.0 > >> > > > >> > > With regard to stability, if included all the changes listed so far > >> seem > >> > > a little bit too much, what about dedicating a 0.6.1 to stability so > >> > > 0.7.0 can include all the shiny things listed above? A 0.6.1 would > >> > > enable us to be more strict with the patches/fixes/updates being > >> > > included as well as signaling a more stable version to users. > >> > > > >> > > > >> > > Thanks, > >> > > Bruno > >> >
