I think it's definitely easier to move everything out of /thirdparty for logistical reasons, rather than trying to trim it more surgically. Most of the intermediate solutions to building Apache Impala are harder if we have to special-case some parts of thirdparty. E.g. I was able to build Apache Impala by using 'svn checkout' to grab thirdparty from Cloudera github, but that would be trickier if thirdparty is still present in modified form.
We could have a thirdparty tree under be/src/ as well. On Wed, May 25, 2016 at 4:46 PM, Henry Robinson <[email protected]> wrote: > On 25 May 2016 at 16:33, Michael Ho <[email protected]> wrote: > > > Yes. The biggest dependency in thirdparty now is the reliance of the CDH > > components, namely > > hadoop, hbase, hive, sentry and some llama stuff. The short term goal as > > discussed in another email > > thread is to initially put a publicly available released version of those > > CDH components in S3 so a user > > can get all the components they need to build and run tests by just > > running the script to bootstrap the > > toolchain. Internally, we can replace those CDH components with our own > > builds of those components > > by somehow instructing the toolchain boostrapping script to pull from our > > internal repo if needed. The > > users can in theory specify their own repo for those components too. > > Whether we will converge to > > the Apache version of things in the long run is not the goal of this bug. > > > > The majority of the stuff left in thirdparty are mostly duplicates of > what > > exist already in > > the toolchain directory: avro-c, sasl, gflags, glog, gperftools, gtest, > > lz4, openldap, rapidjson, re2, snappy, > > thrift so we may as well remove them. > > > > After all the changes above, thirdparty will be pretty much empty except > > for mustache and squeasel. > > If I recalled correctly from the meeting we had a month ago before you > > went on leave, you pointed out > > that they can reside in util directory as they are actually compiled > > together with the webserver.cc. I am > > also open to keeping them in thirdparty directory as this means we can > now > > have a thirdparty directory > > in the ASF repo upstream after all the trimming above. > > > > Thanks for the explanation. I still don't see from your response why you > think Squeasel and Mustache *should* move to util/, even if, as we > discussed, they *can* (there are good reasons to consider moving them - > maybe we don't want thirdparty/ any more, maybe we don't want to treat > external compile dependencies any differently). > > The argument I can see in favour of not moving them is that any external > compile-time dependencies are in an easy-to-find place, which probably > makes getting an Apache release together just a little easier since we can > split the code into "developed as part of Impala" and "developed > externally". > > I'm not going to stand in the way of a small change like this, so feel free > to GVM if you think best. > > > > > > Michael > > > > On Wed, May 25, 2016 at 3:30 PM, Henry Robinson <[email protected]> > wrote: > > > >> I thought the long-term goal was to remove any heavyweight dependencies > >> that are checked into the repository. It's not the existence of > thirdparty/ > >> that's the problem there, it's what we were keeping in it. > >> > >> Mustache and Squeasel change very infrequently, and are only a few K in > >> total. Do you think we should remove thirdparty/ for other reasons? > >> > >> On 25 May 2016 at 14:20, Michael Ho <[email protected]> wrote: > >> > >>> The long term goal is to remove thirdparty. I thought we discussed > about > >>> it. > >>> > >>> On Wed, May 25, 2016 at 2:13 PM, Henry Robinson <[email protected]> > >>> wrote: > >>> > >>>> (Sorry, not near Gerrit) why not keep them in thirdparty/? That makes > >>>> their provenance clear. > >>>> > >>>> Sent from my iPhone > >>>> > >>>> > On May 25, 2016, at 2:06 PM, Dan Hecht (Code Review) < > >>>> [email protected]> wrote: > >>>> > > >>>> > Dan Hecht has posted comments on this change. > >>>> > > >>>> > Change subject: IMPALA-3223: Move squeasel and mustache to the util > >>>> directory > >>>> > > ...................................................................... > >>>> > > >>>> > > >>>> > Patch Set 1: Code-Review+2 > >>>> > > >>>> > Okay. > >>>> > > >>>> > -- > >>>> > To view, visit http://gerrit.cloudera.org:8080/3206 > >>>> > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings > >>>> > > >>>> > Gerrit-MessageType: comment > >>>> > Gerrit-Change-Id: I782e493bc28086a1587274b3c474ea6b6f201855 > >>>> > Gerrit-PatchSet: 1 > >>>> > Gerrit-Project: Impala > >>>> > Gerrit-Branch: cdh5-trunk > >>>> > Gerrit-Owner: Michael Ho <[email protected]> > >>>> > Gerrit-Reviewer: Dan Hecht <[email protected]> > >>>> > Gerrit-Reviewer: Jim Apple <[email protected]> > >>>> > Gerrit-Reviewer: Michael Ho <[email protected]> > >>>> > Gerrit-HasComments: No > >>>> > > >>>> > -- > >>>> > You received this message because you are subscribed to the Google > >>>> Groups "impala-cr" group. > >>>> > To unsubscribe from this group and stop receiving emails from it, > >>>> send an email to [email protected]. > >>>> > For more options, visit > >>>> https://groups.google.com/a/cloudera.com/d/optout. > >>>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> Michael > >>> > >> > >> > > > > > > -- > > Thanks, > > Michael > > > > -- > > You received this message because you are subscribed to the Google Groups > > "impala-cr" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > For more options, visit > https://groups.google.com/a/cloudera.com/d/optout. > > >
