> On May 16, 2013, 10:07 p.m., Benjamin Hindman wrote: > > hadoop/TUTORIAL.sh, line 573 > > <https://reviews.apache.org/r/11110/diff/1/?file=290968#file290968line573> > > > > Curious, why can't you write a script which does: > > ------------------------ > > distribution=... > > > > TUTORIAL.sh ${distribution} > > /path/to/pre-package > > tar czf hadoop-${distribution}-mesos-airbnb.tar.gz hadoop > > /path/to/post-package > > ------------------------ > > > > That is, use composition to keep from needing to repeat anything. > > > > The real problem here is that the tutorial was never meant to be a > > build system for the Mesos distribution. How do other shops build hadoop > > specific distributions? Is there a clean declarative approach people use? > > > > Also, while I can imagine wanting to bring in other contribs or > > libraries before you package the distribution via pre-package, I can't > > imagine how post-package is useful if the only goal of the tutorial is to > > create a distribution (again, not to be a general build utility). > > > > I acknowledge the fact that creating a Mesos distribution is > > non-trivial (or even a Hadoop one for that matter) and so I would be okay > > with adding pre-package to the tutorial but I would like to see a lot more > > explanation given to the user running the tutorial about what the > > pre-package hook script is intended to be used for. In particular, if this > > is being used to add other contribs I'd think that this would need to get > > done before running ant.
I suppose I could do things the way you suggest. I generally like the idea of using the `make' command because it's familiar and well supported by other tools. I'm not sure how others are building Hadoop distributions, but I would assume they make use of the existing Hadoop build system where possible. Since Mesos uses make, it follows that one should be able to build the Mesos Hadoop distribution with make as well. The post-package script is not necessarily required (it could be done by another script afterwards). It seemed sensible at the time to handle the post-package tasks the same way as the pre-package tasks within the tutorial at the time of writing. I think it would be better to separate the creation of the distribution from the tutorial altogether, perhaps using just a Makefile to build the distribution. - Brenden ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11110/#review20665 ----------------------------------------------------------- On May 14, 2013, 12:19 a.m., Brenden Matthews wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11110/ > ----------------------------------------------------------- > > (Updated May 14, 2013, 12:19 a.m.) > > > Review request for mesos. > > > Description > ------- > > From decdc5c271c47b96d706666665313dae27321fff Mon Sep 17 00:00:00 2001 > From: Brenden Matthews <[email protected]> > Date: Mon, 15 Apr 2013 14:11:16 -0700 > Subject: [PATCH 03/24] Include pre/post-package hook scripts in Hadoop > tutorial. > > Review: https://reviews.apache.org/r/11110 > --- > hadoop/TUTORIAL.sh | 24 ++++++++++++++++++++++++ > 1 file changed, 24 insertions(+) > > > Diffs > ----- > > hadoop/TUTORIAL.sh d81c55562d217e8784745d4a8cedf78ada9f5453 > > Diff: https://reviews.apache.org/r/11110/diff/ > > > Testing > ------- > > Used in production at airbnb. > > > Thanks, > > Brenden Matthews > >
