> On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/TUTORIAL.sh, line 569 > > <https://reviews.apache.org/r/10346/diff/1/?file=279031#file279031line569> > > > > What compression does J represent? It doesn't work on my MAC (OSX 10.8). > > > > How about.. s/-cjf/-czf/
This is to use the XZ format instead of gzip. Support for XZ should be included in modern GNU systems (I don't know about others). It's nice but not necessary. See http://tukaani.org/xz/format.html for details. > On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/TUTORIAL.sh, lines 581-589 > > <https://reviews.apache.org/r/10346/diff/1/?file=279031#file279031line581> > > > > I think you added this because the jobtracker was failing on your Mac? > > I would rather we find and fix that bug, than having to skip it. It just > > sets a better precedent for our code. > > > > Can you file a bug (with a stack trace and logs) on your tutorial > > output? I added it because a) it was failing and b) it isn't necessary to run it every time you rebuild the package. I'll have to check again to see if it's still not working. > On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch, line 10 > > <https://reviews.apache.org/r/10346/diff/1/?file=279032#file279032line10> > > > > We definitely want to be able to run this both locally and on the > > cluster. And your change, would render local runs in a non-working state. > > > > I've sent a review out that should work for both local and remote runs. > > I will get it committed. You can base your changes off that to patch > > cdh4.2.0_hadoop-env.sh. If you run from ./build/hadoop instead of ./ it should work (or at least it does for me). I think the latter is more correct than the former, but perhaps you have a better solution. > On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/TUTORIAL.sh, line 165 > > <https://reviews.apache.org/r/10346/diff/1/?file=279031#file279031line165> > > > > i wasn't able to confirm whether skipping the compression level while > > extracting always works with all versions of tar. > > > > so, lets stick with specifying it (here it is 'z') unless we are sure. > > i'm going to revert my change in the launcher (in a review that i sent last > > week) to do the same. It should work on modern GNU systems, but I can't speak for Mac OS. Presumably (from your description) it does not. > On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/TUTORIAL.sh, line 547 > > <https://reviews.apache.org/r/10346/diff/1/?file=279031#file279031line547> > > > > Is this required by the pre-package script? If yes, please add a > > comment here. In my case I use it, but it's not required. I use the hook scripts for doing things like including custom configuration parameters (i.e., 'cp /etc/hadoop/config/* config/') before the package is created, and copying the package to HDFS after it's ready. > On April 8, 2013, 9:55 p.m., Vinod Kone wrote: > > hadoop/TUTORIAL.sh, line 549 > > <https://reviews.apache.org/r/10346/diff/1/?file=279031#file279031line549> > > > > Are these scripts only applicable for Cloudera versions? If yes, may be > > we can just test against that, like we do everywhere else. My scripts are version independent, but you could certainly write scripts that depend on a particular version. - Brenden ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10346/#review18797 ----------------------------------------------------------- On April 8, 2013, 9:21 p.m., Brenden Matthews wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/10346/ > ----------------------------------------------------------- > > (Updated April 8, 2013, 9:21 p.m.) > > > Review request for mesos. > > > Description > ------- > > - Added CDH4.2.0 support to TUTORIAL.sh > - Use .tar.xz for Hadoop dist (LZMA) > - Fixed Hadoop dist extraction in launcher > - Add pre/post packaging hooks to TUTORIAL.sh > - Don't run the jobtracker at the end by default > > > Diffs > ----- > > hadoop/Makefile.am bd8919a > hadoop/TUTORIAL.sh b0f77f3 > hadoop/hadoop-2.0.0-mr1-cdh4.1.2_hadoop-env.sh.patch c61ccec > hadoop/hadoop-2.0.0-mr1-cdh4.2.0_hadoop-env.sh.patch PRE-CREATION > hadoop/hadoop-2.0.0-mr1-cdh4.2.0_mesos.patch PRE-CREATION > src/launcher/launcher.cpp eb8d6fb > > Diff: https://reviews.apache.org/r/10346/diff/ > > > Testing > ------- > > Works in production on Hadoop at airbnb. > > > Thanks, > > Brenden Matthews > >
