On Wed, 25 Feb 2015 18:57:02 +0100 Christian Tzolov <[email protected]> wrote:
> Hi Konstantin, > > Below are some of my findings. Note that so far i've played only with > CentOS6. > > - Missing python-setuptools and asciidoc packages. Don't remember > which project needed them. > - To build spark-rpm you need install scala and set the SCALA_HOME > env. > - Due to https://issues.apache.org/jira/browse/CRUNCH-500 you can't > use Maven > 3.1. I've fixed this in the Crunch project but will not be > available until Crunch 0.12 release. For the time being can create a > patch for Crunch's do-component-build? In the Dockerfile I've > downgraded maven to 3.0.5 > - Hue (hue-ui) requires non root user to build the rpm. Hue wanting a non-root user to build is a feature not a bug :) I've learned the hard way in packaging: *Never* ever build packages as root. > > ---- > > I've tried > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop > using the centos image. It seems like the /etc/profile.d/bigtop.sh is > missing? The 'toolchain' requires java to be installed first (and > JAVA_HOME to be set). This in turn requires 'sudo' package. For Spark > you need to install Scala and set SCALA_HOME. And for Hue you need to > create sudo, passwordless user... > Maybe with the ubunto image is better (haven't tried it) but for me > the above requires too many steps to prepare the environment. > > Hope this helps. > > Cheers, > Christian > > > > > > On Wed, Feb 25, 2015 at 4:12 AM, Konstantin Boudnik <[email protected]> > wrote: > > > Hi Christian. > > > > On Tue, Feb 24, 2015 at 09:24AM, Christian Tzolov wrote: > > > Hi Konstantin, > > > > > > I see your point. My goal however was to generate RPMs for some > > > projects (Spark particularly) and experiment with different > > > versions and configurations. > > > > > > I could not find a proper doc explaining how to bootstrap with > > > Bigtop. > > The > > > wiki pages like ( > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+RHEL7 > > ) > > > are outdated, missing libraries and stil refering to the 'make' > > > tool. > > > > Yup, it is - I just recently figured this out and filed the JIRA > > https://issues.apache.org/jira/browse/BIGTOP-1665 > > which you're welcome to take, if you want. > > > > > I've tried the './gradlew toolchain' to build an environment but > > > it still requires some libs to be preinstalled so i've decided to > > > keep everything > > in > > > a single Dockerfile. > > > > Could you please share what libs were missing? It'd be great to get > > it fixed. > > The issue here is the we are moving forward quite fast and > > sometimes devs env. > > isn't exactly pristine, so think like missing packages might go > > unnoticed. > > > > > Concerning only the packaging tasks I guess 2 environments (for > > > RPM and DEB) are enough? > > > > Yes, that's all Bigtop does on the packaging side. > > > > I have put together an initial draft for new build procedure > > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop > > > > and you're welcome to update/add to it. Hope it helps. > > > > Cos > > > > > On Mon, Feb 23, 2015 at 8:39 PM, Konstantin Boudnik > > > <[email protected]> > > wrote: > > > > > > > Thanks Christian. > > > > > > > > We have considered this on a multiple occasions and as you > > > > found for yourself > > > > the Dockerfile file is pretty simple, so the consensus was to > > > > just check-in a > > > > Gradle code for its generation, hence making containers > > > > preparation as stateless as possible. > > > > > > > > Roman should have this patch somewhere, not sure if he's ready > > > > to > > commit > > > > yet. > > > > But we always can use your approach is a backup plan, although > > > > we'd > > need > > > > Dockerfiles for all supported Linuxes. > > > > > > > > Cos > > > > > > > > On Mon, Feb 23, 2015 at 07:45PM, Christian Tzolov wrote: > > > > > I've put together a Docker container, equipped with all > > > > > necessary > > > > tools > > > > > and libraries and ready to generate RPM packages. Here are > > > > > the Dockerhub/Github references: > > > > > https://registry.hub.docker.com/u/tzolov/bigtop-centos > > > > > https://github.com/tzolov/bigtop-centos > > > > > There are a couple of existing bigtop images in Dockerhub > > > > > but > > neither > > > > of > > > > > them provides neither description nor a Dockerfile. > > > > > > > > > > For simplicity I've decided to keep it as as single image > > > > > for now. Cheers, > > > > > Christian > > > > > > > > Hope that helps, Peter
