Just making sure, so my next question then is what the story for arm64 container builds are.... and wether or not we need different base images (I assume we do).
That said I think you *can* build arm images on an x86 based computer as long as you set up the docket image correctly... and the original poster wasn’t able to build on Arm, so must be a subtle missing piece in our container builds.? > On Jul 8, 2019, at 4:05 PM, Olaf Flebbe <[email protected]> wrote: > > Hi Jay, > > no not a stretch. We already have it. > > yes we have containerized builds for long on all platforms (ppc64le, aarch64 > and amd64). Local builders are deprecated since bigtop-1.2. > > Documented in confluence > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop-trunk > > What specifically are you missing? > > Olaf > > > >> On 08.07.19 21:27, Jay Vyas wrote: >> Hi olaf- we do have containeirzed builders now, right? I might be wrong >> about this, as I haven’t built full packages for bigtop in the last 12 >> months or so, but I used to use docker... Am thinking we should update docs >> to reflect that to build bigtop it’s easier to use our docker containers >> (Evans gave a good presentation on this >> https://events.static.linuxfound.org/sites/events/files/slides/How%20Bigtop%20Leveraged%20Docker%20for%20Build%20Automation%20and%20%20One-click%20Hadoop%20Provisioning%20ApacheBigData2015EU%20pub.pdf). >> ... and really just move away from using local builders of any other >> sort.... is that too much of a stretch? >>> On Jul 8, 2019, at 3:11 PM, Olaf Flebbe <[email protected]> wrote: >>> >>> Hi >>> >>> Right. This is automated compilation of protobuf is done while preparing >>> the docker images. >>> >>> bigtop_toolchain/manifests/protobuf.pp >>> >>> I like to note we use this approach for all platforms for consistency. >>> >>> Best, >>> Olaf >>> >>> >>>> On 08.07.19 10:18, Zhenyu Zheng wrote: >>>> Hi Yuqi, >>>> Thanks for the reply, so can I understand that what bigtop did is similar >>>> to what was suggested in: >>>> https://groups.google.com/forum/#!topic/protobuf/fwLF5_t3q3U ? I mean, >>>> backport some patches >>>> to 2.5.0 and build a protobuf that can work on ARM64? >>>> BR, >>>> Kevin Zheng >>>>> On Mon, Jul 8, 2019 at 4:14 PM Yuqi Gu <[email protected]> wrote: >>>>> Hi Zhenyu Zheng, >>>>> >>>>> Pls try to build Hadoop in docker: >>>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-ubuntu-16.04-aarch64 bash -l >>>>> -c >>>>> 'cd /ws ; hadoop-pkg' >>>>> >>>>> The protobuf patches for Hadoop are list: Here >>>>> < >>>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/do-component-build#L45-L105 >>>>>> >>>>> >>>>> BRs, >>>>> Yuqi >>>>> >>>>> On Mon, 8 Jul 2019 at 15:42, Zhenyu Zheng <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Bigtop: >>>>>> >>>>>> I'm trying to build Hadoop on ARM64 platform and I've meet some problem >>>>>> related to protobuf 2.5.0 seems it does not build on ARM64, and there is >>>>> an >>>>>> issue in Hadoop about this: >>>>>> https://issues.apache.org/jira/browse/HADOOP-13363 which has not yet >>>>> been >>>>>> solved. But I noticed that Bigtop has successfuly built Hadoop 2.8.5 on >>>>>> ARM64, eg. >>>>>> >>>>>> >>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04-aarch64/ >>>>>> so >>>>>> I'm quite curious how bigtop solved this problem? >>>>>> >>>>>> Thanks in advance >>>>>> >>>>>> Kevin Zheng >>>>>> >>>>>
