Since bigtop 1.4 we have all things integrated at gradlew and its easier to build/test on docker. See: https://cwiki.apache.org/confluence/display/BIGTOP/Quickstart+Guide%3A+Bigtop+Integration+Test+Framework+2.0
Olaf Flebbe <[email protected]> 於 2019年7月9日 週二 04:05 寫道： > Hi Jay, > > no not a stretch. We already have it. > > yes we have containerized builds for long on all platforms (ppc64le, > aarch64 and amd64). Local builders are deprecated since bigtop-1.2. > > Documented in confluence > > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop-trunk > > What specifically are you missing? > > Olaf > > > > On 08.07.19 21:27, Jay Vyas wrote: > > Hi olaf- we do have containeirzed builders now, right? I might be wrong > about this, as I haven’t built full packages for bigtop in the last 12 > months or so, but I used to use docker... Am thinking we should update docs > to reflect that to build bigtop it’s easier to use our docker containers > (Evans gave a good presentation on this > https://events.static.linuxfound.org/sites/events/files/slides/How%20Bigtop%20Leveraged%20Docker%20for%20Build%20Automation%20and%20%20One-click%20Hadoop%20Provisioning%20ApacheBigData2015EU%20pub.pdf). > ... and really just move away from using local builders of any other > sort.... is that too much of a stretch? > > > >> On Jul 8, 2019, at 3:11 PM, Olaf Flebbe <[email protected]> wrote: > >> > >> Hi > >> > >> Right. This is automated compilation of protobuf is done while > preparing the docker images. > >> > >> bigtop_toolchain/manifests/protobuf.pp > >> > >> I like to note we use this approach for all platforms for consistency. > >> > >> Best, > >> Olaf > >> > >> > >>> On 08.07.19 10:18, Zhenyu Zheng wrote: > >>> Hi Yuqi, > >>> Thanks for the reply, so can I understand that what bigtop did is > similar > >>> to what was suggested in: > >>> https://groups.google.com/forum/#!topic/protobuf/fwLF5_t3q3U ? I > mean, > >>> backport some patches > >>> to 2.5.0 and build a protobuf that can work on ARM64? > >>> BR, > >>> Kevin Zheng > >>>> On Mon, Jul 8, 2019 at 4:14 PM Yuqi Gu <[email protected]> wrote: > >>>> Hi Zhenyu Zheng, > >>>> > >>>> Pls try to build Hadoop in docker: > >>>> docker run -v `pwd`:/ws bigtop/slaves:1.4.0-ubuntu-16.04-aarch64 bash > -l -c > >>>> 'cd /ws ; hadoop-pkg' > >>>> > >>>> The protobuf patches for Hadoop are list: Here > >>>> < > >>>> > https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/do-component-build#L45-L105 > >>>>> > >>>> > >>>> BRs, > >>>> Yuqi > >>>> > >>>> On Mon, 8 Jul 2019 at 15:42, Zhenyu Zheng <[email protected]> > >>>> wrote: > >>>> > >>>>> Hi Bigtop: > >>>>> > >>>>> I'm trying to build Hadoop on ARM64 platform and I've meet some > problem > >>>>> related to protobuf 2.5.0 seems it does not build on ARM64, and > there is > >>>> an > >>>>> issue in Hadoop about this: > >>>>> https://issues.apache.org/jira/browse/HADOOP-13363 which has not yet > >>>> been > >>>>> solved. But I noticed that Bigtop has successfuly built Hadoop 2.8.5 > on > >>>>> ARM64, eg. > >>>>> > >>>>> > >>>> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04-aarch64/ > >>>>> so > >>>>> I'm quite curious how bigtop solved this problem? > >>>>> > >>>>> Thanks in advance > >>>>> > >>>>> Kevin Zheng > >>>>> > >>>> > > >
