Hi Jay,

no not a stretch. We already have it.

yes we have containerized builds for long on all platforms (ppc64le, aarch64 and amd64). Local builders are deprecated since bigtop-1.2.

Documented in confluence
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop-trunk

What specifically are you missing?

Olaf



On 08.07.19 21:27, Jay Vyas wrote:
Hi olaf- we do have containeirzed builders now, right? I might be wrong about 
this, as I haven’t built full packages for bigtop in the last 12 months or so, 
but I used to use docker... Am thinking we should update docs to reflect that 
to build bigtop it’s easier to use our docker containers (Evans gave a good 
presentation on this 
https://events.static.linuxfound.org/sites/events/files/slides/How%20Bigtop%20Leveraged%20Docker%20for%20Build%20Automation%20and%20%20One-click%20Hadoop%20Provisioning%20ApacheBigData2015EU%20pub.pdf).
 ... and really just move away from using local builders of any other sort.... 
is that too much of a stretch?

On Jul 8, 2019, at 3:11 PM, Olaf Flebbe <[email protected]> wrote:

Hi

Right. This is automated compilation of protobuf is done while preparing the 
docker images.

bigtop_toolchain/manifests/protobuf.pp

I like to note we use this approach for all platforms for consistency.

Best,
Olaf


On 08.07.19 10:18, Zhenyu Zheng wrote:
Hi Yuqi,
Thanks for the reply, so can I understand that what bigtop did is similar
to what was suggested in:
https://groups.google.com/forum/#!topic/protobuf/fwLF5_t3q3U  ? I mean,
backport some patches
to 2.5.0 and build a protobuf that can work on ARM64?
BR,
Kevin Zheng
On Mon, Jul 8, 2019 at 4:14 PM Yuqi Gu <[email protected]> wrote:
Hi Zhenyu Zheng,

Pls try to build Hadoop in docker:
docker run -v `pwd`:/ws bigtop/slaves:1.4.0-ubuntu-16.04-aarch64 bash -l -c
'cd /ws ; hadoop-pkg'

The protobuf patches for Hadoop are list: Here
<
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/do-component-build#L45-L105


BRs,
Yuqi

On Mon, 8 Jul 2019 at 15:42, Zhenyu Zheng <[email protected]>
wrote:

Hi Bigtop:

I'm trying to build Hadoop on ARM64 platform and I've meet some problem
related to protobuf 2.5.0 seems it does not build on ARM64, and there is
an
issue in Hadoop about this:
https://issues.apache.org/jira/browse/HADOOP-13363 which has not yet
been
solved. But I noticed that Bigtop has successfuly built Hadoop 2.8.5 on
ARM64, eg.


https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=hadoop,OS=ubuntu-16.04-aarch64/
so
I'm quite curious how bigtop solved this problem?

Thanks in advance

Kevin Zheng



Reply via email to