Thanks for pointing the resources and suggestions , Roman, :) Regards the multi-arch, my intension is to simplify the images name, so the actual underlying things (arch-depent images) are transparent to Bigtop users/developers. Say, they can just issue same cmd: docker pull bigtop/puppet:1.5-centos-7, on any platform, and then start doing coll stuffs.
And another puprpose is to simplify provision/test process. As you can see there are some arch related yaml files in https://github.com/apache/bigtop/tree/master/provisioner/docker. If multi-arch is enabled, we can remove those config-*-aarch64.yaml files, and use just one unified yaml file for certain distro/packages. Also I think this could help devOps guys if they're using Bigtop to manage deployment. 2018-05-29 14:04 GMT+08:00 Roman Shaposhnik <[email protected]>: > Here's a few points from the peanut gallery: in general I find > Docker multiarch pretty useful, but at the same time its not like > having a schema like: > bigtop/<img_type>:<version_prefix>-<distro>-<distro_ > version>-<arch_name> > wouldn't achieve the same. I'd say go with what feels simpler. > > As for scripts that create those -- that's just puppet in: > https://github.com/apache/bigtop/tree/master/bigtop_toolchain > > So your Dockerfile is super-straightforward -- start with the base, > add puppet via: > https://github.com/apache/bigtop/blob/master/bigtop_ > toolchain/bin/puppetize.sh > and run the puppet. > > In fact, I believe that this bakes the images (unless it bitrotted > somehow): > https://github.com/apache/bigtop/blob/master/docker/ > bigtop-slaves/build.sh > > > Thanks, > Roman. > > On Sun, May 27, 2018 at 7:55 PM, Jun HE <[email protected]> wrote: > > Thanks for comments, Evans. > > > > There should not be much efforts to enable multi-arch. The question is > how > > we plan to support it in Bigtop. > > > > I can think of several items to discuss for this feature: > > 1. Docker images name conventions: > > For this, I'd suggest to using this: > > * for actual underlying docker image running on specific platform, > name > > it as: > > bigtop/<img_type>:<version_prefix>-<distro>-<distro_ > version>-<arch_name>, > > say: bigtop/puppet:trunk-debian-9-amd64, > > bigtop/slaves:1.3.0-fedora-26-aarch64 > > * for virtual image, name it as: bigtop/<img_type>: > > <version_prefix>-<distro>-<distro_version>, > > say: bigtop/puppet:trunk-debian-9, bigtop/slaves:1.3.0-fedora-26 > > 2. How to create these images/manifests > > I didn't find any cmds/scripts in Bigtop source to do the docker hub > > uploading work. So I guess we're doing it manually for now. If we're > going > > to use multi-arch, we need to upload the virtual image's manifest file to > > docker hub besides those arch dependent images. > > Docker cli has a cmd to do such work: docker manifest. But it is not > in > > distros' docker releases. And it is marked as "experimental" feature, > user > > needs to change docker settings to use it. See > > https://docs.docker.com/edge/engine/reference/commandline/ > manifest/#description. > > I created a shell script version ( > > https://github.com/JunHe77/docker-manifest-sh) to do the same thing, use > > same cmd syntax. Hope this can help. > > 3. Update existing settings/yaml files to use the virutal distro image. > > Once the images/manifests are ready in docker hub, we can update > > existing settings/yaml files to use "virtual" bigtop images. This should > > take too much efforts. And we'll get unified, beautiful code for > different > > archs. > > > > Pls raise your thoughts, questions, and etc. Once we settle these > > workflows/specs down, I'm sure it will be ready before Sept. > > > > Thanks. > > > > 2018-05-28 1:45 GMT+08:00 Evans Ye <[email protected]>: > > > >> I'd say this is a great idea. > >> The only reason we haven't adopt multi-arch is because we started to > adopt > >> docker at early stage so there's no such feature then. > >> > >> Jun can you roughly estimate the effort? If we're going for it, we > better > >> have it ready in 1.3.0 before ApacheCon which is around September. > >> > >> > >> 2018-05-17 17:19 GMT+08:00 Naresh Bhat <[email protected]>: > >> > >> > On 17 May 2018 at 13:21, Jun HE <[email protected]> wrote: > >> > > Hi, guys, > >> > > > >> > > As bigtop has simplified the way to create docker images for > distros, > >> by > >> > > BIGTOP-2922, I'm thinking of multi-arch docker images support to > ease > >> > > building/test/development. > >> > > > >> > > Multi-arch feature has been supported since 2017, you may check this > >> > link: > >> > > https://blog.docker.com/2017/11/multi-arch-all-the-things/ > >> > > In simple words, multi-arch is a virtual image which contains > several > >> > > architecture dependent images. When user try to pull this virtual > image > >> > > dockhub will send back the actual image based on the architecture > he is > >> > > using. > >> > > > >> > > Use multi-arch will help: > >> > > * development: user don't have to issue different cmd to pull > >> > puppet/slaves > >> > > images, "docker pull bigtop/slaves:trunk-debian-9" is enough > instead of > >> > "docker > >> > > pull bigtop/slaves:trunk-debian-9-ppc64le" or "docker pull > >> > > bigtop/slaves:trunk-debian-9-aarch64" > >> > > * test: unified yaml file for same distro. refer example in > >> > > provisioner/docker/config_* > >> > > > >> > > So how do you think? :) > >> > > >> > Thanks Jun for initiating the thread. > >> > +1 this makes sense. Is there any reason why we have not consider for > >> > using multi-arch images ? If anybody explains advantage/disadvantage > >> > of using multi-arch docker image in bigtop, it will help us to > >> > understand in a better way. > >> > > >> > > > >> > > Regards, > >> > > > >> > > Jun > >> > > >> >
