Hi, Evans, Thanks for helping to double check the CI jobs. :)
I created a job: https://ci.bigtop.apache.org/job/Bigtop-1.3-packages/, to make sure all packages on branch-1.3 are buildable ( As you may note in this job the config matrix is defined as: ========================================================================= * | | centos-7 | fedora-26 | debian-9 | ubuntu-16.04 | opensuse-42.3 *| | *aarch64-slave | O | O | O | O | O |* Component A | *amd64**-slave** | O | O | O | O | O *| * |* *ppcle64**-slave** | O | O | O | O | O *| ========================================================================= aarch64 is a label for all useable aarch64 nodes, so are amd64 and ppcle64 labels. To build these packages "1.3.0-*-puppet/1.3.0-*-slaves" images need to be presented on each build node, as Jenkins may dispatch same componentA+distro job to any useable arch-slave. To install these images on each single node one way is to pull images from dockerhub, like this does ( https://ci.bigtop.apache.org/job/Docker-Toolchain-Trunk-pull/). But as v1.3 is not officially released I assume that I should not push these images (1.3.0-*-puppet/1.3.0-*-slaves) to docker hub as public avaliable (yes, you can find 1.3.0-puppet images on docker hub now, that's by mistake, :/). That's why I set the matrix to be node+distro in job [1][2], to install all images on every node by building them, and in job [1][2] configuration there is no "docker push". The origin plan is to update job[1][2] configuration as you mentioned, rebuild and push to dockerhub once v1.3 is ready to release. If we can publish the images to docker hub while the release is still in progress, I'm fine with the change. Althrough a little modification is to use arch-slave lables to align with settings in "Bigtop-1.3-packages". :) Regards, Jun Evans Ye <[email protected]> 于2018年9月1日周六 下午9:46写道： > Jun He, > > I just modified two CI jobs[1][2] you created but then I realized I should > discuss with you first before the modification. So, I'm sorry in advance. > > What I'd like to discuss is I saw your way to setup these CI jobs is to > have a matrix of supported distro + arch * build slaves. The matrix looks > like this: > > slave01 slave02 slave-aarch-01 > slave-ppc64le-01 > distro + arch 1 O O X > O > distro + arch 2 O X O > O > > This may have slaves running same arch producing duplicate images. See > below example: > > * slave06: > bigtop/puppet 1.3.0-ubuntu-16.04 6527358ce96e 3 days ago > 318MB > * slave07: > bigtop/puppet 1.3.0-ubuntu-16.04 29f1b8969d31 2 days ago > 318MB > > This may have problem since we can't trace the lineage of downstream > images. For example, the bigtop/puppet image pushed to dockerhub is > 6527358ce96e, but bigtop/slaves image pushed to dockerhub may be based on > 29f1b8969d31(though I didn't check). Hence we need a stable foundation to > evaluate the bigtop software stack. > > I've [1] and [2] modified to build images on whatever slave that is capable > of building it. Once images are built, we can trigger pull job to sync all > the images back to all slaves. Do you think this makes sense? Would love to > hear your feedback. And thanks for your hard work! > > Best, > Evans > > [1] https://ci.bigtop.apache.org/view/Docker/job/Docker-Puppet-1.3/ > [2] https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-1.3.0/ >
