On Mar 14, 2018, at 03:07, Rainer Müller wrote:
> On 2018-03-14 04:08, Ryan Schmidt wrote:
>> I was not aware of the existence of GitLab CI, but I haven't done a survey 
>> of CI systems, mainly because we already selected one many years ago: 
>> Buildbot. The problem, to my mind, was not that of selecting which CI system 
>> to use, but the fact that we want to have a virtual machine template that 
>> the CI system should clone, boot up, run a job in, and then delete, or even 
>> multiple templates, one for each OS version we want to test. I see now that 
>> GitLab CI has support for doing that with both VirtualBox and Parallels.
>> 
>> https://docs.gitlab.com/runner/executors/README.html
>> 
>> That's good, and I didn't know that it had that feature. The Xserves running 
>> the Buildbot might have room for a couple additional virtual machines, 
>> especially after the duplicate 10.6/10.7/10.8 VMs are removed after we 
>> switch those users from libstdc++ to libc++. But out Buildbot VMs run on 
>> VMware ESXi, so if I were to add builds for PRs to the system, I would of 
>> course want to continue using VMware. It might be possible to add support 
>> for VMware to GitLab CI, but it is probably just as possible to add it to 
>> our Buildbot configuration, and personally I would rather do that and 
>> continue to work with the CI system I'm already familiar with than learn a 
>> whole new one, and have to remain proficient with both. Adding this to our 
>> Buildbot config doesn't seem outside the realm of possibility, but beyond 
>> being aware that VMware ESXi has a command line interface by which VMs can 
>> be cloned, started, stopped, and deleted, I haven't looked into it.
> 
> Buildbot already support VMs via libvirt, which in turn works with many
> hypervisors inclusing ESXi.
> 
> https://libvirt.org/drivers.html
> 
> http://docs.buildbot.net/0.8.12/manual/cfg-buildslaves.html#libvirt
> http://docs.buildbot.net/1.1.0/manual/cfg-workers-libvirt.html

Good to know. I can take a look at that.


>> We also have the problem of what exactly to put on the VM templates, and how 
>> to keep them updated. Certainly, a VM template would need to contain an 
>> installation of macOS, and Xcode, and MacPorts, and Java, and the VMware 
>> tools, and the necessary CI software such as Buildbot worker. But one of the 
>> reasons why our Buildbot setup is as fast as it is is that each Buildbot 
>> worker keeps previously-built ports installed (but inactive). So when 
>> someone commits a change to the demeter port, for example, the Buildbot does 
>> not have to first build all of demeter's 430 recursive dependencies; it just 
>> has to activate them. If we instead clone a VM template that doesn't have 
>> any ports installed yet, it will take a lot longer to first install all 
>> those dependencies. Best case, all the dependencies are distributable, were 
>> already built on the post-commit Buildbot workers and uploaded to the 
>> packages server, and so they just need to be downloaded and extracted. Worst 
>> case, they aren't distributable and have to be built from source.
> 
> Parts of /opt/local/var/macports could be put onto a NFS mounts to make
> the distfiles and binary archives persistent. However, we would need to
> ensure these files cannot be corrupted by the build, which might be
> possible to achieve with union mounts.

No comment. I have zero NFS experience.

Reply via email to