Hi Daniel, thanks for following up!
Il giorno mer 30 ott 2019 alle ore 13:05 Daniel Ruggeri <drugg...@primary.net> ha scritto: > > > > On October 29, 2019 8:12:25 AM CDT, Luca Toscano <toscano.l...@gmail.com> > wrote: > >Il giorno mar 29 ott 2019 alle ore 12:23 Joe Orton <jor...@redhat.com> > >ha scritto: > >> > >> On Sun, Oct 27, 2019 at 06:42:58PM +0100, Luca Toscano wrote: > >> > Some updates: > >> > > >> > - We have support for httpd in travis - > >https://travis-ci.org/apache/httpd > >> > >> Nice, thanks Luca & infra! > >> > >> > - In order to configure automatic builds, a travis.yaml file is > >needed > >> > in the base path of the repository, in every branch that needs to > >be > >> > tested IIUC. > >> > - My idea is currently to add in trunk a travis.yaml initial > >> > configuration file, that builds a Docker image like [1] and runs > >the > >> > perl test suite. > >> > > >> > - Building the Docker image in [1] takes quite a while now (between > >9 > >> > and 13 mins on my laptop, depending also on network bandwidth > >etc..) > >> > so it will need more time before it could be as fast as we need, > >but > >> > we have to start from somewhere :) > >> > - In the travis.yaml file we'd need to put config options about > >what > >> > Docker image to build and with what parameters. Ideally I'd like to > >> > store the Docker images in the httpd-framework repository, and > >> > >> Do you mean Dockerfiles rather than images here? > > > >Correct yes, my bad! > > > >> > >> > reference them in the Travis config of the httpd branches, but not > >> > sure if it will be possible. Worst case scenario we'll need to add > >the > >> > Docker image in each httpd branch that we want to test (possibly in > >a > >> > dedicated dir, like "docker" or "testing". > >> > >> I expected two goals for testing: > >> > >> 1. configure, build, and test using a variety of different configure > >> options in the standard Ubuntu environment which Travis provides > >(enable > >> different MPMs, different module sets, different > >> --enable-foo/--with-foo, different gcc versions etc etc) > > > >This is also what I was trying to understand in these days, namely if > >travis could build and test directly without Docker. > > > >> 2. configure build and test in a variety of OS environments - this > >would > >> make sense using container images. > >> > >> I don't have much experience testing inside containers from Travis, > >but > >> if we can do both (1) and (2) inside containers that might make > >sense? > >> Or can we do both separately? If you have a .travis.yml which works > >> then I'd say go commit it and we can work from there. > > > >I have currently a Docker file that works for Debian/Ubuntu (and > >another one for CentOs), and IIUC the same could also be done for > >other OSes (so having different Docker files for OSX and Win, since > >Travis seems to support them). The main problem is that it currently > >takes minutes to build the Docker images, and only after that tests > >will be able to run. I guess that this problem will be the same if we > >run directly tests in Travis, but we can give it a try. Another > >downside about Travis, in my opinion, is that testing locally seems to > >be more difficult that building Docker images (but I admit my > >ignorance so it might be different), this is why I preferred Docker in > >the first place. > > This very closely matches my own testing setup. I *really* like this. Thank > you for driving this! > > > > >> Or can we do both separately? If you have a .travis.yml which works > >> then I'd say go commit it and we can work from there. > > > >I currently don't have a working travis yaml, but I'll come up with > >something during the next days. What I am currently trying to > >understand is if the Docker files could be stored elsewhere (like in > >the httpd-framework repo) and downloaded in Travis on demand, to have > >only one the Travis config in the httpd branches (as opposed to also > >having Docker files etc..). > > > >Let me know your suggestions/thoughts :) > > > >Luca > > The most I know about Travis is how to spell it, but... > What if the various Dockerfile-based testing environments were treated as a > separate "CI-able" item? That is, what if we monitored for upstream changes > then proactively rebuilt and pushed the testing images as > "httpdtest-centos-7" and "httpdtest-ubuntu-trusty", etc? That way, the > testing image is built/ready so the $work needed on check-in is just to build > and test httpd. > > A significant downside is that these discreet test "enviroments" will all > have their own lifecycles and maintenance burden. I found on my Debian-based > Docker test environment that subtle upstream changes caused unexpected > breakage. This was pretty disruptive to workflow because I had to get out of > my "httpd tester" hat and don my "Debian administrator" hat in the middle of > testing whatever I was working on to resolve the breakage. With that in mind, > I view the pre-build mechanism as a way to avoid breakages during a testing > cycle. I like the idea, just asked in the task to infra if something like the following is feasible: - have a separate github repo for the Dockerfiles (that could even be the one of the test suite replicated in github in theory) - after each commit to that repo, an automatic Docker build process is launched to create base test images in Dockerhub (how to handle authentication etc..? Open question) - Add travis config to each httpd branch that we need to test, that simply pulls the latest images from Dockerhub, checks out the httpd git commit that is under test, builds it with some ./configure and runs the perl test suite. This step would be great to be run on different platforms. I took a look to Jira and other Apache projects in Github, and didn't find anything similar to our use case (even if I am pretty sure we are not the first ones :). An alternative could be to have something like the following: - Travis used directly (as Joe mentioned) on multiple platforms to build httpd after each commit, maybe with different configure options etc.. Apache Arrow uses the Travis matrix functionality in a similar way https://github.com/apache/arrow/blob/master/.travis.yml - add the "make docker platform-x" functionality (as previously suggested) to the test suite's repo, and have people to run manually docker tests when needed for multiple platforms. So basically extending buildbot using Travis, and enrich our testing tooling. Anyway, I'll wait a bit for infra and keep going with my research, and as test I'll try to push a Travis config to trunk during the next days (to have an idea about wait times etc..). Luca