Hello Wouter, all Sorry for taking so long to answer about this.
El 02/10/18 a las 17:31, Wouter Verhelst escribió: > On Wed, Sep 26, 2018 at 07:36:03PM +0200, Wouter Verhelst wrote: >> I have been working on improving the setup to split out a full build >> into multiple jobs -- see >> https://salsa.debian.org/wouter/webwml/pipelines/19296 for an example. >> Since the timeout is per job and not per pipeline, that would work >> around this problem. However, it doesn't seem to be working properly >> yet; the "make install" step seems to be calling "wml" occasionally, >> which means things take time, and we don't want that (because it would >> make things time out). I'm not exactly sure whether that's my fault or >> the build system's. > > I've now fixed this by making every individual language step do "make > install" rather than just "make all", and by having the "pages" phase > rsync each individual language directory into the main "public" > directory, which should then do the final publication. > Thanks for all the work on this, I just merged it[1] into the main webwml repo. [1]: https://salsa.debian.org/webmaster-team/webwml/merge_requests/30 > It works, except that deploying the created output fails with an error > message that the produced data is too large. I'm sure this is something > which the salsa administrators could fix if we asked them nicely. > As we have talked in IRC during the tests, the current status is that the CI jobs build the website but it is not published as "pages", because of a size limitation (100MB) and (if I understood correctly) we should review the documentation and try to use deployment environments: https://docs.gitlab.com/ee/ci/environments.html Any help is appreciated on this, and I will do my best to reply quicker with tests and feedback. Meanwhile: * we can learn if commits break the builds (I'm not sure who gets notifications (I get them, but not sure if because of my particular mix of notification-levels and project-membership), you can probably tweak your profile settings in Salsa if you don't get them and want to get them), and we can also have a look at the CI results prior to merge some merge request. * People can browse/download the results of the CI builds in the Salsa web interface, going to the webwml project, CI/CD, Jobs (or Pipelines). Note that the artifacts corresponding to languages (other than English) will exist (containing the /english folder) even if that language was not built (if the language was built, the zip will contain the /english and the /$language folders). > At any rate, I plan to replace the code in my merge request with the > code from this branch[1], it seems like a better option. > > There are still a few things that could be done to improve this, > however: > > - The webwml Makefile is currently set up to ignore failures and > continue. For a continuous integration system, this is suboptimal, > although I understand that it makes sense for the main website. I > would like to suggest that ignoring failures is not done if some > environment variable is set; the CI jobs can then just set that > variable and allow things to fail if something is broken > - The jobs start off with running "apt-get install" for various > packages. This works, but it is more efficient to create an image on > the docker hub which already has those packages preinstalled. I'll > look at doing so. I will create bugs to track these two proposals. > - I'm not sure what happens if changes are made to multiple languages in > a single git push. There are two possibilities: either gitlab will > merge pushed caches from a single stage so that the cached result will > have all files from all translations (which is good, that's what we > want), or pushing a cache will overwrite all other caches from the > same phase (in which case the publish stage will just push the English > version of the site together with and whichever translation happened > to be finished last). I'll need to experiment and see if that needs > fixing. Even if the latter of those two possibilities is the case, > however, I still think that seeing an auto build break is a good idea, > so regardless of that I still think this is progress. +1 I'll keep an eye on things in the following days, and report back to the mailing list. Cheers -- Laura Arjona Reina https://wiki.debian.org/LauraArjona
