On 5/17/21 3:32 PM, Francesco Chemolli wrote: > On Mon, May 17, 2021 at 8:32 PM Alex Rousskov wrote: > > On 5/17/21 2:17 AM, Francesco Chemolli wrote: > > $ make all push > > Does that "make push" command automatically switch Jenkins CI to using > the new/pushed containers? Or is that a separate manual step?
> Right now that's automatic. > I'm pondering the best way to have a staging step; Docker supports > versioning of images but we're using that feature to implement > multiarch. In order to rely on that I'll need to change the naming > conventions we rely on. I think you should be free to use whatever versioning/naming scheme you think works best for supporting safe Jenkins CI upgrades. Ideally, the new docker images (and any other changes) should be tested (against official branches) _before_ the official tests are switched to use them. I hope you find a way to implement that without a lot of work. If you need to tap into Foundation resources, please say so! > The overarching objectives are: > - focus on where most users probably are > - allow iterating quickly giving some signal on tracked branches > - allow landing quickly with more signal > - be slow but exhaustive on every other platform we can cover. Do not > block landing new code on less common or harder to test on platforms, > but still rely on devs to own their changes there, too. What exactly do you mean by "own their changes there"? The rest is general/vague enough to allow for many interpretations I can agree with, but that last bit seems rather specific, so I wanted to ask... > I think that full round of PR tests (i.e. one initial set plus one > staging set) should not exceed ~24 hours, _including_ any wait/queuing > time. > Is everyone okay with such a slow turnaround? Well, you should not overload the question by calling a 24-hour delay for an all-green signal slow :-). It is all relative, of course: Most Squid PRs take a lot longer to review and fix. There are open source projects where PRs wait for a week or more, so 24 hours for a full round (and usually just an hour or two for the first signal!) would feel very fast for them. Etc. Yes, 24 hours is not "interactive", but it is acceptable for the vast majority of PRs AFAICT, and you are providing dockers for those who want to test in a more interactive mode. > I think you are proposing to make some tests optional. Which ones? > > > Not the tests, but the platforms. From my test results consumer point of view, they are [platform-specific] [build] tests. > Things like gentoo, fedora rawhide, freebsd, openbsd. > Testing is slower there, so results will lag and we need to batch, > running a full test suite every week or so OK, so you propose to make slow Jenkins platform-specific tests (i.e. Jenkins tests on platforms where tests run slowly today) optional and those platforms are gentoo, fedora rawhide, freebsd, openbsd. Got it! What happens when such a slow test fails and the PR author ignores the failure of that optional test and/or the PR is already closed? The answer to that question is the key to evaluating any proposal that declares any tests optional IMO... > FWIW, I do not think reducing the number of #ifdefs will solve our > primary CI problems. > Each test build is right now 4 full builds and test suites: > autodetected, minimal, maximum, everytthing-in-except-for-auth And all of that takes less than 10 minutes on decent hardware which we can rent or buy! To me, it feels like we are creating an unsolvable problem here (i.e. testing a lot of things quickly on a raspberry Pi) and then trying extra hard to solve that unsolvable problem. We should either stop testing a lot of things or we should use appropriate resources for testing a lot of things quickly. Yes, we can reduce testing time by removing #ifdefs, but that is a lot of work and does not really scale. #ifdefs should be removed primarily for other reasons. Cheers, Alex. _______________________________________________ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev