Am Freitag, 9. Februar 2018, 08:13:01 CET schrieb Thiago Macieira:
> On Thursday, 8 February 2018 23:02:36 PST Kevin Kofler wrote:
> > IMHO, you need to rethink your whole CI approach. This is increasingly
> > being the one bottleneck slowing down Qt development and releases. It
> > might make more sense to try a different approach, such as allowing all
> > commits through initially, then making CI runs at regular intervals, and
> > triggering reverts if things broke.
> Which will happen ALL the time. We'll never get back down: when we released
> Qt 4.2, 4.3 and 4.4, we were happy if only 10 tests failed (that only
> happened for QWS). For the other platforms, the normal number was a hundred
> tests failing. Then we spent weeks trying to get the number down.
> Sorry, I don't want to go back to that.
That's not exactly how I remember it from the Qt 4.6 and 4.7 times where there
was no CI yet, but we were actively fixing tests. There were always a few red
tests on testr.troll.no, but they were mostly flacky tests. What is it
compared to now with about 100 BLACKLIST files in qtbase alone? (there was no
blacklist at the time)
That said, I agree we should not go back to that.
> > Qt is being developed very much as a corporate project. (I write "as"
> > rather than "like" because that's what Qt is, despite Open Governance.)
> > It would help to look at how community Free Software projects do things.
> > They tend to be more efficient. And some company-developed Free Software
> > projects have already adopted such processes.
> It would not do us well to look at poorer practices than what we have. Just
> because everyone else is where we were 10 years ago is no reason for us to
> go back to it. Show us a *better* model, one that still prevents failures
> from being added, and we'll consider it.
How pretentious are you, thinking that everybody else has poorer practice?
Other projects still manage to release product with passing test. And might
not get ludicrous release delay "because of the CI system" that was supposed
to help, but gets in the way instead.
Anyway, here is some example of models which works:
In LLVM, devs commit directly. buildbots build the trunk continuously. In case
of failure, the buildbot maintainer quickly find out which commit likely broke
the test and reverts the commit imediatly (or commit a fix if the fix is
This works because there are buildbot maintainers taking care of the build
status, and they are not afraid to revert. Also the build and test time is
reasonable (while still being big), and individual developer can easily run
all the tests on their machine before submitting patches. (Very few platform
On other projects I've seen on github with travis and co.: the tests are run
for every pull request individually, before it is integrated. The reviewer
sees the result of the tests and decide whether to merge on not based on the
result. If one sees that the failures is obviously unrelated to the patch, the
reviewer will override and merge anyway.
I think this model could be easily applied to Qt.
In particular, there should be an option to merge a patch directly without
going through CI. (If you are scared of abuse, limit it to admin or
This could be used (exceptionally) for urgent patches such as patches that
fixes the CI and that are not being integrated quickly enough because of other
unrelated failures, continuing to block the CI for days. That way, the CI
would not get in the way.
In Summary: The CI should be a tool helping the development, and not slowing
it down. And having a way to override the CI for important patches should be
an easy and quick way to improve things a bit.
Woboq - Qt services and support - https://woboq.com - https://code.woboq.org
Development mailing list