hashar claimed this task. hashar added a comment. I remember some similar conversation a few days ago which I think i replied to but it is good thing to have it there.
The reason the tests are rerun is because either: **a) something is about to be changed** **b) another repository the job depends upon might have changed** Lets describe the two cases: **a) something is about to be changed** When changes enter Zuul gate via Code-Review +2 (pipeline gate-and-submit), they are enqueued speculating the patches ahead in the queue are going to pass the tests. Ensuring the branch will still be passing. As an example, imagine two changes based on the same parent commit (tip of master branch): * Patch A change some internal API in a non compatible way and update all calls. Tests pass * Patch B rely on the same internal API as it was in master branch. If one +2 B and then +2 A Zuul tests in parallel: Patch B -> pass tests, get merged Patch A with Patch B applied -> probably fails tests since Patch A did not update the code introduced by B. If one +2 A and then +2 B Zuul tests in parallel: Patch A -> pass tests, get merged Patch B with Patch A applied -> probably fails tests because of A changing the API. **b) another repository the job depends upon might have changed** The job running unit tests for extensions depends at least on three repositories: mediawiki/core , mediawiki/vendor and the extension itself. When a patch has been proposed for an extension 'master' branch, it has been tested with the tip of the master branches of core and vendor in whatever state they were when the patch was proposed. The extension change can be +2ed a few day later and core/vendor master branches most probably have evolved even if the extension change parent commit is still the tip of the extension master branch. Thus we rerun the tests simply because the other repositories branches might have changed. Combined with the first use case, you can ensure that your extension is being tested with code in other repositories that is about to land in. Ie: Patch A propose a change to mediawiki/core and is +2ed Patch B propose a change to an extension and is +2ed Zuul will enter in the queue: Patch A of mediawiki/core Patch B of extension tested with mediawiki/core with Patch A applied ** conclusion ** I don't think we can easily skip running tests without a massive effort to build the dependency tree of all repositories involved in tests. It seems much easier to brute retest everything. I can understand developers being sometime angry because it takes a few minutes to have a change merged. But in most case I don't think we need to have merge to happen in a few seconds. In case of emergency, one can still cherry pick the patch and deploy it, thus bypassing all tests. We will probably end up testing all extensions together at some point, which will make the tests to take 15 - 20 minutes. But then Zuul tests them in parallel so that speed things up even if they are enqueued sequentially. If tests are deemed too slow, that would need developers efforts to make them faster, and I believe there is a lot of low hanging fruits that would let us easily cute the running time. The Zuul behavior is explained in upstream documentation at http://ci.openstack.org/zuul/gating.html . I wrote the second part "Cross projects dependencies". I am more than willing to help enhance that documentation if it is unclear. TASK DETAIL https://phabricator.wikimedia.org/T796 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>. To: hashar Cc: wikibugs-l, greg, ori, Legoktm, hashar, Krinkle, zeljkofilipin _______________________________________________ Wikibugs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
