[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
please reopen if this is still an issue ** Changed in: systemd (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in Auto Package Testing: Fix Released Status in systemd package in Ubuntu: Invalid Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
Thanks Iain! I'll keep an eye on this. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in Auto Package Testing: Fix Released Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
Done. Please let me know here if this is still a problem. ** Project changed: autopkgtest-cloud => auto-package-testing ** Changed in: auto-package-testing Status: New => Fix Released ** Changed in: auto-package-testing Assignee: (unassigned) => Iain Lane (laney) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in Auto Package Testing: Fix Released Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
Yeah OK, I'm pretty sure that change is bogus. I think it'll mean that if we hit the 'cache' we'll post back to the wrong status URL, since we don't check the environment matches. I'll just revert that for now. Sorry again. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in Auto Package Testing: Fix Released Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
Thanks, I would guess that I broke this somehow in https://git.launchpad.net/autopkgtest- cloud/commit/?id=58adff467c108506f979e229aa8e9e955a7d0d0a - which was a commit I made because systemd PRs were taking a long time after their jobs finished (several hours) to report any results due to that script having terrible runtime. I'll stare at that code and see if I can see what I did wrong. Sorry about that. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in autopkgtest-cloud: New Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/autopkgtest-cloud/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
Another example: https://github.com/systemd/systemd/pull/11802 refers to the correct amd64 log https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac /autopkgtest-bionic-upstream-systemd-ci-systemd-ci/bionic/amd64/s /systemd-upstream/20190222_161608_7fe1f@/log.gz . But https://github.com/systemd/systemd/pull/11804 refers to the exact same log. It is also the only result for 11804: https://api.github.com/repos/systemd/systemd/statuses/b427959d65ba0edff385146d38825bb169458554 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in autopkgtest-cloud: New Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/autopkgtest-cloud/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
It seems to me that the logs are internally consistent, i. e. the mentioned UPSTREAM_PULL_REQUEST in the log does match the test results. But they get sent to the wrong PR, i. e. to the wrong statuses API. E. g. https://api.github.com/repos/systemd/systemd/commits/99894b867f1293f56d181d62f5015c5a0a8adbda/status was triggered by https://github.com/systemd/systemd/pull/11682 but the referenced logs are for PR #11767. This caused the landing of a regression (that the tests would have noticed). ** Package changed: autopkgtest (Ubuntu) => autopkgtest-cloud -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in autopkgtest-cloud: New Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/autopkgtest-cloud/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1817344] Re: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results
** Also affects: systemd (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1817344 Title: Ubuntu CI that runs tests via autopkgtest for systemd on GitHub reports the wrong results Status in autopkgtest package in Ubuntu: New Status in systemd package in Ubuntu: New Bug description: I'm not sure this is the right place to report this, but apparently it's the best way to reach out to Dimitri John Ledkov (https://github.com/xnox) and Iain Lane (https://github.com/iainlane), who I believe at least know who maintains Ubuntu CI there. I'm copying the following verbatim from https://github.com/systemd/systemd/pull/11531#issuecomment-464731263 (where we are kind of discussing the issue): > The two PRs you referenced to fail in the `make check` unit tests (test-path). That's correct that test-path failed but it failed in https://github.com/systemd/systemd/pull/11685 while Ubuntu CI reported it in https://github.com/systemd/systemd/pull/11686 (that is, in https://github.com/systemd/systemd/pull/11686 in the logs for some reason UPSTREAM_PULL_REQUEST is 11685 instead of 11686 and something like that has happened often lately). In https://github.com/systemd/systemd/pull/11743#issuecomment-464637797, the issue is that Ubuntu CI showed the results for the first version of the PR (where there was a bug) and it didn't run the tests against the latest version. But in that case it's hard to tell because there is no way to figure out what exactly Ubuntu CI tested. That's why I asked @xnox to add `git describe` to https://salsa.debian.org/systemd- team/systemd/blob/master/debian/extra/checkout-upstream so that by looking at UPSTREAM_PULL_REQUEST and the output of `git describe` it would be possible to find out what Ubuntu CI reports. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1817344/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
