Hi Andrija, Thanks for your reply. I agree that it makes sense to run smoke tests across several hypervisors to get broader validation coverage.
To answer your question, here is some additional information: 1. We already ran the matrix smoke tests with the health-check PR: https://github.com/apache/cloudstack/pull/12198. You can find the test results in the PR comments. These tests were based on the following commit: commit 8b72a16d61f669ad1141a4c2aefcc331fc90e453 Author: Wei Zhou <[email protected]> Date: Fri Aug 21 11:30:17 2026 +0200 build: fixes after forward-merge There are 4 commits after that, but they are hypervisor-agnostic changes (API, quota, DNS). In my opinion, the test results on OL8 provide sufficient coverage for those changes. There are several test failures with VMware and Xen/XCP-ng. Most of these failures appear to be caused by issues in the smoke tests or environments rather than actual bugs. There is another PR that addresses these test failures: https://github.com/apache/cloudstack/pull/13628 2. I have also kicked off several additional smoke tests covering various hypervisor combinations. I will share the results once they are completed. The tests cover Ubuntu 24, Ubuntu 26, Rocky Linux 8, Oracle Linux 9, with VMware 7.0 U3, VMware 8.0 U3, XCP-ng 8.3, XenServer 8.4, and openSUSE 15. The current test runs are: - ref-trl-12279-k-Mr8-wei-zhou — Running smoke tests - ref-trl-12280-k-Mu24-wei-zhou — Running smoke tests - ref-trl-12282-v-Mol9-wei-zhou — Running smoke tests - ref-trl-12283-x-Mol9-wei-zhou — Running smoke tests - ref-trl-12285-x-Mol9-wei-zhou — Running smoke tests - ref-trl-12286-v-Mol9-wei-zhou — Running smoke tests - ref-trl-n16-k-Ms15-wei-zhou — Running smoke tests - ref-trl-n17-k-Mu26-wei-zhou — Running smoke tests So we should have broader hypervisor coverage for RC3. I will post the results when the tests finish. Thanks again for raising the question and helping make sure we have sufficient validation coverage before the vote. Kind regards, Wei On Tue, Aug 25, 2026 at 9:44 PM Andrija Panic <[email protected]> wrote: > Hi Wei, > > First of all, thank you very much for preparing RC3 and for all the work > involved in moving the 4.23 release forward. > > I have one testing-coverage question before casting my vote. I checked the > public test trail around PR #13821 and its merge commit 158fe4f,, which is > the parent of the RC3 commit. > > I can see the package builds, the regular GitHub/Maven and simulator > checks, manual validation, and two Blue Orangutan smoke-test runs. However, > both Blue Orangutan runs appear to use the same environment: an OL8 > management server with two KVM-OL8 hosts. > > Historically, when validating a release candidate across the supported > hypervisors, we have used the Blue Orangutan test matrix covering KVM, > VMware/vSphere, and XCP-ng. I could not find corresponding VMware or XCP-ng > results for this RC, or for PR #13821. > > I may simply have missed a private or external Jenkins run. If the full > matrix has already been executed, could someone please point me to the > results? Otherwise, would it make sense to run the usual KVM, VMware, and > XCP-ng matrix against RC3 before the vote closes? > > This is absolutely not intended as criticism of the RC preparation. I just > want to make sure I correctly understand the validation coverage before > voting and I will do my testing contribution as well! > > Thanks again for all the work, Wei! > > > On Tue, Aug 25, 2026 at 5:29 PM Daan Hoogland <[email protected]> wrote: > > > +1 (binding) > > > > tested signatures and the commit reference, other than this mainly > > based on test work during the development process. > > > > On Tue, Aug 25, 2026 at 4:43 PM Wei Zhou <[email protected]> > wrote: > > > > > > Hi All, > > > > > > I have created a 4.23.0.0 release (RC3), with the following artifacts > up > > > for testing and a vote: > > > > > > Git Branch and Commit SHA: > > > https://github.com/apache/cloudstack/tree/4.23.0.0-RC20260825T1422 > > > Commit: 463f8d0294702a920e8020b62ae3b67f52ae1473 > > > > > > Source release (checksums and signatures are available at the same > > > location): > > > https://dist.apache.org/repos/dist/dev/cloudstack/4.23.0.0/ > > > > > > PGP release keys (signed using > 500B5E88394EEFEA6F2CF5EA1503DFE7C8226103): > > > https://dist.apache.org/repos/dist/release/cloudstack/KEYS > > > > > > The vote will be open for 7 days, and will close at 15:00 UTC on > > > September 1, 2026. > > > > > > For sanity in tallying the vote, can PMC members please be sure to > > indicate > > > "(binding)" with their vote? > > > > > > [ ] +1 approve > > > [ ] +0 no opinion > > > [ ] -1 disapprove (and reason why) > > > > > > For users convenience, the packages from this release candidate (RC3) > are > > > available here: > > > https://download.cloudstack.org/testing/4.23.0.0-RC3/ > > > > > > > > > Kind regards, > > > Wei > > > > > -- > > Andrija Panić >
