Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Thanks Rohit, I agree on documenting the secondary storage mount on the management server to avoid the issue (as you mentioned that's a known practice for seeding the system VM template). I think it is useful to bundle the 3 major system VM templates in the cloudstack-management package as it simplifies the upgrading process even though it increases significantly the size of the package. On that note, this PR created by Daniel: https://github.com/apache/cloudstack/pull/5602 is a great improvement for more granular control over the system VM templates to be downloaded and reducing the package size. However, I think it may introduce more complexity as administrators may want to use the auto-upgrade feature for their hypervisor, in which case we will need one package per hypervisor for the cloudstack-management package instead of simply one in which the 3 major system VM templates are included. On the original issue posted by Daniel, this PR has been tested and fixes the issue: https://github.com/apache/cloudstack/pull/5598. Regards, Nicolas Vazquez From: Rohit Yadav Sent: Thursday, October 21, 2021 1:17 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 All, Daniel, Paul, I had shared the Debian pkg changelog issue on this thread when I shared the convenience packages, it turns out Nicolas doesn't have en_* locale set as default which caused this when creating the release artifacts. The bundling of systemvmtemplates in cloudstack-management package is a feature to make ACS turnkey for users and assist in automatic upgrades/seeding. The trade-off of this feature is the increased size of the cloudstack-maangement package which may be optimised/reduced in the future. However, having look at the k8s binaries, ~1GB additional size does not sound like a big issue for most users with good Internet connections in their lab/datacenters. This further solves the issue of setting up ACS in environments where there's no/restricted Internet access (admin only needs to mirror the repos). For community convenience builds/packages it won't make sense to favour a specific hypervisor and that's why we bundle all of the three most popular hypervisors (Vmware, KVM, XenServer/XCP-ng). Since the project votes and releases source code, my understaind is we can't still bundle vmware/vim and other jars within the source code. However, I think most nonoss dependencies [2] we need with main/4.16 may be redistributable (someone from legal@ needs checking, the last thread I had started in private@ Mike helped got rid of the one know non-redist pkg/code). On the issue of 'noredist' build profile, it may not be needed as 'noredist' has been the default build profile and all the convenience packages built and hosted by the community and 3rd party servers [1] are noredist packages. Essentially you've two build profiles, one which is oss only and doesn't bundle nonoss components incl. systemvmtemplates, and noredist (we may discuss to change the name) which is all-included turnkey mvn build profile. The issue of secondary storage mounting/access on management server is a environment issue, we can document that but we implictly require that for two cases: (1) when seeding systevmtemplate for fresh installation (see docs that mention how/where to mount and seed it) and (2) for config drive feature to work (I don't remember if it's all of kvm, vmware, xenserver or only xenserver/vmware) we expect mgmt server to be able to mount secondary storage on mgmt server host. This PR https://github.com/apache/cloudstack/pull/5598, when oss packages are built (i.e. systemvmtemplates will not be bundled) should fix Daniel's issue, however, it then requires that systemvmtemplates are registered prior to the upgrade (like before). [1] https://cloudstack.apache.org/downloads.html [2] https://github.com/shapeblue/cloudstack-nonoss Regards. From: Nicolas Vazquez Sent: Thursday, October 21, 2021 04:06 To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Thanks Daniel and Paul for your feedback. The issue with changelog was related to having the Spanish locale set, will make sure it will be fixed on RC2. Will analyze the proposals for the other issues you have described. Regards, Nicolas Vazquez From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 3:52 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, I see, I am not familiar with the details of VMware SDK JARs. However, to avoid mixing things up I suggest we create an issue to discuss this topic in the future. What I pointed out is that the process of downloading system VMs templates was added in the "noredist" profile, which is, right now, used to build ACS with VMware JARs (not judging if we need or not
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
All, Daniel, Paul, I had shared the Debian pkg changelog issue on this thread when I shared the convenience packages, it turns out Nicolas doesn't have en_* locale set as default which caused this when creating the release artifacts. The bundling of systemvmtemplates in cloudstack-management package is a feature to make ACS turnkey for users and assist in automatic upgrades/seeding. The trade-off of this feature is the increased size of the cloudstack-maangement package which may be optimised/reduced in the future. However, having look at the k8s binaries, ~1GB additional size does not sound like a big issue for most users with good Internet connections in their lab/datacenters. This further solves the issue of setting up ACS in environments where there's no/restricted Internet access (admin only needs to mirror the repos). For community convenience builds/packages it won't make sense to favour a specific hypervisor and that's why we bundle all of the three most popular hypervisors (Vmware, KVM, XenServer/XCP-ng). Since the project votes and releases source code, my understaind is we can't still bundle vmware/vim and other jars within the source code. However, I think most nonoss dependencies [2] we need with main/4.16 may be redistributable (someone from legal@ needs checking, the last thread I had started in private@ Mike helped got rid of the one know non-redist pkg/code). On the issue of 'noredist' build profile, it may not be needed as 'noredist' has been the default build profile and all the convenience packages built and hosted by the community and 3rd party servers [1] are noredist packages. Essentially you've two build profiles, one which is oss only and doesn't bundle nonoss components incl. systemvmtemplates, and noredist (we may discuss to change the name) which is all-included turnkey mvn build profile. The issue of secondary storage mounting/access on management server is a environment issue, we can document that but we implictly require that for two cases: (1) when seeding systevmtemplate for fresh installation (see docs that mention how/where to mount and seed it) and (2) for config drive feature to work (I don't remember if it's all of kvm, vmware, xenserver or only xenserver/vmware) we expect mgmt server to be able to mount secondary storage on mgmt server host. This PR https://github.com/apache/cloudstack/pull/5598, when oss packages are built (i.e. systemvmtemplates will not be bundled) should fix Daniel's issue, however, it then requires that systemvmtemplates are registered prior to the upgrade (like before). [1] https://cloudstack.apache.org/downloads.html [2] https://github.com/shapeblue/cloudstack-nonoss Regards. From: Nicolas Vazquez Sent: Thursday, October 21, 2021 04:06 To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Thanks Daniel and Paul for your feedback. The issue with changelog was related to having the Spanish locale set, will make sure it will be fixed on RC2. Will analyze the proposals for the other issues you have described. Regards, Nicolas Vazquez From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 3:52 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, I see, I am not familiar with the details of VMware SDK JARs. However, to avoid mixing things up I suggest we create an issue to discuss this topic in the future. What I pointed out is that the process of downloading system VMs templates was added in the "noredist" profile, which is, right now, used to build ACS with VMware JARs (not judging if we need or not to handle this process in an isolated profile). I then suggested to separate that process into specific build profiles, so we can activate/deactivate the download process whenever we want, and also to avoid mix this process (download of system VMs templates) with VMware JAR packaging. Best regards, Daniel Salvador On 20/10/2021 15:37, Paul Angus wrote: > Thanks Daniel, > > My point is that I'm not sure that the VMware jars _need_ to be noredist, > which would simplify things for everyone. > > Kind Regards > > > Paul Angus > > -Original Message- > From: Daniel Augusto Veronezi Salvador > Sent: Wednesday, October 20, 2021 6:20 PM > To: dev@cloudstack.apache.org > Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 > > Paul, > > > Noredist packages are only necessary to VMWare. > > KVM/XenServer infrastructures don't use noredist packages. > > > Best regards, > Daniel Salvador > > On 20/10/2021 12:29, Paul Angus wrote: >> I vaguely thought that the VMware jars come under a compatible license these >> days, so don't need to be noredist anymore? >> >> >> -Original Message- >> From: Daniel Augusto Veronez
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Thanks Daniel and Paul for your feedback. The issue with changelog was related to having the Spanish locale set, will make sure it will be fixed on RC2. Will analyze the proposals for the other issues you have described. Regards, Nicolas Vazquez From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 3:52 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, I see, I am not familiar with the details of VMware SDK JARs. However, to avoid mixing things up I suggest we create an issue to discuss this topic in the future. What I pointed out is that the process of downloading system VMs templates was added in the "noredist" profile, which is, right now, used to build ACS with VMware JARs (not judging if we need or not to handle this process in an isolated profile). I then suggested to separate that process into specific build profiles, so we can activate/deactivate the download process whenever we want, and also to avoid mix this process (download of system VMs templates) with VMware JAR packaging. Best regards, Daniel Salvador On 20/10/2021 15:37, Paul Angus wrote: > Thanks Daniel, > > My point is that I'm not sure that the VMware jars _need_ to be noredist, > which would simplify things for everyone. > > Kind Regards > > > Paul Angus > > -Original Message- > From: Daniel Augusto Veronezi Salvador > Sent: Wednesday, October 20, 2021 6:20 PM > To: dev@cloudstack.apache.org > Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 > > Paul, > > > Noredist packages are only necessary to VMWare. > > KVM/XenServer infrastructures don't use noredist packages. > > > Best regards, > Daniel Salvador > > On 20/10/2021 12:29, Paul Angus wrote: >> I vaguely thought that the VMware jars come under a compatible license these >> days, so don't need to be noredist anymore? >> >> >> -Original Message- >> From: Daniel Augusto Veronezi Salvador >> Sent: Wednesday, October 20, 2021 2:53 PM >> To: dev@cloudstack.apache.org >> Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 >> >> Hi Nicolas, thanks for the work. >> >> I started testing this RC and unfortunately I ran into some problems related >> to building/upgrading ACS. >> >>From the less to the more complex: >> >> - When building ".deb", there is a problem with the Debian changelog: >> invalid weekday 'lun'; >> >> - Regarding auto upgrading "systemvms-templates", I analyzed the code and >> concluded that "systemvms-templates" will only "autoupgrade" if we use the >> profile "noredist" when building the packages, which is necessary only to >> VMWare. Some people, might not want the noredist JARs, and therefore will >> not build with this profile. With that said, there are some points regarding >> it that I think are important: >> - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, >> when using "noredist" while executing the 4.16.0.0 build. It happens due to >> "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, >> and VMware "systemvms-templates". Every time that we build the packages, it >> will download all these systemvms-templates; we are unable to decide which >> one is necessary or not. If we do not use XenServer, or other hypervisors, >> why download it? >> - If we build the packages with "noredist", when we are first running >> ACS after upgrading it to 4.16, it will try to autoupgrade the >> systemvms-templates, however, some KVM/XenServer infrastructures don't allow >> the management server to access the secondary storage directly (except for >> the first systemvm-template seed). This will cause >> "mount.nfs: access denied by server while mounting ". >> After generating this error, ACS continues the upgrade and updates the >> database; however, the new systemvm-template will not be in the secondary >> storage. If we try to deploy a system (like virtual router), it will fail >> (as expected). >> - I rolled back my environment to 4.15, did a manual seed of >> systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" >> and tried to upgrade the environment to 4.16. Then, ACS automatically tried >> to upgrade the systemvm-templates; even though the system VM template for >> 4.16 is already there, it throws some errors: >> >> 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) >> (logid:) Updat
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Paul, I see, I am not familiar with the details of VMware SDK JARs. However, to avoid mixing things up I suggest we create an issue to discuss this topic in the future. What I pointed out is that the process of downloading system VMs templates was added in the "noredist" profile, which is, right now, used to build ACS with VMware JARs (not judging if we need or not to handle this process in an isolated profile). I then suggested to separate that process into specific build profiles, so we can activate/deactivate the download process whenever we want, and also to avoid mix this process (download of system VMs templates) with VMware JAR packaging. Best regards, Daniel Salvador On 20/10/2021 15:37, Paul Angus wrote: Thanks Daniel, My point is that I'm not sure that the VMware jars _need_ to be noredist, which would simplify things for everyone. Kind Regards Paul Angus -Original Message- From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 6:20 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, Noredist packages are only necessary to VMWare. KVM/XenServer infrastructures don't use noredist packages. Best regards, Daniel Salvador On 20/10/2021 12:29, Paul Angus wrote: I vaguely thought that the VMware jars come under a compatible license these days, so don't need to be noredist anymore? -Original Message- From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 2:53 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Hi Nicolas, thanks for the work. I started testing this RC and unfortunately I ran into some problems related to building/upgrading ACS. From the less to the more complex: - When building ".deb", there is a problem with the Debian changelog: invalid weekday 'lun'; - Regarding auto upgrading "systemvms-templates", I analyzed the code and concluded that "systemvms-templates" will only "autoupgrade" if we use the profile "noredist" when building the packages, which is necessary only to VMWare. Some people, might not want the noredist JARs, and therefore will not build with this profile. With that said, there are some points regarding it that I think are important: - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, when using "noredist" while executing the 4.16.0.0 build. It happens due to "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, and VMware "systemvms-templates". Every time that we build the packages, it will download all these systemvms-templates; we are unable to decide which one is necessary or not. If we do not use XenServer, or other hypervisors, why download it? - If we build the packages with "noredist", when we are first running ACS after upgrading it to 4.16, it will try to autoupgrade the systemvms-templates, however, some KVM/XenServer infrastructures don't allow the management server to access the secondary storage directly (except for the first systemvm-template seed). This will cause "mount.nfs: access denied by server while mounting ". After generating this error, ACS continues the upgrade and updates the database; however, the new systemvm-template will not be in the secondary storage. If we try to deploy a system (like virtual router), it will fail (as expected). - I rolled back my environment to 4.15, did a manual seed of systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" and tried to upgrade the environment to 4.16. Then, ACS automatically tried to upgrade the systemvm-templates; even though the system VM template for 4.16 is already there, it throws some errors: 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating VMware System Vms 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating KVM System Vms 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Grabbing lock to register templates. 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Failed to calculate template checksum java.nio.file.NoSuchFileException: /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 at java.base/sun.nio.fs.UnixException.translateToIOException(Un
RE: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Thanks Daniel, My point is that I'm not sure that the VMware jars _need_ to be noredist, which would simplify things for everyone. Kind Regards Paul Angus -Original Message- From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 6:20 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Paul, Noredist packages are only necessary to VMWare. KVM/XenServer infrastructures don't use noredist packages. Best regards, Daniel Salvador On 20/10/2021 12:29, Paul Angus wrote: > I vaguely thought that the VMware jars come under a compatible license these > days, so don't need to be noredist anymore? > > > -Original Message- > From: Daniel Augusto Veronezi Salvador > Sent: Wednesday, October 20, 2021 2:53 PM > To: dev@cloudstack.apache.org > Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 > > Hi Nicolas, thanks for the work. > > I started testing this RC and unfortunately I ran into some problems related > to building/upgrading ACS. > > From the less to the more complex: > > - When building ".deb", there is a problem with the Debian changelog: > invalid weekday 'lun'; > > - Regarding auto upgrading "systemvms-templates", I analyzed the code and > concluded that "systemvms-templates" will only "autoupgrade" if we use the > profile "noredist" when building the packages, which is necessary only to > VMWare. Some people, might not want the noredist JARs, and therefore will not > build with this profile. With that said, there are some points regarding it > that I think are important: > - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, > when using "noredist" while executing the 4.16.0.0 build. It happens due to > "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, > and VMware "systemvms-templates". Every time that we build the packages, it > will download all these systemvms-templates; we are unable to decide which > one is necessary or not. If we do not use XenServer, or other hypervisors, > why download it? > - If we build the packages with "noredist", when we are first running > ACS after upgrading it to 4.16, it will try to autoupgrade the > systemvms-templates, however, some KVM/XenServer infrastructures don't allow > the management server to access the secondary storage directly (except for > the first systemvm-template seed). This will cause > "mount.nfs: access denied by server while mounting ". > After generating this error, ACS continues the upgrade and updates the > database; however, the new systemvm-template will not be in the secondary > storage. If we try to deploy a system (like virtual router), it will fail (as > expected). > - I rolled back my environment to 4.15, did a manual seed of > systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" > and tried to upgrade the environment to 4.16. Then, ACS automatically tried > to upgrade the systemvm-templates; even though the system VM template for > 4.16 is already there, it throws some errors: > > 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) > (logid:) Updating System Vm template IDs > 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating System Vm template IDs > 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating VMware System Vms > 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot > upgrade system Vms hypervisor is not used, so not failing upgrade > 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Updating KVM System Vms > 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Grabbing lock to register templates. > 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] > (main:null) (logid:) Failed to calculate template checksum > java.nio.file.NoSuchFileException: > /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 > at > java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) > at > java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) > at > java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) > at java.base/java.nio.file.Files.newByteChannel(Files.java:371) > at java
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Paul, Noredist packages are only necessary to VMWare. KVM/XenServer infrastructures don't use noredist packages. Best regards, Daniel Salvador On 20/10/2021 12:29, Paul Angus wrote: I vaguely thought that the VMware jars come under a compatible license these days, so don't need to be noredist anymore? -Original Message- From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 2:53 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Hi Nicolas, thanks for the work. I started testing this RC and unfortunately I ran into some problems related to building/upgrading ACS. From the less to the more complex: - When building ".deb", there is a problem with the Debian changelog: invalid weekday 'lun'; - Regarding auto upgrading "systemvms-templates", I analyzed the code and concluded that "systemvms-templates" will only "autoupgrade" if we use the profile "noredist" when building the packages, which is necessary only to VMWare. Some people, might not want the noredist JARs, and therefore will not build with this profile. With that said, there are some points regarding it that I think are important: - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, when using "noredist" while executing the 4.16.0.0 build. It happens due to "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, and VMware "systemvms-templates". Every time that we build the packages, it will download all these systemvms-templates; we are unable to decide which one is necessary or not. If we do not use XenServer, or other hypervisors, why download it? - If we build the packages with "noredist", when we are first running ACS after upgrading it to 4.16, it will try to autoupgrade the systemvms-templates, however, some KVM/XenServer infrastructures don't allow the management server to access the secondary storage directly (except for the first systemvm-template seed). This will cause "mount.nfs: access denied by server while mounting ". After generating this error, ACS continues the upgrade and updates the database; however, the new systemvm-template will not be in the secondary storage. If we try to deploy a system (like virtual router), it will fail (as expected). - I rolled back my environment to 4.15, did a manual seed of systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" and tried to upgrade the environment to 4.16. Then, ACS automatically tried to upgrade the systemvm-templates; even though the system VM template for 4.16 is already there, it throws some errors: 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating VMware System Vms 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating KVM System Vms 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Grabbing lock to register templates. 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Failed to calculate template checksum java.nio.file.NoSuchFileException: /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) at java.base/java.nio.file.Files.newByteChannel(Files.java:371) at java.base/java.nio.file.Files.newByteChannel(Files.java:422) at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) at java.base/java.nio.file.Files.newInputStream(Files.java:156) at com.cloud.upgrade.SystemVmTemplateRegistration.calculateChecksum(SystemVmTemplateRegistration.java:330) at com.cloud.upgrade.SystemVmTemplateRegistration.validateTemplates(SystemVmTemplateRegistration.java:690) at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplates(SystemVmTemplateRegistration.java:713) at com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:824) at com.cloud.utils.db.TransactionCallbac
RE: [VOTE] Apache Cloudstack 4.16.0.0 RC1
I vaguely thought that the VMware jars come under a compatible license these days, so don't need to be noredist anymore? -Original Message- From: Daniel Augusto Veronezi Salvador Sent: Wednesday, October 20, 2021 2:53 PM To: dev@cloudstack.apache.org Subject: Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Hi Nicolas, thanks for the work. I started testing this RC and unfortunately I ran into some problems related to building/upgrading ACS. From the less to the more complex: - When building ".deb", there is a problem with the Debian changelog: invalid weekday 'lun'; - Regarding auto upgrading "systemvms-templates", I analyzed the code and concluded that "systemvms-templates" will only "autoupgrade" if we use the profile "noredist" when building the packages, which is necessary only to VMWare. Some people, might not want the noredist JARs, and therefore will not build with this profile. With that said, there are some points regarding it that I think are important: - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, when using "noredist" while executing the 4.16.0.0 build. It happens due to "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, and VMware "systemvms-templates". Every time that we build the packages, it will download all these systemvms-templates; we are unable to decide which one is necessary or not. If we do not use XenServer, or other hypervisors, why download it? - If we build the packages with "noredist", when we are first running ACS after upgrading it to 4.16, it will try to autoupgrade the systemvms-templates, however, some KVM/XenServer infrastructures don't allow the management server to access the secondary storage directly (except for the first systemvm-template seed). This will cause "mount.nfs: access denied by server while mounting ". After generating this error, ACS continues the upgrade and updates the database; however, the new systemvm-template will not be in the secondary storage. If we try to deploy a system (like virtual router), it will fail (as expected). - I rolled back my environment to 4.15, did a manual seed of systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" and tried to upgrade the environment to 4.16. Then, ACS automatically tried to upgrade the systemvm-templates; even though the system VM template for 4.16 is already there, it throws some errors: 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating VMware System Vms 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating KVM System Vms 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Grabbing lock to register templates. 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Failed to calculate template checksum java.nio.file.NoSuchFileException: /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) at java.base/java.nio.file.Files.newByteChannel(Files.java:371) at java.base/java.nio.file.Files.newByteChannel(Files.java:422) at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) at java.base/java.nio.file.Files.newInputStream(Files.java:156) at com.cloud.upgrade.SystemVmTemplateRegistration.calculateChecksum(SystemVmTemplateRegistration.java:330) at com.cloud.upgrade.SystemVmTemplateRegistration.validateTemplates(SystemVmTemplateRegistration.java:690) at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplates(SystemVmTemplateRegistration.java:713) at com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:824) at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) at com.cl
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
Hi Nicolas, thanks for the work. I started testing this RC and unfortunately I ran into some problems related to building/upgrading ACS. From the less to the more complex: - When building ".deb", there is a problem with the Debian changelog: invalid weekday 'lun'; - Regarding auto upgrading "systemvms-templates", I analyzed the code and concluded that "systemvms-templates" will only "autoupgrade" if we use the profile "noredist" when building the packages, which is necessary only to VMWare. Some people, might not want the noredist JARs, and therefore will not build with this profile. With that said, there are some points regarding it that I think are important: - Package "cloudstack-management" goes from ~100Mb to more than 1.5Gb, when using "noredist" while executing the 4.16.0.0 build. It happens due to "cloudstack/engine/schema/pom.xml" automatically downloading XenServer, KVM, and VMware "systemvms-templates". Every time that we build the packages, it will download all these systemvms-templates; we are unable to decide which one is necessary or not. If we do not use XenServer, or other hypervisors, why download it? - If we build the packages with "noredist", when we are first running ACS after upgrading it to 4.16, it will try to autoupgrade the systemvms-templates, however, some KVM/XenServer infrastructures don't allow the management server to access the secondary storage directly (except for the first systemvm-template seed). This will cause "mount.nfs: access denied by server while mounting ". After generating this error, ACS continues the upgrade and updates the database; however, the new systemvm-template will not be in the secondary storage. If we try to deploy a system (like virtual router), it will fail (as expected). - I rolled back my environment to 4.15, did a manual seed of systemvmtemplate-4.16.0-kvm.qcow2.bz2, built the packages without "noredist" and tried to upgrade the environment to 4.16. Then, ACS automatically tried to upgrade the systemvm-templates; even though the system VM template for 4.16 is already there, it throws some errors: 2021-10-19 23:08:02,684 DEBUG [c.c.u.d.Upgrade41520to41600] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,904 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm template IDs 2021-10-19 23:08:02,910 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating VMware System Vms 2021-10-19 23:08:02,915 WARN [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.16.0 VMware SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade 2021-10-19 23:08:02,962 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating KVM System Vms 2021-10-19 23:08:02,966 INFO [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Grabbing lock to register templates. 2021-10-19 23:08:02,976 ERROR [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Failed to calculate template checksum java.nio.file.NoSuchFileException: /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.16.0-kvm.qcow2.bz2 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) at java.base/java.nio.file.Files.newByteChannel(Files.java:371) at java.base/java.nio.file.Files.newByteChannel(Files.java:422) at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) at java.base/java.nio.file.Files.newInputStream(Files.java:156) at com.cloud.upgrade.SystemVmTemplateRegistration.calculateChecksum(SystemVmTemplateRegistration.java:330) at com.cloud.upgrade.SystemVmTemplateRegistration.validateTemplates(SystemVmTemplateRegistration.java:690) at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplates(SystemVmTemplateRegistration.java:713) at com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:824) at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) at com.cloud.utils.db.Transaction.execute(Transaction.java:40) at com.cloud.utils.db.Transaction.execute(Transaction.java:47) at com.cloud.upgrade.SystemVmTemplateRegistration.updateSystemVmTemplates(SystemVmTemplateRegistration.java:802) at com.cloud.upgrade.dao.Upgrade41520to41600.updateSystemVmTemplates(Upgrade41520to41600.java:105) at com.cloud.upgrade.DatabaseUpgradeChecker.updateSystemVmTemplates(DatabaseUpgradeChecker.java:258) at
Re: [VOTE] Apache Cloudstack 4.16.0.0 RC1
I've built and uploaded convenience packages here: https://download.cloudstack.org/testing/41600-RC1/ Few notes: * The Debian changelog has an issue to build deb pkgs I had to replace "lun" with "Mon" and "oct" with "Oct", this is needed when not using the packaging/build-deb.sh script but when people run "dpkg-buildpackage -uc -us -b" to build deb packages * GPG keys and checksums are okay * With 4.16, registering systemvmtemplate prior to upgrade is no longer required Regards. From: Nicolas Vazquez Sent: Monday, October 18, 2021 23:55 To: dev@cloudstack.apache.org ; users Subject: [VOTE] Apache Cloudstack 4.16.0.0 RC1 Hi All, I have created a 4.16.0.0 release (RC1) with the following artefacts up for testing and a vote: Git Branch and Commit SH: https://github.com/apache/cloudstack/tree/4.16.0.0-RC20211018T1454 Commit: 506f8a8269551acb9b07b029da0f9e4fe858d150 Source release (checksums and signatures are available at the same location): https://dist.apache.org/repos/dist/dev/cloudstack/4.16.0.0/ PGP release keys (signed using 656E1BCC8CB54F84): https://dist.apache.org/repos/dist/release/cloudstack/KEYS The vote will be open until 21st October 2021, 21.00 CET (72h). For sanity in tallying the vote, can PMC members please be sure to indicate "(binding)" with their vote? [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) 4.16.0.0 System VM templates are available from here: https://download.cloudstack.org/systemvm/4.16/ Regards, Nicolas Vazquez
