Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
On Mon, Mar 13, 2023 at 11:21:39AM -0700, Erich Eickmeyer wrote: > > > My point is that lowlatency shouldn't be grouped-in to these flavors, but > > > should be given higher priority and grouped-in with generic since it's > > > still used in desktop systems by default and is directly affecting the > > > testing of an official flavor of Ubuntu. This was one of the reasons we > > > had to miss testing week because we didn't even have kernel parity. > > Impossible. we run out of disk space and cannot complete the > > lowlatency builds with generic any more. Thus it is now treated > > exactly like all other derivatives. > If you ran out of disk space perhaps you should have requested more disk > space? I'd think that's something that should have been easily > accommodated by IS. This is running out of package space *during package builds*. The Launchpad build instances, for sanity of operability, are all pre-provisioned and one-size-fits-all. There is no capability in Launchpad to request more disk space for a particular package build (the information is not even available at the time the VM is instantiated what build will be done on it). This is a longstanding design decision that the Kernel Team has no power to change and the Launchpad Team has no capacity to address in any sort of near term. Your choices for Lunar are to have the lowlatency kernel flavor as a separate source package or to have it not at all. > because disk space shouldn't be a limitation in this day and age. That is not constructive or realistic. > > We do not invest additional engineering & resource time, to prioritise > > lowlatency flavour, over other flavours, which have more images and > > higher usage. > Well, thanks, that alienates an entire user base and a whole official > flavor of Ubuntu, which basically puts you at odds with the community. > I'm sure that's not your intention, but seeing as how Ubuntu is a > community-driven distribution, you probably should rethink this. All the other community flavors in Ubuntu use the generic kernel flavor. You are asking the Ubuntu Kernel Team to do something which first of all is not possible from an engineering perspective, but secondly would involve them committing to provide additional labor for a specific community flavor which, from a community governance perspective, is not in your power to require. What Dimitri is communicating is that the support the lowlatency kernel flavor receives is the same, no better no worse, as the support provided for all of the kernel flavors that actually provide the funding for the Kernel Team to do that work. If you want to have a conversation about the Ubuntu Studio team taking responsibility for a community-maintained kernel flavor, we could have that conversation. But it's not reasonable to ask that the Kernel Team make a HIGHER committment to the lowlatency kernel in the service of Ubuntu Studio, than they do to the kernels for paying customers. That said, with my Release Team hat on, I will state clearly that I am not happy with the state of kernels in the devel series; either in lunar, or in other series of the recent past. We have a pretty consistent history of kernel security updates being synced from the stable release to devel-proposed and then languishing there for months with insufficient effort to get them migrated to the release pocket, with the result that Ubuntu Developers who dogfood our development releases have the least secure systems out of the box of any Ubuntu users, from the time a new series opens to the time the Kernel Team releases kernels to the devel series based on the new upstream version that's targeted for the next release. Dimitri as one of the few core-devs on the Kernel Team is who I see primarily engaging with the Release Team to get kernels in the devel series unstuck from -proposed. But it's not on him personally to manage this, there's a systemic issue of undercommitment of resources to the devel series. And complaining about the state of where things are now in the lunar release pocket isn't going to change anything. It would be a misinvestment to work on rebasing the other kernel flavors on 6.1 now with a 6.2 upload around the corner. > > > reason (two blockers this round). To not have kernel equality here could > > > cause false positives in kernel-level testing. JACK and the audio stack, > > > in particular, are directly affected by the kernel, and what might work > > > in 6.1 might not work in 5.19 with various devices. This could cause > > > false bug reports and create a lot more problems for triage. I think it's reasonable for you to decide not to invest time in testing against a kernel that is not the final kernel for the release, where you believe that will result in wasted time due to false-positives. But 6.1 is also not the final kernel for the release, so I don't see any reason that being at parity with the generic kernel in the release pocket *right now* would make a
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
A general reminder to *everyone* with my Community Council hat on: Whether you are Canonical, an Ubuntu Flavor team member, or just a general community member, if you feel yourself starting to get hostile / aggressive in tone, step back and take a breather. There are an increasing number of times I myself see conflict between people - whether it involves Canonical employees, Ubuntu Flavor teams, or otherwise - spilling into the lists, and it seems people are starting to skirt against CoC with those cases. My advice is, on this case, Dimitri and Erich, both of you can go take a breather for a bit, and calm a little before returning to this. (I'm not a fan of seeing this level of dissent / aggressiveness between people on the public lists, and as Philipp and Mauro at the Canonical Community Team know, this is an issue that seems to be becoming systemic and we have to remind people about the CoC and how to be nice towards others, or at least be constructive without coming off as hostile). Thomas Ward Ubuntu Community Council Member On 3/13/23 14:48, Steve Langasek wrote: On Mon, Mar 13, 2023 at 06:03:00PM +, Dimitri John Ledkov wrote: We pushed 6.1 out, and migrated, on generic only, to migrate lots of packages in proposed, specifically nvidia & everything entangled with it, and thus unblock autopkgtesting of all the userspace packages which were otherwise failing on v5.19. There is no intention to port all flavours to 6.1. Again, this is one of the reasons we had do miss testing week among another it was a mistake to skip testing week. you should have tested ubuntu studio during the testing week like all the other flavours did. As there are a lot of changes in lunar, that landed and affect ubuntu studio. For example, all cloud images which use various cloud kernel flavours, based on v5.19 did participate. Can you explain who made the call to skip testing week? as to me, it seemed, it's a requirement to release a flavour. Is studio going to skip Lunar release? Testing week is not a release-team-driven activity and flavor engagement in it has no bearing on a flavor's eligibility for inclusion in a stable release. Flavors are required to hit a beta milestone and a release milestone. How they conduct their activities to ensure that these milestones are releasable is for them to decide. -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
On Mon, Mar 13, 2023 at 06:03:00PM +, Dimitri John Ledkov wrote: > > > We pushed 6.1 out, and migrated, on generic only, to migrate lots of > > > packages in proposed, specifically nvidia & everything entangled with > > > it, and thus unblock autopkgtesting of all the userspace packages > > > which were otherwise failing on v5.19. There is no intention to port > > > all flavours to 6.1. > > Again, this is one of the reasons we had do miss testing week among another > it was a mistake to skip testing week. you should have tested ubuntu > studio during the testing week like all the other flavours did. As > there are a lot of changes in lunar, that landed and affect ubuntu > studio. For example, all cloud images which use various cloud kernel > flavours, based on v5.19 did participate. > Can you explain who made the call to skip testing week? as to me, it > seemed, it's a requirement to release a flavour. Is studio going to > skip Lunar release? Testing week is not a release-team-driven activity and flavor engagement in it has no bearing on a flavor's eligibility for inclusion in a stable release. Flavors are required to hit a beta milestone and a release milestone. How they conduct their activities to ensure that these milestones are releasable is for them to decide. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
On Mon, 13 Mar 2023 at 17:54, Erich Eickmeyer wrote: > > On Monday, March 13, 2023 10:29:50 AM PDT Dimitri John Ledkov wrote: > > Ideally all flavours would be at 6.2 already, but due to various > > reasons they are not. > > This is understandable and perfectly reasonable. > > > This is not unique to lowlatency flavour, and applies to kvm, azure, > > raspi, and many more kernel flavours all of which are still on v5.19 > > in Lunar. > > My point is that lowlatency shouldn't be grouped-in to these flavors, but > should be given higher priority and grouped-in with generic since it's still > used in desktop systems by default and is directly affecting the testing of an > official flavor of Ubuntu. This was one of the reasons we had to miss testing > week because we didn't even have kernel parity. > Impossible. we run out of disk space and cannot complete the lowlatency builds with generic any more. Thus it is now treated exactly like all other derivatives. We do not invest additional engineering & resource time, to prioritise lowlatency flavour, over other flavours, which have more images and higher usage. > > We pushed 6.1 out, and migrated, on generic only, to migrate lots of > > packages in proposed, specifically nvidia & everything entangled with > > it, and thus unblock autopkgtesting of all the userspace packages > > which were otherwise failing on v5.19. There is no intention to port > > all flavours to 6.1. > > Again, this is one of the reasons we had do miss testing week among another it was a mistake to skip testing week. you should have tested ubuntu studio during the testing week like all the other flavours did. As there are a lot of changes in lunar, that landed and affect ubuntu studio. For example, all cloud images which use various cloud kernel flavours, based on v5.19 did participate. Can you explain who made the call to skip testing week? as to me, it seemed, it's a requirement to release a flavour. Is studio going to skip Lunar release? > reason (two blockers this round). To not have kernel equality here could cause > false positives in kernel-level testing. JACK and the audio stack, in > particular, are directly affected by the kernel, and what might work in 6.1 > might not work in 5.19 with various devices. This could cause false bug > reports and create a lot more problems for triage. > > > in Lunar, no further 6.1 builds will be done for any kernel flavour > > for the time being. And v6.2 landing, across all flavours, is in > > progress. > > Understandable. I'm just trying to prevent the problem at hand in the future, > hence requesting that the decision to split the lowlatency into a lesser > flavor > be reverted and have it built and treated as if it were the generic kernel > since it is installed by default in an official flavor of Ubuntu on desktop > systems. It is just clear to me that it truly does not get equal treatment, > which confirms my fears, which is why I want the decision that was made > reverted so that proper testing can proceed as it was before this change. It was not a choice to split it, but necessity. We had to split lowlatency into a separate build, as generic & lowlatency from a single builder was unable to complete anymore due to build-time, disk usage, and upload time. It's either split builds, or no builds at all. Thus a revert, would just cause generic & lowlatency fail to build from source, always. -- okurrr, Dimitri -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
On Monday, March 13, 2023 11:03:00 AM PDT Dimitri John Ledkov wrote: > On Mon, 13 Mar 2023 at 17:54, Erich Eickmeyer wrote: > > On Monday, March 13, 2023 10:29:50 AM PDT Dimitri John Ledkov wrote: > > > Ideally all flavours would be at 6.2 already, but due to various > > > reasons they are not. > > > > This is understandable and perfectly reasonable. > > > > > This is not unique to lowlatency flavour, and applies to kvm, azure, > > > raspi, and many more kernel flavours all of which are still on v5.19 > > > in Lunar. > > > > My point is that lowlatency shouldn't be grouped-in to these flavors, but > > should be given higher priority and grouped-in with generic since it's > > still used in desktop systems by default and is directly affecting the > > testing of an official flavor of Ubuntu. This was one of the reasons we > > had to miss testing week because we didn't even have kernel parity. > > Impossible. we run out of disk space and cannot complete the > lowlatency builds with generic any more. Thus it is now treated > exactly like all other derivatives. If you ran out of disk space perhaps you should have requested more disk space? I'd think that's something that should have been easily accommodated by IS. If it's a matter of ISO image space, then you need to be using iso_level=3 per the change I collaborated with to ubuntu-cdimage prior to 22.04's release which future-proofed all Ubuntu .iso images beyond the 4GB level. Either way, it sounds like you're either not communicating your needs to the right team(s) or doing something wrong, because disk space shouldn't be a limitation in this day and age. > We do not invest additional engineering & resource time, to prioritise > lowlatency flavour, over other flavours, which have more images and > higher usage. Well, thanks, that alienates an entire user base and a whole official flavor of Ubuntu, which basically puts you at odds with the community. I'm sure that's not your intention, but seeing as how Ubuntu is a community-driven distribution, you probably should rethink this. > > > We pushed 6.1 out, and migrated, on generic only, to migrate lots of > > > packages in proposed, specifically nvidia & everything entangled with > > > it, and thus unblock autopkgtesting of all the userspace packages > > > which were otherwise failing on v5.19. There is no intention to port > > > all flavours to 6.1. > > > > Again, this is one of the reasons we had do miss testing week among > > another > > it was a mistake to skip testing week. you should have tested ubuntu > studio during the testing week like all the other flavours did. As > there are a lot of changes in lunar, that landed and affect ubuntu > studio. For example, all cloud images which use various cloud kernel > flavours, based on v5.19 did participate. > > Can you explain who made the call to skip testing week? as to me, it > seemed, it's a requirement to release a flavour. Is studio going to > skip Lunar release? That was my call as Ubuntu Studio flavor lead to skip testing week. I found out two days before the start of testing week (Feature Freeze) that a major component of our test case (studio-controls) was not going to work properly with the pipewire setup due to health issues on the part of the lead developer, so to have users use the ISO tracker without a modified test case, which needs to happen anyhow, would lead to poor test results. Basically, we were not ready for testing week. As far as a requirement to release a flavor? We usually have had testing weeks after beta release. Honestly, this is between us and the release team. > > reason (two blockers this round). To not have kernel equality here could > > cause false positives in kernel-level testing. JACK and the audio stack, > > in particular, are directly affected by the kernel, and what might work > > in 6.1 might not work in 5.19 with various devices. This could cause > > false bug reports and create a lot more problems for triage. > > > > > in Lunar, no further 6.1 builds will be done for any kernel flavour > > > for the time being. And v6.2 landing, across all flavours, is in > > > progress. > > > > Understandable. I'm just trying to prevent the problem at hand in the > > future, hence requesting that the decision to split the lowlatency into a > > lesser flavor be reverted and have it built and treated as if it were the > > generic kernel since it is installed by default in an official flavor of > > Ubuntu on desktop systems. It is just clear to me that it truly does not > > get equal treatment, which confirms my fears, which is why I want the > > decision that was made reverted so that proper testing can proceed as it > > was before this change. > It was not a choice to split it, but necessity. > We had to split lowlatency into a separate build, as generic & > lowlatency from a single builder was unable to complete anymore due to > build-time, disk usage, and upload time. > It's either split builds, or
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
On Monday, March 13, 2023 10:29:50 AM PDT Dimitri John Ledkov wrote: > Ideally all flavours would be at 6.2 already, but due to various > reasons they are not. This is understandable and perfectly reasonable. > This is not unique to lowlatency flavour, and applies to kvm, azure, > raspi, and many more kernel flavours all of which are still on v5.19 > in Lunar. My point is that lowlatency shouldn't be grouped-in to these flavors, but should be given higher priority and grouped-in with generic since it's still used in desktop systems by default and is directly affecting the testing of an official flavor of Ubuntu. This was one of the reasons we had to miss testing week because we didn't even have kernel parity. > We pushed 6.1 out, and migrated, on generic only, to migrate lots of > packages in proposed, specifically nvidia & everything entangled with > it, and thus unblock autopkgtesting of all the userspace packages > which were otherwise failing on v5.19. There is no intention to port > all flavours to 6.1. Again, this is one of the reasons we had do miss testing week among another reason (two blockers this round). To not have kernel equality here could cause false positives in kernel-level testing. JACK and the audio stack, in particular, are directly affected by the kernel, and what might work in 6.1 might not work in 5.19 with various devices. This could cause false bug reports and create a lot more problems for triage. > in Lunar, no further 6.1 builds will be done for any kernel flavour > for the time being. And v6.2 landing, across all flavours, is in > progress. Understandable. I'm just trying to prevent the problem at hand in the future, hence requesting that the decision to split the lowlatency into a lesser flavor be reverted and have it built and treated as if it were the generic kernel since it is installed by default in an official flavor of Ubuntu on desktop systems. It is just clear to me that it truly does not get equal treatment, which confirms my fears, which is why I want the decision that was made reverted so that proper testing can proceed as it was before this change. -- Erich Eickmeyer Project Leader - Ubuntu Studio Technical Lead - Edubuntu Revival signature.asc Description: This is a digitally signed message part. -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
Re: [ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
Ideally all flavours would be at 6.2 already, but due to various reasons they are not. This is not unique to lowlatency flavour, and applies to kvm, azure, raspi, and many more kernel flavours all of which are still on v5.19 in Lunar. We pushed 6.1 out, and migrated, on generic only, to migrate lots of packages in proposed, specifically nvidia & everything entangled with it, and thus unblock autopkgtesting of all the userspace packages which were otherwise failing on v5.19. There is no intention to port all flavours to 6.1. in Lunar, no further 6.1 builds will be done for any kernel flavour for the time being. And v6.2 landing, across all flavours, is in progress. On Mon, 13 Mar 2023 at 17:02, Erich Eickmeyer wrote: > > Hi everyone, > > I'm bringing this up as a matter of concern for the Ubuntu Studio daily > images. I believe sometime after the release of 22.04, the lowlatency kernel > was split from the build of the generic kernel. From what I understand it was > to make the build process easier and shorter, but I could be misremembering. > While that would make sense were the lowlatency kernel *not* being used on > desktop systems by default, the reality is that it is being used as such. > > My main concerns at the time was that the lowlatency kernel would not have > testing or quality control parity with the generic kernel. I have been assured > multiple times that the quality controls have not changed, so this is not a > quality control concern, so please do not misunderstand my concern here. > > What I'm looking at now is a situation where the daily builds of lunar for > Ubuntu Studio are not at kernel parity with other flavors simply because it > builds using the lowlatency kernel. For instance, I can look at any other > flavor and see the 6.1 generic kernel whereas Ubuntu Studio is still sitting > at > 5.19 lowlatency. This means my testers aren't even testing on something > *anywhere close* to what the final kernel will be, which, albiet, is expected > to be 6.2. However, to be unable to test in parity with the other flavors is > highly disappointing and is a huge setback. > > So, please, rather than treating the lowlatency kernel as a secondary, > derivative kernel, I would like to kindly request that it be brought back to > an equal kernel and built with the generic kernel since it's still being > installed in desktop systems by default so that it can be tested on live > images. That way, we could test it correctly and Ubuntu Studio isn't left > behind as it is currently. > > Thank you in advance, > Erich > -- > Erich Eickmeyer > Project Leader - Ubuntu Studio > Technical Lead - Edubuntu-- > ubuntu-devel mailing list > ubuntu-de...@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- okurrr, Dimitri -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
[ubuntu-studio-devel] Lowlatency Kernel is behind in Ubuntu Studio
Hi everyone, I'm bringing this up as a matter of concern for the Ubuntu Studio daily images. I believe sometime after the release of 22.04, the lowlatency kernel was split from the build of the generic kernel. From what I understand it was to make the build process easier and shorter, but I could be misremembering. While that would make sense were the lowlatency kernel *not* being used on desktop systems by default, the reality is that it is being used as such. My main concerns at the time was that the lowlatency kernel would not have testing or quality control parity with the generic kernel. I have been assured multiple times that the quality controls have not changed, so this is not a quality control concern, so please do not misunderstand my concern here. What I'm looking at now is a situation where the daily builds of lunar for Ubuntu Studio are not at kernel parity with other flavors simply because it builds using the lowlatency kernel. For instance, I can look at any other flavor and see the 6.1 generic kernel whereas Ubuntu Studio is still sitting at 5.19 lowlatency. This means my testers aren't even testing on something *anywhere close* to what the final kernel will be, which, albiet, is expected to be 6.2. However, to be unable to test in parity with the other flavors is highly disappointing and is a huge setback. So, please, rather than treating the lowlatency kernel as a secondary, derivative kernel, I would like to kindly request that it be brought back to an equal kernel and built with the generic kernel since it's still being installed in desktop systems by default so that it can be tested on live images. That way, we could test it correctly and Ubuntu Studio isn't left behind as it is currently. Thank you in advance, Erich -- Erich Eickmeyer Project Leader - Ubuntu Studio Technical Lead - Edubuntu signature.asc Description: This is a digitally signed message part. -- ubuntu-studio-devel mailing list ubuntu-studio-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-studio-devel
