Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
On Mittwoch, 7. Februar 2024 19:48:27 CET Friedrich W. H. Kossebau wrote: > Am Sonntag, 4. Februar 2024, 19:22:28 CET schrieb Ben Cooksley: > > On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau > > > > wrote: > > > Hi, > > > > > > ((cc:kde-frameworks-devel for heads-up, replies please only to > > > kde-core-deve)) > > > > > > I hit the problem that when working on a repo which would like to use > > > latest > > > KF development state to integrate some new KF API just added in > > > cooperation > > > with that very repo wanting to use it, I cannot do so when someone had > > > added a > > > flatpak job on CI to that repo. > > > > > > Because with such flatpak jobs it seems they are limiting the available > > > KF > > > version not to the current latest one, as expected for continuous > > > integration, > > > > > > but some older (anywhere documented?) snapshot: > > > "runtime-version": "6.6-kf6preview", > > > > Please see https://invent.kde.org/packaging/flatpak-kde-runtime/-/tree/kf6 > > for what is in the KF6 preview. > > Thanks. So documented by implementation :) > > > > What can be done here to reestablish the old immediate continuous > > > integration > > > workflow? Where new APIs (also from KF) are instantly available? > > > > With Flatpak new APIs were never instantly available - there has always > > been a delay as the Flatpak Runtime uses the most recent released version > > of our software. > > I guess this was shadowed by feature development of KF having stalled during > the Qt5/KF5->Qt6/KG6 porting phase as well as Qt6-based flatpaks jobs only > activated recently. > > > > Right now this is a new extra burden which makes working on new features > > > with > > > KF and apps more complicated. Thus less interesting, and one/I would > > > rather > > > duplicate code in apps to get things done. > > > > > > Blocking latest KF API from usage also means less testing of that before > > > the > > > initial release. > > > > > > > > > Besides all the resource costs to create flatpaks on master builds by > > > default > > > every time, when those are usually not used by anyone anyway. > > > > Those applications that have a hard dependency on features being added to > > Frameworks are not good candidates for making use of our Continuous > > Delivery systems i'm afraid. > > Both Flatpak and Craft based (Linux Appimages, Android APKs, Windows and > > macOS) CD jobs are best optimised for those applications that rely on the > > stable Frameworks releases. > > > > There are ways (in .craft.ini) to make newer Frameworks available, but > > that > > requires that the system recompiles that Framework each time you trigger a > > build and is therefore not recommended. > > > > Allowing those systems to use the "latest" artifacts of Frameworks would > > be > > a non-trivial exercise. > > So effectively this means: > * KF - no CI on new API with non-KF repos, only KF-internal CI > * KF - no CD, only released versions > > Which makes KF a second class product, with lesser testing :( > And less interesting to contribute to, given it gets worse CI/CD care. CI builds can (as they always could on Gitlab and currently mostly do) use latest KF master. CD builds can't (and never could, binary factory had the same limitation). Nothing is getting worse here, the only thing you cannot have is unconditionally using latest KF and expect CD builds for that based on cached runtime/platform parts. The most common way of working for apps seems to be supporting at least the last released version of KF, if needed with version ifdefs to already integrate newer code. In most cases that is very limited effort, and as soon as more than one contributor is involved that is usually highly desired anyway to not randomly force full rebuilds on others. There are cases where this is not feasible, e.g. in case of very tight coupling or QML code that cannot easily be ifdef'ed. In that case you either wait for the next release, or you can only have CD builds in the release branch I'd say. Using expensive "full-stack" CD builds for that would need a very good justification given the cost IMHO. I can't speak for KDE Games obviously, but for the stuff I am involved in I most definitely wouldn't want to miss CD builds anymore. There's a bunch of people daily-driving Itinerary from those for example, and that is such a big help for QA. Bug reports are practically always still relevant, the turnaround time for feedback is days or even hours rather than weeks or months, and a whole bunch of issues gets caught before even hitting the releases. On top of that the Flatpak build provides an extra safety net there for finding incompatibilities with two major external dependencies (poppler and zxing), by using the latest versions of those. One thing I do agree on though is that there is no point in running jobs (CI nor CD) that really /nobody/ cares about anymore, and the occasional cleanup there is probably
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
Am Sonntag, 4. Februar 2024, 19:22:28 CET schrieb Ben Cooksley: > On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau > > wrote: > > Hi, > > > > ((cc:kde-frameworks-devel for heads-up, replies please only to > > kde-core-deve)) > > > > I hit the problem that when working on a repo which would like to use > > latest > > KF development state to integrate some new KF API just added in > > cooperation > > with that very repo wanting to use it, I cannot do so when someone had > > added a > > flatpak job on CI to that repo. > > > > Because with such flatpak jobs it seems they are limiting the available KF > > version not to the current latest one, as expected for continuous > > integration, > > > > but some older (anywhere documented?) snapshot: > > "runtime-version": "6.6-kf6preview", > > Please see https://invent.kde.org/packaging/flatpak-kde-runtime/-/tree/kf6 > for what is in the KF6 preview. Thanks. So documented by implementation :) > > What can be done here to reestablish the old immediate continuous > > integration > > workflow? Where new APIs (also from KF) are instantly available? > > With Flatpak new APIs were never instantly available - there has always > been a delay as the Flatpak Runtime uses the most recent released version > of our software. I guess this was shadowed by feature development of KF having stalled during the Qt5/KF5->Qt6/KG6 porting phase as well as Qt6-based flatpaks jobs only activated recently. > > Right now this is a new extra burden which makes working on new features > > with > > KF and apps more complicated. Thus less interesting, and one/I would > > rather > > duplicate code in apps to get things done. > > > > Blocking latest KF API from usage also means less testing of that before > > the > > initial release. > > > > > > Besides all the resource costs to create flatpaks on master builds by > > default > > every time, when those are usually not used by anyone anyway. > > Those applications that have a hard dependency on features being added to > Frameworks are not good candidates for making use of our Continuous > Delivery systems i'm afraid. > Both Flatpak and Craft based (Linux Appimages, Android APKs, Windows and > macOS) CD jobs are best optimised for those applications that rely on the > stable Frameworks releases. > > There are ways (in .craft.ini) to make newer Frameworks available, but that > requires that the system recompiles that Framework each time you trigger a > build and is therefore not recommended. > > Allowing those systems to use the "latest" artifacts of Frameworks would be > a non-trivial exercise. So effectively this means: * KF - no CI on new API with non-KF repos, only KF-internal CI * KF - no CD, only released versions Which makes KF a second class product, with lesser testing :( And less interesting to contribute to, given it gets worse CI/CD care. One challenge I face myself here are community maintained products, like KDE games. I have had some pleasure in spending some time recently (well, a lot actually) on them to make sure they all properly move to Q6/KF6 for Gear 24.02. And would assess myself success here. Now others have on their own enabled flatpak builds for all the repos on the master branch. Because they "could" and it worked at that moment in time with the dependencies. I have some experimental work for the games collected during the Qt6/KF6 port work which still needs some brush up & further tuning. It would require changes to KF, libkdegames & then all the games. The current setup which seems to defaults to "binary builds everywhere" makes completing that work now something which seems to go against current standards and makes me feel uncomfortable, as if I do something wrong/against the plans, as I would have to disable all the flatpak builds. (Besides having to do all the care/extra work here for an ecosystem I have no business with). For me this is an extra hurdle to work in KDE. And things are made worse for me given I am not convinced by those platforms (besides AppImage perhaps) which now are ruling contribution/development here. I understand people fancy easy deployment on their favourite platforms, so do I. There was a time when also Debian package info was in some KDE projects IIRC. But please think about those who are not into your respective platform, do not force them into (supporting) yours. > > So, how to solve those problems? Did I miss something? > > Could flatpak builds on master branches be made on-demand rather? Cheers Friedrich
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
Am Montag, 5. Februar 2024, 00:15:00 CET schrieb Julius Künzel: > > Besides all the resource costs to create flatpaks on master builds by > > default > every time, when those are usually not used by anyone anyway. > > It is important to mention that the pipelines on master usually publish to > the nightly repos on cdn.kde.org/flatpak I guess you were not aware of that > otherwise I wonder what makes you so confident to know nobody uses it? That was said with the context of at least some people (including me) doing development bursts when there is a time window, pushing multiple times in a row on the same evening (to have each set of change CI-tested on its own, like for platforms one does not have access to). So all the flatpaks in those intermediate builds serve no usage purpose, for developers not into the flatpak ecosystem. Only the last might, if any people actually have a reson to use nightly builds. And was said thinking of applications were there might be no-one interested/ motivated to actually run nightly-builds version for the normal life needs, where you want to rely on stable things to do your actual things in life those are just tools for. And nightly flatpaks might be rather interesting for people test-driving bug fixes occasionally. Or testing translations, though scripty does not trigger flatpak builds, does it? For that some on-demand might be more resource-friendly, instead of defaulting to "always build and force all contributors into supporting it", as I experience that here. Cheers Friedrich
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
Am Montag, 5. Februar 2024, 10:58:07 CET schrieb Ben Cooksley: > On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau > > wrote: > > So, how to solve those problems? Did I miss something? > > Could flatpak builds on master branches be made on-demand rather? > > For the record, my rebuild of the 6.6-kf6preview Flatpak Runtime/SDK was > successful, and the failure that kicked this off in KUserFeedback has now > been fixed. > https://invent.kde.org/frameworks/kuserfeedback/-/jobs/1561435 Though what kicked off this very thread was making libkdegames & Co. use current KF development version. And seeing that what used to work without issues in the last years now fail and being impeded, due to the new flatpak support :( Not a flatpak user, so even more sad being side-effected here. Cheers Friedrich
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau wrote: > Hi, > > ((cc:kde-frameworks-devel for heads-up, replies please only to > kde-core-deve)) > > I hit the problem that when working on a repo which would like to use > latest > KF development state to integrate some new KF API just added in > cooperation > with that very repo wanting to use it, I cannot do so when someone had > added a > flatpak job on CI to that repo. > > Because with such flatpak jobs it seems they are limiting the available KF > version not to the current latest one, as expected for continuous > integration, > but some older (anywhere documented?) snapshot: > > "runtime-version": "6.6-kf6preview", > > What can be done here to reestablish the old immediate continuous > integration > workflow? Where new APIs (also from KF) are instantly available? > > Right now this is a new extra burden which makes working on new features > with > KF and apps more complicated. Thus less interesting, and one/I would > rather > duplicate code in apps to get things done. > > Blocking latest KF API from usage also means less testing of that before > the > initial release. > > Besides all the resource costs to create flatpaks on master builds by > default > every time, when those are usually not used by anyone anyway. > > So, how to solve those problems? Did I miss something? > Could flatpak builds on master branches be made on-demand rather? > For the record, my rebuild of the 6.6-kf6preview Flatpak Runtime/SDK was successful, and the failure that kicked this off in KUserFeedback has now been fixed. https://invent.kde.org/frameworks/kuserfeedback/-/jobs/1561435 > Cheers > Friedrich > > > Cheers, Ben
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
> Besides all the resource costs to create flatpaks on master builds by default every time, when those are usually not used by anyone anyway. It is important to mention that the pipelines on master usually publish to the nightly repos on cdn.kde.org/flatpak I guess you were not aware of that otherwise I wonder what makes you so confident to know nobody uses it? Cheers, Julius 04.02.2024 19:23:06 Ben Cooksley : > On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau > wrote: >> Hi, >> >> ((cc:kde-frameworks-devel for heads-up, replies please only to >> kde-core-deve)) >> >> I hit the problem that when working on a repo which would like to use latest >> KF development state to integrate some new KF API just added in cooperation >> with that very repo wanting to use it, I cannot do so when someone had added >> a >> flatpak job on CI to that repo. >> >> Because with such flatpak jobs it seems they are limiting the available KF >> version not to the current latest one, as expected for continuous >> integration, >> but some older (anywhere documented?) snapshot: >> >> "runtime-version": "6.6-kf6preview", > > Please see https://invent.kde.org/packaging/flatpak-kde-runtime/-/tree/kf6 > for what is in the KF6 preview. > >> >> What can be done here to reestablish the old immediate continuous integration >> workflow? Where new APIs (also from KF) are instantly available? > > With Flatpak new APIs were never instantly available - there has always been > a delay as the Flatpak Runtime uses the most recent released version of our > software. > >> >> Right now this is a new extra burden which makes working on new features with >> KF and apps more complicated. Thus less interesting, and one/I would rather >> duplicate code in apps to get things done. >> >> Blocking latest KF API from usage also means less testing of that before the >> initial release. >> >> Besides all the resource costs to create flatpaks on master builds by default >> every time, when those are usually not used by anyone anyway. > > Those applications that have a hard dependency on features being added to > Frameworks are not good candidates for making use of our Continuous Delivery > systems i'm afraid. > Both Flatpak and Craft based (Linux Appimages, Android APKs, Windows and > macOS) CD jobs are best optimised for those applications that rely on the > stable Frameworks releases. > > There are ways (in .craft.ini) to make newer Frameworks available, but that > requires that the system recompiles that Framework each time you trigger a > build and is therefore not recommended. > > Allowing those systems to use the "latest" artifacts of Frameworks would be a > non-trivial exercise. > >> >> So, how to solve those problems? Did I miss something? >> Could flatpak builds on master branches be made on-demand rather? >> >> Cheers >> Friedrich > > Cheers, > Ben
Re: Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
On Mon, Feb 5, 2024 at 4:28 AM Friedrich W. H. Kossebau wrote: > Hi, > > ((cc:kde-frameworks-devel for heads-up, replies please only to > kde-core-deve)) > > I hit the problem that when working on a repo which would like to use > latest > KF development state to integrate some new KF API just added in > cooperation > with that very repo wanting to use it, I cannot do so when someone had > added a > flatpak job on CI to that repo. > > Because with such flatpak jobs it seems they are limiting the available KF > version not to the current latest one, as expected for continuous > integration, > but some older (anywhere documented?) snapshot: > > "runtime-version": "6.6-kf6preview", > Please see https://invent.kde.org/packaging/flatpak-kde-runtime/-/tree/kf6 for what is in the KF6 preview. > > What can be done here to reestablish the old immediate continuous > integration > workflow? Where new APIs (also from KF) are instantly available? > With Flatpak new APIs were never instantly available - there has always been a delay as the Flatpak Runtime uses the most recent released version of our software. > > Right now this is a new extra burden which makes working on new features > with > KF and apps more complicated. Thus less interesting, and one/I would > rather > duplicate code in apps to get things done. > > Blocking latest KF API from usage also means less testing of that before > the > initial release. > Besides all the resource costs to create flatpaks on master builds by > default > every time, when those are usually not used by anyone anyway. > Those applications that have a hard dependency on features being added to Frameworks are not good candidates for making use of our Continuous Delivery systems i'm afraid. Both Flatpak and Craft based (Linux Appimages, Android APKs, Windows and macOS) CD jobs are best optimised for those applications that rely on the stable Frameworks releases. There are ways (in .craft.ini) to make newer Frameworks available, but that requires that the system recompiles that Framework each time you trigger a build and is therefore not recommended. Allowing those systems to use the "latest" artifacts of Frameworks would be a non-trivial exercise. > So, how to solve those problems? Did I miss something? > Could flatpak builds on master branches be made on-demand rather? > Cheers > Friedrich > Cheers, Ben
Flatpak jobs on KDE CI vs. continuous integration on main/master/devel branches
Hi, ((cc:kde-frameworks-devel for heads-up, replies please only to kde-core-deve)) I hit the problem that when working on a repo which would like to use latest KF development state to integrate some new KF API just added in cooperation with that very repo wanting to use it, I cannot do so when someone had added a flatpak job on CI to that repo. Because with such flatpak jobs it seems they are limiting the available KF version not to the current latest one, as expected for continuous integration, but some older (anywhere documented?) snapshot: "runtime-version": "6.6-kf6preview", What can be done here to reestablish the old immediate continuous integration workflow? Where new APIs (also from KF) are instantly available? Right now this is a new extra burden which makes working on new features with KF and apps more complicated. Thus less interesting, and one/I would rather duplicate code in apps to get things done. Blocking latest KF API from usage also means less testing of that before the initial release. Besides all the resource costs to create flatpaks on master builds by default every time, when those are usually not used by anyone anyway. So, how to solve those problems? Did I miss something? Could flatpak builds on master branches be made on-demand rather? Cheers Friedrich
