[EPEL-devel] Re: EPEL-ANNOUNCE Incompatible Upgrade of singularity-ce in EPEL 7 / 8 / 9
The singularity-ce incompatible upgrade has now been pushed to stable. This is the final announcement prescribed by the EPEL Incompatible Upgrades Policy: https://docs.fedoraproject.org/en-US/epel/epel-policy-incompatible-upgrades/ Cheers, DT On 9 Feb 2024, at 10:45, David Trudgian wrote: > > After approval in the last EPEL meeting [1], I have submitted an incompatible > upgrade of singularity-ce to testing for EPEL 7, 8 & 9. > > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-cbd86d2020 > > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-f299fbc570 > > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-05e20edbbf > > These updates upgrade singularity-ce from v3.11.5 to v4.1.1. > > The following incompatibilities should be noted by users: > > 1) Some functionality previously provided by the `singularity remote` command > is split into new `singularity registry` and `singularity keyserver` commands. > 2) The `singularity remote add` command now sets a newly added remote as the > default (unless suppressed). Previously the user had to set the default > remote manually. > 3) The deprecated and unmaintained `—vm` flag to start singularity inside a > Virtual Machine has been removed. > > 4) Bind mounts are now performed in the order in which they are specified. > Previously, image based bind mounts were performed before others. > 5) Current working directory on the host is now created in the container, > restoring a behaviour from Singularity <3.6.0 unless suppressed. > 6) If the current directory paths on the container and host contains symlinks > to different locations, the current working directory is not mounted. > > Changes 1,2,3 are expected to have minimal impact, and 4,5,6 are likely to be > considered bug fixes by many users. > > No changes are required to existing configuration files for this update. > > A complete description of changes and additions between v3.11 and v4.1.1 is > available in the upstream documentation at https://sylabs.io/docs/ > > In the absence of any issues, the updates will be pushed to stable after a > one week testing period has passed, with a follow-up notification here. > > Cheers, > > David Trudgian > > > [1] https://pagure.io/epel/issue/265#comment-894790 > -- > ___ > epel-announce mailing list -- epel-annou...@lists.fedoraproject.org > To unsubscribe send an email to epel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/epel-annou...@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
[EPEL-devel] Incompatible Upgrade of singularity-ce in EPEL 7 / 8 / 9
After approval in the last EPEL meeting [1], I have submitted an incompatible upgrade of singularity-ce to testing for EPEL 7, 8 & 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-cbd86d2020 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-f299fbc570 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-05e20edbbf These updates upgrade singularity-ce from v3.11.5 to v4.1.1. The following incompatibilities should be noted by users: 1) Some functionality previously provided by the `singularity remote` command is split into new `singularity registry` and `singularity keyserver` commands. 2) The `singularity remote add` command now sets a newly added remote as the default (unless suppressed). Previously the user had to set the default remote manually. 3) The deprecated and unmaintained `—vm` flag to start singularity inside a Virtual Machine has been removed. 4) Bind mounts are now performed in the order in which they are specified. Previously, image based bind mounts were performed before others. 5) Current working directory on the host is now created in the container, restoring a behaviour from Singularity <3.6.0 unless suppressed. 6) If the current directory paths on the container and host contains symlinks to different locations, the current working directory is not mounted. Changes 1,2,3 are expected to have minimal impact, and 4,5,6 are likely to be considered bug fixes by many users. No changes are required to existing configuration files for this update. A complete description of changes and additions between v3.11 and v4.1.1 is available in the upstream documentation at https://sylabs.io/docs/ In the absence of any issues, the updates will be pushed to stable after a one week testing period has passed, with a follow-up notification here. Cheers, David Trudgian [1] https://pagure.io/epel/issue/265#comment-894790 -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
[EPEL-devel] Incompatible Upgrade Request - singularity-ce
Dear all, Following advice from Neal elsewhere on this list [1], I’m requesting that the singularity-ce EPEL packages may be updated to 4.1.0 following the incompatible upgrade procedure. The justification for the upgrade is that 3.x singularity-ce is no longer maintained upstream. Note that because singularity-ce necessarily bundles many Go dependencies specified in upstream go.mod/go.sum, lack of maintenance can quickly lead to inherited security issues. Upstream singularity-ce (https://github.com/sylabs/singularity) maintenance is limited to the latest x.y minor version, currently 4.1. The upstream project has committed more formally to semantic versioning conventions since 4.0.0, so any 4.y.z minor (y) and patch (z) version updates are expected to be compatible upgrades for EPEL purposes. At present, singularity-ce in EPEL 7/8/9 has been held at 3.11.5 due to incompatible changes. Incompatibilities from 3.11.5 -> 4.1.0 are as follows: (a) The CLI has split some functionality previously provided by the `singularity remote` command into new `singularity registry` and `singularity keyserver` commands. (b) The `singularity remote add` command now sets a newly added remote as the default (unless suppressed). Previously the user had to set the default remote manually. (c) The `—vm` flag to start `singularity` inside a Virtual Machine has been removed. This was not intended to be user facing, but was used by a separate and proprietary Singularity Desktop project that has been retired for some time. `—vm` was unmaintained, and I don’t believe there is any practical use outside of this context. (d) Bind mounts are now performed in the order in which they are specified. Previously, image based bind mounts were performed before others. (e) Current working directory on the host is now created in the container, restoring a behaviour from Singularity <3.6.0 unless suppressed. (f) If the current directory paths on the container and host contains symlinks to different locations, the current working directory is not mounted. The above are expected to have a minor impact on users. Arguably (d)/(e)/(f) are bug fixes as they address issues reported by users as unexpected behaviour. Changes (e)/(f) were also previously included in the apptainer project's upgrade to their v1.2.0 that was not submitted as an incompatible upgrade. Highlighting also for Dave Dykstra’s benefit that any thoughts around the relevance of incompatible upgrade policy to (b) & (c) will also be relevant to apptainer’s upcoming 1.3.0 upgrade, as apptainer has pulled both of these changes from singularity-ce upstream. Other changes in behaviour from 3.11.5 -> 4.1.0 are compatible. Configuration files do not need to be modified. Many Thanks, Dave Trudgian [1] https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/message/IQENLTYUAIMOTAX4LUWHYINOPSCRWCIS/ -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
[EPEL-devel] Re: Packaging a newer singularity-ce as singularity-ce4
Thanks for your comments. >> I’ve had some discussion with Jonathan Wright elsewhere about the topic of >> this message, but wanted to verify my understanding is correct before I >> embark on it, and thought I’d do so on list. >> >> singularity-ce is currently packaged at v4.0.3 in Fedora Rawhide, and >> v.3.11.5 elsewhere (Fedora releases and EPEL). >> >> We want to make a v4 available to EPEL users, as many would be interested in >> it, but I wouldn’t consider it a compatible update because there are some >> CLI changes, and small behaviour changes. >> >> My understanding is that in order to provide a 4.x in EPEL, without any >> incompatible update happening for users: >> >> 1) I create a new package, singularity-ce4, to package the 4.x version. In >> rawhide, this will be the same as the singularity-ce package currently in >> rawhide, but needs new package review etc. > > Creating a versioned package does NOT require a new review[1], though > if you feel that packaging changes are going to be large enough to > warrant one, you may still request it. The linked document mentions - "The package MUST be properly named according to the naming guidelines and MUST NOT conflict with all other versions of the same package.” (https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuidelines/#_package_review_process) I read this as both versions should be installable at the same time? This is not easily the case here, as singularity has to provide a `run-singularity` executable that may called from a '#!/usr/bin/env run-singularity’ embedded in our container image format. It’s also perhaps complicated by the fact that we already conflict with the apptainer package that provides /usr/bin/[run-]singularity and attempts to migrate configuration. We both have a ‘Provides/Conflicts: sif-runtime’, and `Provides: alternative-for(singularity)` around this. I’d assume preserving the right behaviour there deserves review. This is all complication arising from singularity-ce and apptainer being separate forks of the project previously known as ’singularity’. >> 2) For rawhide / upcoming f40 *only*, the new singularity-ce4 package will >> provide/obsolete singularity-ce as it is the same thing … and singularity-ce >> can be retired in rawhide. > >> 3) When singularity-ce4 is added to EPEL it will *not* provide/obsolete >> singularity-ce, but a message can be added to %post to inform people about >> the availability of v4. > > Do not do this. %post messages are really only intended to inform > users of failures and, frankly, no one reads them until something has > gone wrong. Even then, it's only going to be the sysop for the machine > that sees it, who may not be the person who deals with Singularity. Agreed. This was a prior suggestion made to me. > > I don't know anything about Singularity, but if it has a user > interface of any kind (like the CLI), what you might want to do is add > a wrapper around it that prints your message. That's much more likely > to be viewed by the people who would care. We can certainly do that. > >> At some point in the future, if 3.x is no longer maintainable for good >> reason, then the incompatible update procedure can be pursued to make >> singularity-ce4 provide/obsolete singularity-ce in EPEL 7/8/9 - and >> singularity-ce is fully retired. EPEL 10 will only get singularity-ce4. > > Is v3 still supported upstream today? If not, you probably want to > make the message above a deprecation notice and add an EOL date. v3 is only minimally supported upstream, and entirely for the purposes of having it in Fedora & EPEL without forcing an incompatible upgrade. It’s unsupported in any other form (outside of commercial LTS). I am both the upstream maintainer and the packager here, both in the context of employment. It would be beneficial if I didn’t need to spend any time on maintaining v3 at all, but we are attempting to meet the broad expectations of EPEL, given we are both upstream and packaging... Since I am both the upstream maintainer, and the downstream packager of singularity-ce, it seems somewhat hard to argue I can’t keep it updated for at least a while with backports to avoid forcing an incompatible update. I do the bundled dependency updates etc. upstream, rather than via packaging patches or similar, for my own convenience. In previous discussions with others involved in EPEL it seemed that it was perhaps considered reasonable to try and maintain the v3 in EPEL until such time as it drops out of a stable Fedora, or there is a security issue with a fix that is not reasonably practical to back port. -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:
[EPEL-devel] Packaging a newer singularity-ce as singularity-ce4
Hi all, I’ve had some discussion with Jonathan Wright elsewhere about the topic of this message, but wanted to verify my understanding is correct before I embark on it, and thought I’d do so on list. singularity-ce is currently packaged at v4.0.3 in Fedora Rawhide, and v.3.11.5 elsewhere (Fedora releases and EPEL). We want to make a v4 available to EPEL users, as many would be interested in it, but I wouldn’t consider it a compatible update because there are some CLI changes, and small behaviour changes. My understanding is that in order to provide a 4.x in EPEL, without any incompatible update happening for users: 1) I create a new package, singularity-ce4, to package the 4.x version. In rawhide, this will be the same as the singularity-ce package currently in rawhide, but needs new package review etc. 2) For rawhide / upcoming f40 *only*, the new singularity-ce4 package will provide/obsolete singularity-ce as it is the same thing … and singularity-ce can be retired in rawhide. 3) When singularity-ce4 is added to EPEL it will *not* provide/obsolete singularity-ce, but a message can be added to %post to inform people about the availability of v4. At some point in the future, if 3.x is no longer maintainable for good reason, then the incompatible update procedure can be pursued to make singularity-ce4 provide/obsolete singularity-ce in EPEL 7/8/9 - and singularity-ce is fully retired. EPEL 10 will only get singularity-ce4. Apologies for the multiple complex queries lately. I really appreciate your guidance! Dave Trudgian -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
[EPEL-devel] Re: Bundling newer 3rd party binaries than are packaged separately
Many thanks for clarifying the bundling of 3rd party binaries. > On 23 Jan 2024, at 21:18, Stephen Gallagher wrote: > > If you are bundling any software, you need to `Provides: > bundled(software)`. This is so we can easily locate affected packages > when e.g. a security issue necessitates fixing it. I will make sure to include those. Also CC’ing Dave Dykstra who maintains the apptainer package I referenced earlier in my original email. As I understand it, he will need to add the Provides for the binaries currently bundled (without corresponding Provides) in the apptainer packaging. > Also, since it wasn't clear from your text above: It's (generally) > alright under these circumstances to bundle the extra packages, but > you need to meet certain requirements: > > * The code that you're bundling still has to be built in Fedora. That > probably means compiling it as part of your SingularityCE build. You > may not ship code that was compiled somewhere else (e.g. upstream). This will be the case. The 3rd party binaries are built from source within the SingularityCE build. > * If you are shipping executables exclusively for use with your > package, make sure they are properly namespaced in > /usr/libexec/singularityce (or similar). This is to ensure that no > other package accidentally tries to use your bundled version. This is also the case. The binaries are installed into a specific libexec dir. Thanks again, Dave Trudgian -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
[EPEL-devel] Bundling newer 3rd party binaries than are packaged separately
Hi all, I currently package singularity-ce for Fedora and EPEL. Upstream, we bundle current versions of squashfuse and conmon with our source and own binary packages… because many distros package versions that are too old to work with SingularityCE, and users installing our upstream binary packages just want them to work. In Fedora & EPEL packaging I disable the building of the bundled squashfuse and conmon in the spec file, and we rely on the distro’s squashfuse and conmon packages. This is fine with Fedora, and has been okay up until now for EPEL. However, I want to move forward with packaging SingularityCE 4.x for EPEL and there we need a newer squashfuse than is available in EPEL7. Given our user base, leaving EPEL7 without the update wouldn’t be popular, even if it is approaching EOL. I wanted to verify if whether it's acceptable to bundle a newer squashfuse in the SingularityCE package as a binary under our libexec dir, given that an older squashfuse is provided by an existing squashfuse package? If so, are we required to declare the bundling in the spec file, as we are doing for bundled go deps in the spec? What has given me pause is reviewing the bundling guidelines at https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling - where it seems, at least for libraries, that a `Provides: bundled() = ` is required… and it’s not really clear to me whether the discussion there about libraries can be directly applied to *executables* that might be bundled? I note that the apptainer spec / package is already bundling a newer squashfuse binary into its libexec dir without a corresponding Provides: … so perhaps it’s fine to go ahead? Apologies if I have missed prior discussion on this. https://src.fedoraproject.org/rpms/apptainer/blob/rawhide/f/apptainer.spec https://packages.fedoraproject.org/pkgs/apptainer/apptainer/epel-7.html#files And it looks like their next release might be bundling a newer fuse2fs than is in the distro e2fstools package too, plus a newer fuse-overlayfs than is in the distro package: https://src.fedoraproject.org/rpms/apptainer/blob/rawhide/f/apptainer.spec https://packages.fedoraproject.org/pkgs/apptainer/apptainer/fedora-rawhide.html#files Thanks, Dave Trudgian -- ___ epel-devel mailing list -- epel-devel@lists.fedoraproject.org To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
