Re: [gentoo-dev] rfc: usrmerge script
On Tue, Mar 23, 2021 at 10:23:11AM +0100, Michał Górny wrote: > On Sun, 2021-03-21 at 12:39 -0500, William Hubbs wrote: > > All, > > > > the following is a script which will migrate a Gentoo system to the usr > > merge layout. This is similar to the unsymlink-lib tool used to migrate > > a system from the 17.0 to the 17.1 profiles. > > > > I'm attaching it here to get some comments before I package it, so > > please let me know if I have missed something. > > To be honest, I don't think critical system modifications should be done > in shell script, and especially not via a fringe non-standards complaint > shell implementation in busybox. Even if you can assume you make no > mistakes, shell is unreliable by design. For example, your script may > start behaving in unexpected ways if you run out of space. I went with busybox shell in this case because busybox is a static binary and I figured with everything being moved around on the system it would be a good choice. I could pretty easily go with straight sh/bash, I just was focusing on bb since the commands are built in. You are right about shell behaving in weird ways if you run out of space, but that's true for anything that happens to be running when a system runs out of space. That is why I put the rollback directory in /var by default figuring there is more space there than in /, especially on systems with separate /var. The run_command wrapper in the script causes an immediate exit when the command it runs fails so things don't go any further than the failing command, so I'm not sure what else I can do with this situation. One option is to always echo the command we are about to run then just not run it if -d is specified. That means you would always see the commands the script is running. > You don't seem to be handling file collisions at all. Even today we > have files like /bin/bzip2 and /usr/bin/bzip2, not to mention shared > libraries. Silently ignoring the problem or requiring the users to > manually ensure their system is clean is not going to solve it. I have added -i to the cp commands in my latest version of this so I'll see when we have duplicate names, and yes that is an issue, even without the /usr merge. I'm curious why we are duplicating all of these names in the first place honestly. One example of this is in coreutils, and we even say in the ebuild comment that we need to figure out why we are doing it. There are a couple of ways forward for this. 1) attempt to open bugs on the packages and remove the duplicate names by dropping symlinks and putting the files in their cannonical locations. this could lead to breakages if one of the alternate names is expected by something until the /usr merge is done. 2) try to guess at resolving the duplicate names as part of the usr merge process. a. symmlinks in /bin or /sbin that point to the same name in /usr/bin or /usr/sbin should be removed. b. symlinks in /usr/bin or /usr/sbin that point to the same name in / should be removed. c. Other symlinks that I can think of should be preserved. Which path for handling this is best? Do you have any thoughts? > You don't seem to provide any helpful messages. When things fail, user > will be left in the blue with an error message from some system tool (or > rather, cheap-ass busybox rewrite, I guess). If the rollback directory is populated, you can use -r to recover, and the script will not let you perform the /usr merge if the rollback directory is not populated. > Also, have you verified that busybox's cp(1) actually preserves all file > properties (including xattrs, ACLs, caps...)? The documentation for busybox states that -p preserves file attributes if possible, and by doing ls -l in the chroot I can see that ownership/permissions are preserved. So, logically I would guess it preserves the others. If switching back to non-busybox is fine for this operation, I have no problem doing that either. > Please don't forget to include tests with it. Docker's good for testing > stuff like this. Docker can't be used during src_test that I'm aware of, so I'm not sure how Docker could be used to test this. William signature.asc Description: PGP signature
[gentoo-dev] Cleanup of sunrise remains
Dear all, Sunrise was discontinued on June 2016. [1] Its remains confuse our users. There are several places, where sunrise is still advertised. Now it is time for the final shutdown. [3] I suggest to move the ebuilds [2] to an archive somewhere for 6 months for the unlikely case that there are any valuable ebuilds left. If there is any useful package left, please pick it up, overhaul it and move it either to the tree, or guru. [1] https://archives.gentoo.org/gentoo-project/message/df16291a26bbc5c3a219ab26fc7bac99 [2] https://gitweb.gentoo.org/proj/sunrise.git/ [3] https://bugs.gentoo.org/777918 -- Best, Jonas OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH v2] gstreamer-meson.eclass: New eclass required for gstreamer-1.18.0+
On Tue, 2021-03-23 at 13:44 +0100, Haelwenn (lanodan) Monnier wrote:
> Gstreamer switched to meson in 1.16.0 and removed autotools support in 1.18.0,
> this eclass is an update of gstreamer.eclass.
>
> One significant change between autotools and meson is that in the latter we
> don't have easily extractable semantics in the buildsystem to get a list
> of plugins with extraneous dependencies that we currently split in other
> packages.
> Hence the rather ugly but currently required GST_PLUGINS_DISABLED block.
> The gstreamer_system_link function also got lost in translation.
>
> Differences from version 1:
> - Move to EAPI-7, including moving deps from DEPEND to BDEPEND when
> appropriate
> - Port python script to perl, this allows to avoid having to add PYTHON_COMPAT
> into a python-unrelated eclass
> - Drop errorneous MULTILIB_USEDEP on virtual/pkgconfig
> - Fix running tests: defining multilib_src_test, media-libs/gstreamer[test]
> dep
> - Fix ebuild emesonargs being ignored
> - Remove legacy prune_libtool_files
> - virtualx wrapped for testing
>
> Fixes: https://bugs.gentoo.org/690468
>
> Signed-off-by: Haelwenn (lanodan) Monnier
> ---
> eclass/gstreamer-meson.eclass | 320 ++
> 1 file changed, 320 insertions(+)
> create mode 100644 eclass/gstreamer-meson.eclass
>
> diff --git a/eclass/gstreamer-meson.eclass b/eclass/gstreamer-meson.eclass
> new file mode 100644
> index 000..14a825f76b5
> --- /dev/null
> +++ b/eclass/gstreamer-meson.eclass
> @@ -0,0 +1,320 @@
> +# Copyright 1999-2021 Gentoo Authors
> +# Distributed under the terms of the GNU General Public License v2
> +
> +# @ECLASS: gstreamer-meson.eclass
> +# @MAINTAINER:
> +# gstrea...@gentoo.org
> +# @AUTHOR:
I think the list actually goes the other way around, i.e. you should add
yourself on top.
> +# Michał Górny
> +# Gilles Dartiguelongue
> +# Saleem Abdulrasool
> +# foser
> +# zaheerm
> +# Steven Newbury
> +# Haelwenn (lanodan) Monnier
> +# @SUPPORTED_EAPIS: 7
> +# @BLURB: Helps building core & split gstreamer plugins.
No full stop here, it's not a proper sentence.
> +# @DESCRIPTION:
> +# Eclass to make external gst-plugins emergable on a per-plugin basis
> +# and to solve the problem with gst-plugins generating far too much
> +# unneeded dependencies.
> +#
> +# GStreamer consuming applications should depend on the specific plugins
> +# they need as defined in their source code. Usually you can find that
> +# out by grepping the source tree for 'factory_make'. If it uses playbin
> +# plugin, consider adding media-plugins/gst-plugins-meta dependency, but
> +# also list any packages that provide explicitly requested plugins.
> +
> +inherit multilib virtualx meson toolchain-funcs xdg-utils multilib-minimal
I'm pretty sure you can sort them a bit.
> +
> +case "${EAPI:-0}" in
> + 7)
> + ;;
> + *)
> + die "EAPI=\"${EAPI}\" is not supported"
> + ;;
> +esac
> +
> +# @ECLASS-VARIABLE: GST_PLUGINS_ENABLED
> +# @DESCRIPTION:
> +# Defines the plugins to be built.
> +# May be set by an ebuild and contain more than one indentifier, space
> +# seperated (only src_configure can handle mutiple plugins at this time).
> +: ${GST_PLUGINS_ENABLED:=${PN/gst-plugins-/}}
> +
> +# @ECLASS-VARIABLE: GST_PLUGINS_DISABLED
> +# @DESCRIPTION:
> +# Defines the plugins to not be built, GST_PLUGINS_ENABLED overrides it.
> +# May be set by an ebuild and contain more than one indentifier, space
> +# seperated (only src_configure can handle mutiple plugins at this time).
> +case "${GST_ORG_MODULE}" in
> + # copied GST_PLUGINS_DISABLED from media-libs/${GST_ORG_MODULE} then
> added GST_PLUGINS_ENABLED
> + gst-plugins-bad)
> + # removed from list: shm ipcpipeline gl
> + GST_PLUGINS_DISABLED="aom avtp androidmedia applemedia
> assrender bluez bs2b bz2 chromaprint closedcaption colormanagement curl
> curl-ssh2 d3dvideosink d3d11 dash dc1394 decklink directfb directsound dtls
> dts dvb faac faad fbdev fdkaac flite fluidsynth gme gsm iqa kate kms ladspa
> libde265 libmms lv2 mediafoundation microdns modplug mpeg2enc mplex msdk
> musepack neon nvcodec ofa openal openexr openh264 openjpeg openmpt openni2
> opensles opus resindvd rsvg rtmp sbc sctp smoothstreaming sndfile soundtouch
> spandsp srt srtp svthevcenc teletext tinyalsa transcode ttml uvch264 va
> voaacenc voamrwbenc vulkan wasapi wasapi2 webp webrtc webrtcdsp wildmidi
> winks winscreencap x265 zbar zxing wpe magicleap v4l2codecs hls opencv"
> + GST_PLUGINS_DISABLED="${GST_PLUGINS_DISABLED} accurip adpcmdec
> adpcmenc aiff asfmux audiobuffersplit audiofxbad audiolatency audiomixmatrix
> audiovisualizers autoconvert bayer camerabin2 coloreffects deb ugutils
> dvbsubenc dvbsuboverlay dvdspu faceoverlay festival fieldanalysis freeverb
> frei0r gaudieffects gdp geometrictransform id3tag inter interlace ivfpars e
> ivtc jp2kdecimator jpegformat librfb midi mpegdemux
[gentoo-dev] [PATCH v2] gstreamer-meson.eclass: New eclass required for gstreamer-1.18.0+
Gstreamer switched to meson in 1.16.0 and removed autotools support in 1.18.0,
this eclass is an update of gstreamer.eclass.
One significant change between autotools and meson is that in the latter we
don't have easily extractable semantics in the buildsystem to get a list
of plugins with extraneous dependencies that we currently split in other
packages.
Hence the rather ugly but currently required GST_PLUGINS_DISABLED block.
The gstreamer_system_link function also got lost in translation.
Differences from version 1:
- Move to EAPI-7, including moving deps from DEPEND to BDEPEND when appropriate
- Port python script to perl, this allows to avoid having to add PYTHON_COMPAT
into a python-unrelated eclass
- Drop errorneous MULTILIB_USEDEP on virtual/pkgconfig
- Fix running tests: defining multilib_src_test, media-libs/gstreamer[test] dep
- Fix ebuild emesonargs being ignored
- Remove legacy prune_libtool_files
- virtualx wrapped for testing
Fixes: https://bugs.gentoo.org/690468
Signed-off-by: Haelwenn (lanodan) Monnier
---
eclass/gstreamer-meson.eclass | 320 ++
1 file changed, 320 insertions(+)
create mode 100644 eclass/gstreamer-meson.eclass
diff --git a/eclass/gstreamer-meson.eclass b/eclass/gstreamer-meson.eclass
new file mode 100644
index 000..14a825f76b5
--- /dev/null
+++ b/eclass/gstreamer-meson.eclass
@@ -0,0 +1,320 @@
+# Copyright 1999-2021 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+# @ECLASS: gstreamer-meson.eclass
+# @MAINTAINER:
+# gstrea...@gentoo.org
+# @AUTHOR:
+# Michał Górny
+# Gilles Dartiguelongue
+# Saleem Abdulrasool
+# foser
+# zaheerm
+# Steven Newbury
+# Haelwenn (lanodan) Monnier
+# @SUPPORTED_EAPIS: 7
+# @BLURB: Helps building core & split gstreamer plugins.
+# @DESCRIPTION:
+# Eclass to make external gst-plugins emergable on a per-plugin basis
+# and to solve the problem with gst-plugins generating far too much
+# unneeded dependencies.
+#
+# GStreamer consuming applications should depend on the specific plugins
+# they need as defined in their source code. Usually you can find that
+# out by grepping the source tree for 'factory_make'. If it uses playbin
+# plugin, consider adding media-plugins/gst-plugins-meta dependency, but
+# also list any packages that provide explicitly requested plugins.
+
+inherit multilib virtualx meson toolchain-funcs xdg-utils multilib-minimal
+
+case "${EAPI:-0}" in
+ 7)
+ ;;
+ *)
+ die "EAPI=\"${EAPI}\" is not supported"
+ ;;
+esac
+
+# @ECLASS-VARIABLE: GST_PLUGINS_ENABLED
+# @DESCRIPTION:
+# Defines the plugins to be built.
+# May be set by an ebuild and contain more than one indentifier, space
+# seperated (only src_configure can handle mutiple plugins at this time).
+: ${GST_PLUGINS_ENABLED:=${PN/gst-plugins-/}}
+
+# @ECLASS-VARIABLE: GST_PLUGINS_DISABLED
+# @DESCRIPTION:
+# Defines the plugins to not be built, GST_PLUGINS_ENABLED overrides it.
+# May be set by an ebuild and contain more than one indentifier, space
+# seperated (only src_configure can handle mutiple plugins at this time).
+case "${GST_ORG_MODULE}" in
+ # copied GST_PLUGINS_DISABLED from media-libs/${GST_ORG_MODULE} then
added GST_PLUGINS_ENABLED
+ gst-plugins-bad)
+ # removed from list: shm ipcpipeline gl
+ GST_PLUGINS_DISABLED="aom avtp androidmedia applemedia
assrender bluez bs2b bz2 chromaprint closedcaption colormanagement curl
curl-ssh2 d3dvideosink d3d11 dash dc1394 decklink directfb directsound dtls dts
dvb faac faad fbdev fdkaac flite fluidsynth gme gsm iqa kate kms ladspa
libde265 libmms lv2 mediafoundation microdns modplug mpeg2enc mplex msdk
musepack neon nvcodec ofa openal openexr openh264 openjpeg openmpt openni2
opensles opus resindvd rsvg rtmp sbc sctp smoothstreaming sndfile soundtouch
spandsp srt srtp svthevcenc teletext tinyalsa transcode ttml uvch264 va
voaacenc voamrwbenc vulkan wasapi wasapi2 webp webrtc webrtcdsp wildmidi winks
winscreencap x265 zbar zxing wpe magicleap v4l2codecs hls opencv"
+ GST_PLUGINS_DISABLED="${GST_PLUGINS_DISABLED} accurip adpcmdec
adpcmenc aiff asfmux audiobuffersplit audiofxbad audiolatency audiomixmatrix
audiovisualizers autoconvert bayer camerabin2 coloreffects deb ugutils
dvbsubenc dvbsuboverlay dvdspu faceoverlay festival fieldanalysis freeverb
frei0r gaudieffects gdp geometrictransform id3tag inter interlace ivfpars e
ivtc jp2kdecimator jpegformat librfb midi mpegdemux mpegpsmux mpegtsdemux
mpegtsmux mxf netsim onvif pcapparse pnm proxy rawparse removesilence rist
rtmp2 rtp sdp segmentclip siren smooth speed subenc switchbin timecode
videofilters videoframe_audiolevel videoparsers videosignal vmnc y4m"
+ ;;
+ gst-plugins-base)
+ GST_PLUGINS_DISABLED="cdparanoia libvisual opus tremor"
+ GST_PLUGINS_DISABLED="${GST_PLUGINS_DISABLED} adder app
audioconvert
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel wrote: > > > > Council decided years ago that we don't support separate /usr without > > > an initramfs, but we haven't completed that transition yet. > > > > Which doesn't imply that we deliberately break things. > > That's right. Though we should at some point start thinking about an end of > support for separate usr without initramfs. > Just to clarify - it is already unsupported at a distro level. It is just that some individual packages still work with it. The current Council decisions on the issue are (just providing for general reference): - "Since that particular setup may already be subtly broken today depending on the installed software, Council recommends using an early boot mount mechanism, e.g. initramfs, to mount /usr if /usr is on a separate partition." Accepted unanimously. [1] - "The intention is to eventually not require maintainers to support a separate /usr without an early boot mechanism once the Council agrees that the necessary docs/migration path is in place." Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1] - "The Council agrees that all preparations for dropping support for separate /usr without an initramfs or similar boot mechanism are complete. A news item will be prepared, and users will be given one month to switch after the news item has been sent." Accepted with 5 yes votes, 1 no vote, 1 abstention. [2] Current policy documentation: Developers are not required to support using separate /usr filesystem without an initramfs. [3] 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202 -- Rich
[gentoo-dev] Two package up for grabs
Hi community, already quite a while ago, base-system team had decided to no longer maintain the following packages: sys-fs/reiser4progs (up to date, 3 open bug reports) sys-libs/libaal (required by reiser4progs, no open bug reports) From those three open bug reports against reiser4progs, two are from 2013/2014 so they most likely can be closed as OBSOLETE. If you are interested in maintaining these packages, just replace base-system with you name in metadata.xml and reassign the open bug reports to you. In case nobody picks the package up in the following seven days, I will drop both packages to maintainer-needed. Thanks and kind regards -- Lars Wendler Gentoo package maintainer GPG: 21CC CF02 4586 0A07 ED93 9F68 498F E765 960E 9B39 pgph4FbK38d2s.pgp Description: Digitale Signatur von OpenPGP
Re: [gentoo-dev] rfc: usrmerge script
On Sun, 2021-03-21 at 12:39 -0500, William Hubbs wrote: > All, > > the following is a script which will migrate a Gentoo system to the usr > merge layout. This is similar to the unsymlink-lib tool used to migrate > a system from the 17.0 to the 17.1 profiles. > > I'm attaching it here to get some comments before I package it, so > please let me know if I have missed something. To be honest, I don't think critical system modifications should be done in shell script, and especially not via a fringe non-standards complaint shell implementation in busybox. Even if you can assume you make no mistakes, shell is unreliable by design. For example, your script may start behaving in unexpected ways if you run out of space. You don't seem to be handling file collisions at all. Even today we have files like /bin/bzip2 and /usr/bin/bzip2, not to mention shared libraries. Silently ignoring the problem or requiring the users to manually ensure their system is clean is not going to solve it. You don't seem to provide any helpful messages. When things fail, user will be left in the blue with an error message from some system tool (or rather, cheap-ass busybox rewrite, I guess). Also, have you verified that busybox's cp(1) actually preserves all file properties (including xattrs, ACLs, caps...)? Please don't forget to include tests with it. Docker's good for testing stuff like this. -- Best regards, Michał Górny
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
Hi Andreas, On 2021/03/23 00:54, Andreas K. Huettel wrote: >>> Council decided years ago that we don't support separate /usr without >>> an initramfs, but we haven't completed that transition yet. >> Which doesn't imply that we deliberately break things. > That's right. Though we should at some point start thinking about an end of > support for separate usr without initramfs. > > Why? Because the number of required hacks and complexity will only increase, > as will the number of uncooperative upstreams. It's called a strategic > retreat. :D > > My suggestion would be that the next profile version (21? 22?) declares > separate /usr a broken configuration, and explicitly encourages devs to > introduce all ebuild simplifications that are made possible by this. (Like > this symlink - no more conditional code.) No more discussions about "not > breaking things" at that point. Why was it ever copied in the first place? For as long as I can recall (been using Gentoo since 2003) I've always been using a symlink here, not to mention a separate /usr (which has only been in the last year or so mounted from initrd along with /lib/firmware - which was the trigger point when it became too big to be contained on our normal / partition, the other fix would have been to be more selective in which firmware to install but that would be a higher administrative overhead). To this day I still believe that / should contain a minimal viable bootable system (give me a shell and just enough to perform basic tasks like activating LVM, repairing and mounting filesystems, and ideally some or another editor such as busybox vi is good enough, mostly everything else can go to /usr even with it being "split"). I still don't see why a split /usr is a bad thing. In fact, there are a significant number cases where I've had FS corruption which I was able to recover without the need for additional bootable media (which when working remotely via IP KVMs can be difficult at best) due to "living in the past" as you say. There is no reason a symbolic link can't cross a filesystem boundary ... it's hard links that can't. Kind Regards, Jaco > (Or to put it another way, I think we should stop wasting time and effort > here just to be able to live in the past.) >
