Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
On Sat, 30 Mar 2024 03:07:13 - "Eddie Chapman" wrote: > Given what we've learnt in the last 24hrs about xz utilities, you > could forgive a paranoid person for seriously considering getting rid > entirely of them from their systems, especially since there are > suitable alternatives available. Some might say that's a bit > extreme, xz-utils will get a thorough audit and it will all be fine. > But when a malicious actor has been a key maintainer of something as > complex as a decompression utility for years, I'm not sure I could > ever trust that codebase again. Maybe a complete rewrite will emerge, > but I'm personally unwilling to continue using xz utils in the > meantime for uncompressing anything on my systems, even if it is done > by an unprivileged process. > > I see that many system package ebuilds unconditionally expect > app-arch/xz-utils to be installed simply to be able to decompress the > source archive in SRC_URI. So simply specifying -lzma on your system > isn't going to get rid of it. > > No one could have been expected to foresee what's happened with > xz-utils, but now that it's here, perhaps Gentoo (and other projects > that do) should consider not relying on a single decompression > algorithm for source archives, even just as an insurance against some > other yet unknown disaster with one algorithm or another in future? > > And yes I'm sure there will be individual packages that currently > absolutely need xz-utils installed during the build process, and one > or two that absolutely have to have it available at runtime, but those > bridges can be crossed as and when. > > Eddie > > I think this is an overreaction and we should wait for the dust to settle before making drastic disruptive changes.
[gentoo-dev] Current unavoidable use of xz utils in Gentoo
Given what we've learnt in the last 24hrs about xz utilities, you could forgive a paranoid person for seriously considering getting rid entirely of them from their systems, especially since there are suitable alternatives available. Some might say that's a bit extreme, xz-utils will get a thorough audit and it will all be fine. But when a malicious actor has been a key maintainer of something as complex as a decompression utility for years, I'm not sure I could ever trust that codebase again. Maybe a complete rewrite will emerge, but I'm personally unwilling to continue using xz utils in the meantime for uncompressing anything on my systems, even if it is done by an unprivileged process. I see that many system package ebuilds unconditionally expect app-arch/xz-utils to be installed simply to be able to decompress the source archive in SRC_URI. So simply specifying -lzma on your system isn't going to get rid of it. No one could have been expected to foresee what's happened with xz-utils, but now that it's here, perhaps Gentoo (and other projects that do) should consider not relying on a single decompression algorithm for source archives, even just as an insurance against some other yet unknown disaster with one algorithm or another in future? And yes I'm sure there will be individual packages that currently absolutely need xz-utils installed during the build process, and one or two that absolutely have to have it available at runtime, but those bridges can be crossed as and when. Eddie
Re: [gentoo-dev] [PATCH] metadata/install-qa-check.d: make the FHS paths warning easier to read
On 3/29/24 12:54 AM, Haelwenn (lanodan) Monnier wrote:
>> - strip ${D} so that people can better visualize what sort of path gets
>> installed. This has the downside of not being able to copy/paste the
>> path in order to inspect the image directory, but I think this is a
>> very small downside. Usually by the time you see this message, portage
>> has cleaned up. And if it hasn't, you can still copy/paste that from:
>>
>> Completed installing sys-cluster/legion- into
>> /var/tmp/portage/sys-cluster/legion-/image
>
> Could maybe be confusable with installers not supporting DESTDIR though
> (although the sandbox messages look *very* different).
I'm not worried about confusing it with lack of DESTDIR support, since
lack of DESTDIR support would equal a failed src_install rather than a
FHS warning.
It's really nothing to do with the sandbox specifically.
--
Eli Schwartz
OpenPGP_0x84818A6819AF4A9B.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH] flag-o-matic.eclass: simplify implementation and work in all cases
Eli Schwartz writes:
> It curently uses some magic test to decide whether handcrafted code
> works with or without -latomic. But it can claim that -latomic is not
> needed for that case, while it is still needed for other cases.
>
>> okay so append-atomic-flags does not work for me in this case
>> noise-suppression-for-voice is doing `struct RnNoiseStats { uint32_t a, b,
>> c, d; }; std::atomic m_stats;`
>> not just a single large integer
>
> It is simplest to always add -latomic when an ebuild gets that deep
> feeling that yeah, it would like some atomics please. The downsides to
> listing a linker library are exactly:
>
> - it might be unavailable
> - it might be unneeded
>
> And the former case is trivial to solve -- this function already does so
> -- while the latter case has a sanctioned approach that is already used
> for other intrinsic compiler libraries, but not for atomic "because the
> build system would have a hard time if we had to build atomic early on"
> which isn't a very good reason to break ebuilds which aren't building
> sys-devel/gcc.
>
> As a side benefit, we now handle -latomic such that a package which
> requires it, but only for parts of the installed package, does not
> overlink to libatomic in *all* binaries/libraries, even if the default
> LDFLAGS are overridden and the global -Wl,--as-needed disappears.
>
> Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358
> Bug: https://bugs.gentoo.org/820101
> Bug: https://bugs.gentoo.org/925672
> Signed-off-by: Eli Schwartz
LGTM. Thanks.
> ---
> eclass/flag-o-matic.eclass | 80 +-
> 1 file changed, 19 insertions(+), 61 deletions(-)
>
> diff --git a/eclass/flag-o-matic.eclass b/eclass/flag-o-matic.eclass
> index 5ce7601fdde2..0e5271c7824f 100644
> --- a/eclass/flag-o-matic.eclass
> +++ b/eclass/flag-o-matic.eclass
> @@ -1015,69 +1015,27 @@ test-compile() {
> }
>
> # @FUNCTION: append-atomic-flags
> -# @USAGE: [bytes]
> # @DESCRIPTION:
> -# Attempts to detect if appending -latomic is required to use
> -# a specific-sized atomic intrinsic, and if so, appends it. If the bytesize
> -# is not specified, then check the four most common byte sizes (1, 2, 4, 8).
> -# >=16-byte atomics are not included in this default set and must be
> explicitly
> -# passed if required. This may require you to add a macro definition like
> -# -Duint128_t=__uint128_t to your CFLAGS.
> +# Attempts to detect if appending -latomic works, and does so.
> append-atomic-flags() {
> - # this implementation is as described in bug #820101
> - local code
> -
> - # first, ensure we can compile a trivial program
> - # this is because we can't distinguish if test-compile
> - # fails because -latomic is actually needed or if we have a
> - # broken toolchain (like due to bad FLAGS)
> - read -r -d '' code <<- EOF
> - int main(void)
> - {
> - return 0;
> - }
> - EOF
> -
> - # if toolchain is broken, just return silently. it's better to
> - # let other pieces of the build fail later down the line than to
> - # make people think that something to do with atomic support is the
> - # cause of their problems.
> - test-compile "c+ld" "${code}" || return
> -
> - local bytesizes
> - [[ "${#}" == "0" ]] && bytesizes=( "1" "2" "4" "8" ) || bytesizes="${@}"
> -
> - for bytesize in ${bytesizes[@]}
> - do
> - # this sample program is informed by the great testing from the
> buildroot project:
> - #
> https://github.com/buildroot/buildroot/commit/6856e417da4f3aa77e2a814db2a89429af072f7d
> - read -r -d '' code <<- EOF
> - #include
> - int main(void)
> - {
> - uint$((${bytesize} * 8))_t a = 0;
> - __atomic_add_fetch(, 3, __ATOMIC_RELAXED);
> - __atomic_compare_exchange_n(, , 2, 1,
> __ATOMIC_RELAXED, __ATOMIC_RELAXED);
> - return 0;
> - }
> - EOF
> -
> - # do nothing if test program links fine
> - test-compile "c+ld" "${code}" && continue
> -
> - # ensure that the toolchain supports -latomic
> - test-flags-CCLD "-latomic" &>/dev/null || die "-latomic is
> required but not supported by $(tc-getCC)"
> -
> - append-libs "-latomic"
> -
> - # verify that this did indeed fix the problem
> - test-compile "c+ld" "${code}" || \
> - die "libatomic does not include an implementation of
> ${bytesize}-byte atomics for this toolchain"
> -
> - # if any of the required bytesizes require -latomic, no need to
> continue
> - # checking the others
> - return
> - done
> + # Make sure that the flag is actually valid. If it isn't, then maybe the
> + #
Re: [gentoo-dev] [PATCH] metadata/install-qa-check.d: make the FHS paths warning easier to read
Eli Schwartz writes:
> * QA Notice: The ebuild is installing to one or more unexpected paths:
> *
> * /var/tmp/portage/sys-cluster/legion-/image/usr/bin/legion_prof_files
> *
> /var/tmp/portage/sys-cluster/legion-/image/usr/bin/serializer_examples
> *
> * Please fix the ebuild to use correct FHS/Gentoo policy paths.
>
> This message is hard to understand. Is it saying that the resulting
> package contains files prefixed with ${D} which would be immensely
> broken? Is it saying that these paths are *directories* and the FHS does
> not approve of directories in /usr/bin/*/?
>
> In fact, it's the latter. Fix this in two ways:
>
> - clarify that it's an unexpected directory, not just some kind of path
>
> - strip ${D} so that people can better visualize what sort of path gets
> installed. This has the downside of not being able to copy/paste the
> path in order to inspect the image directory, but I think this is a
> very small downside. Usually by the time you see this message, portage
> has cleaned up. And if it hasn't, you can still copy/paste that from:
>
> Completed installing sys-cluster/legion- into
> /var/tmp/portage/sys-cluster/legion-/image
>
> Signed-off-by: Eli Schwartz
> ---
LGTM. Thanks.
> metadata/install-qa-check.d/08gentoo-paths | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/metadata/install-qa-check.d/08gentoo-paths
> b/metadata/install-qa-check.d/08gentoo-paths
> index 5b8607fd5f96..0b92a7a1c132 100644
> --- a/metadata/install-qa-check.d/08gentoo-paths
> +++ b/metadata/install-qa-check.d/08gentoo-paths
> @@ -70,9 +70,9 @@ gentoo_path_check() {
> # report
> # --
> if [[ -n ${bad_paths[@]} ]]; then
> - eqawarn "QA Notice: The ebuild is installing to one or more
> unexpected paths:"
> + eqawarn "QA Notice: The ebuild is installing to one or more
> unexpected directories:"
> eqawarn
> - eqatag -v non-gentoo-paths "${bad_paths[@]}"
> + eqatag -v non-gentoo-paths "${bad_paths[@]#${D%/}}"
> eqawarn
> eqawarn "Please fix the ebuild to use correct FHS/Gentoo policy
> paths."
> fi
signature.asc
Description: PGP signature
[gentoo-dev] [PATCH] python-utils-r1.eclass: epytest, error out on missing async plugin
Explicitly error out if epytest is run without an appropriate async
plugin, and the test suite contains async tests. Currently, these tests
are skipped with a warning but that is usually a mistake, and one can
easily miss it when pytest-asyncio or a similar plugin is installed
on the test system. However, a missing dependency can result
in the tests being skipped afterwards on the tinderbox.
Signed-off-by: Michał Górny
---
eclass/python-utils-r1.eclass | 3 +++
1 file changed, 3 insertions(+)
diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass
index caa39813feec..bbf751399476 100644
--- a/eclass/python-utils-r1.eclass
+++ b/eclass/python-utils-r1.eclass
@@ -1349,6 +1349,9 @@ epytest() {
# override filterwarnings=error, we do not really want -Werror
# for end users, as it tends to fail on new warnings from deps
-Wdefault
+ # however, do error out if the package failed to load
+ # an appropriate async plugin
+ -Werror::pytest.PytestUnhandledCoroutineWarning
# override color output
"--color=${color}"
# count is more precise when we're dealing with a large number
--
2.44.0
Re: [gentoo-dev] [PATCH] metadata/install-qa-check.d: make the FHS paths warning easier to read
On Thu, 2024-03-28 at 23:54 -0400, Eli Schwartz wrote:
> * QA Notice: The ebuild is installing to one or more unexpected paths:
> *
> * /var/tmp/portage/sys-cluster/legion-/image/usr/bin/legion_prof_files
> *
> /var/tmp/portage/sys-cluster/legion-/image/usr/bin/serializer_examples
> *
> * Please fix the ebuild to use correct FHS/Gentoo policy paths.
>
> This message is hard to understand. Is it saying that the resulting
> package contains files prefixed with ${D} which would be immensely
> broken? Is it saying that these paths are *directories* and the FHS does
> not approve of directories in /usr/bin/*/?
>
> In fact, it's the latter. Fix this in two ways:
>
> - clarify that it's an unexpected directory, not just some kind of path
>
> - strip ${D} so that people can better visualize what sort of path gets
> installed. This has the downside of not being able to copy/paste the
> path in order to inspect the image directory, but I think this is a
> very small downside. Usually by the time you see this message, portage
> has cleaned up. And if it hasn't, you can still copy/paste that from:
>
> Completed installing sys-cluster/legion- into
> /var/tmp/portage/sys-cluster/legion-/image
>
> Signed-off-by: Eli Schwartz
> ---
> metadata/install-qa-check.d/08gentoo-paths | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/metadata/install-qa-check.d/08gentoo-paths
> b/metadata/install-qa-check.d/08gentoo-paths
> index 5b8607fd5f96..0b92a7a1c132 100644
> --- a/metadata/install-qa-check.d/08gentoo-paths
> +++ b/metadata/install-qa-check.d/08gentoo-paths
> @@ -70,9 +70,9 @@ gentoo_path_check() {
> # report
> # --
> if [[ -n ${bad_paths[@]} ]]; then
> - eqawarn "QA Notice: The ebuild is installing to one or more
> unexpected paths:"
> + eqawarn "QA Notice: The ebuild is installing to one or more
> unexpected directories:"
> eqawarn
> - eqatag -v non-gentoo-paths "${bad_paths[@]}"
> + eqatag -v non-gentoo-paths "${bad_paths[@]#${D%/}}"
> eqawarn
> eqawarn "Please fix the ebuild to use correct FHS/Gentoo policy
> paths."
> fi
LGTM. Thanks!
--
Best regards,
Michał Górny
signature.asc
Description: This is a digitally signed message part
