[gentoo-dev] Last rites: dev-python/carbon
# Fabian Groffen (2024-04-16) # Official latest Python support 3.8, replacement app-metrics/go-carbon # is more performant and designed to be a drop-in replacement. # Removal on 2024-05-16, bug #929444 dev-python/carbon -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: dev-python/PySensors
# Fabian Groffen (2024-04-13) # Python wrapper around liblmsensors, no reverse dependencies # Removal on 2024-05-13, bug #929495 dev-python/PySensors -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
On 06-04-2024 12:57:23 +0100, Eddie Chapman wrote: > There is one significant thing that breaks, which is Gemato > (app-portage/gemato). Gemato requires lzma support in core python in > order to do GPG signature verification. This means you will have to say > goodbye (for now) to verifying upstream GPG signatures on distfiles, and > verification of Portage metadata after doing an emerge --sync. These > features have been added to Portage relatively recently (2022?) so are > "nice to have", without them your system is just less hardened, but > still with the very high level of security that Gentoo systems have has > always had prior to these features, in my opinion. Personally I can live > without them for now. Verifying hashes in Manifest files still works > fine and that's the main thing. You may disagree in which case, well, > don't do this then. I'm going to figure out an alternative way I can > verify Portage metadata soon, as there are other ways if you are creative. If you just want to verify signatures and manifests after sync, qmanifest from portage-utils can help you do this. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] maintainer-needed: dev-python/thrift
I do not have any means to test/use this anymore, so I'm forced to drop it to maintainer-needed. It has bug https://bugs.gentoo.org/928502 open for an update and ebuild improvements. Fabian On 01-04-2022 14:56:15 +0200, Fabian Groffen wrote: > Dropped this mask again, I missed a dep in the tree. > > Fabian > > On 01-04-2022 13:53:24 +0200, Fabian Groffen wrote: > > # Fabian Groffen (2022-04-01) > > # unused package, not enabling tests, bug #796830 > > # Removal in 30 days > > dev-python/thrift -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: mail-filter/libsrs_alt
# Fabian Groffen (2024-01-26) # compilation issues, no reverse dependencies in the tree # Removal on 2024-02-26 mail-filter/libsrs_alt -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New bootstrap-prefix global USE-flag and patch to llvm.eclass
On 12-09-2023 20:32:19 +0100, Alexey Sokolov wrote:
> Bug: https://bugs.gentoo.org/758167
> Full PR is at https://github.com/gentoo/gentoo/pull/32730
>
> Several LLVM packages require this early return, otherwise they fail to
> build on Darwin. I'll also need this USE-flag for
> sys-devel/clang-common, to distinguish between stage2 and stage3 of
> bootstrap-prefix.sh to configure clang differently.
>
> --
> Best regards,
> Alexey "DarthGandalf" Sokolov
> From de2bd1abc3e5c7607413633d132c604c6a801802 Mon Sep 17 00:00:00 2001
> From: Alexey Sokolov
> Date: Mon, 11 Sep 2023 23:26:49 +0100
> Subject: [PATCH] llvm.eclass: add global USE flag bootstrap-prefix
>
> Mask it everywhere except for prefix profiles
>
> Without this, stage2's LLVM packages fail to build.
>
> Bug: https://bugs.gentoo.org/758167
> Signed-off-by: Alexey Sokolov
> ---
> eclass/llvm.eclass| 7 +++
> profiles/base/use.mask| 4
> profiles/features/prefix/use.mask | 4
> profiles/use.desc | 1 +
> 4 files changed, 16 insertions(+)
>
> diff --git a/eclass/llvm.eclass b/eclass/llvm.eclass
> index 8198650aad9a7..87c2cedb3a376 100644
> --- a/eclass/llvm.eclass
> +++ b/eclass/llvm.eclass
> @@ -64,6 +64,8 @@ esac
> if [[ ! ${_LLVM_ECLASS} ]]; then
> _LLVM_ECLASS=1
>
> +IUSE="bootstrap-prefix"
> +
> # make sure that the versions installing straight into /usr/bin
> # are uninstalled
> DEPEND="!!sys-devel/llvm:0"
> @@ -242,6 +244,11 @@ llvm_fix_tool_path() {
> llvm_pkg_setup() {
> debug-print-function ${FUNCNAME} "${@}"
>
> + if use bootstrap-prefix; then
> + # AppleClang has unparseable version numbers, but it's
> irrelevant anyway
> + return
> + fi
> +
I might misunderstand this, but is this USE-flag supposed to be set only
during bootstrap, e.g. when host-provided Clang is used? If so, would
it be possible to use has_version or something instead?
Thanks,
Fabian
> if [[ ${MERGE_TYPE} != binary ]]; then
> LLVM_SLOT=$(get_llvm_slot "${LLVM_MAX_SLOT}")
>
> diff --git a/profiles/base/use.mask b/profiles/base/use.mask
> index 1d4f5b92865df..cc86fde21097a 100644
> --- a/profiles/base/use.mask
> +++ b/profiles/base/use.mask
> @@ -8,6 +8,10 @@
> # eudev is masked for removal
> eudev
>
> +# Alexey Sokolov (2023-09-11)
> +# Only needed during bootstrap of prefix
> +bootstrap-prefix
> +
> # David Seifert (2023-09-09)
> # EOL upstream in 2 months, causes major headaches for OpenSSL 1.1
> # masking. Removal on 2023-10-09.
> diff --git a/profiles/features/prefix/use.mask
> b/profiles/features/prefix/use.mask
> index 482ce57f04485..1f43ca23fd101 100644
> --- a/profiles/features/prefix/use.mask
> +++ b/profiles/features/prefix/use.mask
> @@ -4,6 +4,10 @@
> # prefix USE flag should always be unmasked in prefix profiles
> -prefix
>
> +# Alexey Sokolov (2023-09-11)
> +# Allow bootstrapping the prefix
> +-bootstrap-prefix
> +
> # USE flags inherited by the base/use.defaults file that shouldn't be in
> Prefix
> gpm
>
> diff --git a/profiles/use.desc b/profiles/use.desc
> index 6034f3bf6fc31..37c64f43759da 100644
> --- a/profiles/use.desc
> +++ b/profiles/use.desc
> @@ -29,6 +29,7 @@ big-endian - Big-endian toolchain support
> bindist - Flag to enable or disable options for prebuilt (GRP) packages (eg.
> due to licensing issues)
> blas - Add support for the virtual/blas numerical library
> bluetooth - Enable Bluetooth Support
> +bootstrap-prefix - !!internal use only!! DO NOT SET THIS FLAG YOURSELF!,
> used for bootstrapping Gentoo Prefix
> branding - Enable Gentoo specific branding
> build - !!internal use only!! DO NOT SET THIS FLAG YOURSELF!, used for
> creating build images and the first half of bootstrapping [make stage1]
> bzip2 - Use the bzlib compression library
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-dev] Re: [gentoo-dev-announce] Last rites: obsolete acct-* packages
On 18-07-2023 11:42:39 +0300, Зураб Квачадзе wrote: > How do we handle this case, then. > Imagine we have a leaf package acct-user/foo, which has a reserved UID of 123. > It gets last rited and its entry is removed from uid-gid.txt. After a while > appears a new package acct-user/bar, which takes the 123 UID. Then a user, say > Bob, updates their system, which haven't been updated for some time. What if > they still have acct-user/foo, when acct-user/bar with the same UID is > installed? Should we even care about such cases? IMO we should, thus 123 should not be removed from uid-gid.txt, and instead be marked as reserved or something with a date. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Package stabilization groups
On 16-07-2023 10:57:54 -0400, Matt Turner wrote: > Hello, > > Many of us have started using `pkgdev bugs` to file stabilization > bugs. It works well (Thanks Arthur!) and I encourage everyone to give > it a try. > > Where possible, it files one stabilization bug per package. This makes > arch testers' jobs easier and makes the task easier to automate. > > But sometimes we do want to stabilize packages together. For example > major versions of x11-wm/mutter and gnome-base/gnome-shell are tied > together. If a new mutter is stabilized without the new gnome-shell, > the tree will still be consistent, but emerge -u @world will warn > users that the mutter upgrade is blocked. > > There was some brief discussion on IRC about how to document these > groupings, and two main ideas were suggested: > > - add a field to metadata.xml to specify the group by an arbitrary name. > E.g. > Each package in the group would specify the same value of name="..." > > - maintain the groups in a separate place (similar to portage @sets). > > Can anyone think of particular advantages or disadvantages to one > solution versus the other? Any other (better) ideas? I don't know how widespread the problem is, and how much it can be generalised, but could you perhaps use a virtual, such that stabilisation of the virtual means the deps must be satisfied? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: x11-themes/gtk-engines-quartz
# Fabian Groffen (2023-06-21) # Ancient OSX integration package, not keyworded for any current arch # Removal on 2023-07-21. Bug #908938. x11-themes/gtk-engines-quartz -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Last rites: app-admin/cygwin-rebase
On 26-05-2023 09:06:30 +0200, Fabian Groffen wrote: > # Fabian Groffen (2023-05-26) > # Cygwin package for which keyword/profile support was dropped > # Removal on 2023-06-25. Bug #907194. > app-admin/cygwin-rebase These two packages are appended to mask: dev-libs/pthreads4w sys-devel/parity -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: app-admin/cygwin-rebase
# Fabian Groffen (2023-05-26) # Cygwin package for which keyword/profile support was dropped # Removal on 2023-06-25. Bug #907194. app-admin/cygwin-rebase -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] Remove obsolete FEATURES=force-prefix
Sounds fine to me too. On 22-01-2023 09:16:53 +, James Le Cuirot wrote: > On Sun, 2023-01-22 at 09:18 +0100, Ulrich Müller wrote: > > Signed-off-by: Ulrich Müller > > --- > > bin/dohtml.py| 6 ++ > > bin/eapi.sh | 4 ++-- > > lib/portage/const.py | 3 +-- > > lib/portage/package/ebuild/config.py | 11 +++ > > man/make.conf.5 | 8 +--- > > 5 files changed, 9 insertions(+), 23 deletions(-) > > +1 -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: sys-apps/superiotool
# Fabian Groffen (2022-12-27) # Old SVN version, with open bugs #830031, #770946, #712534, all fixed # in app-admin/coreboot-utils package. (Conflict in #888581) Removal # on 2023-01-26. sys-apps/superiotool -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 1/4] acct-group.eclass: Fix for when building in a rooted prefix (EROOT)
On 07-12-2022 11:52:58 -0500, Mike Gilbert wrote:
> On Wed, Dec 7, 2022 at 4:24 AM James Le Cuirot wrote:
> > The new eclasses also skip the operation if you are root. As that bug report
> > says, running a prefixed system as root is probably unsupported. I was doing
> > this as root into a ROOTed prefix though, which is slightly different.
> > Should
> > we also skip the operation if EPREFIX non-empty? I'll think about it.
>
> I would be in favor of skipping adding users/groups if EPREFIX is
> non-empty, at least as a temporary solution.
>
> If someone presents a use case where adding users to
> ${EROOT}/etc/passwd makes sense, we can revisit it then.
Would have to look if RAP uses this.
@heroxbd do you know if that is used?
Thanks,
Fabian
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-dev] Last rites: sys-apps/baselayout-prefix
# Fabian Groffen (2022-08-06) # superseeded by sys-apps/baselayout, removal in 30 days. bug #836114 sys-apps/baselayout-prefix -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] bin/isolated-functions.sh USERLAND setting
On 28-07-2022 20:11:24 +0200, Fabian Groffen wrote:
> On 28-07-2022 19:57:50 +0200, Arfrever Frehtes Taifersar Arahesis wrote:
> > But this code starts with 'if [[ -z ${USERLAND} ]]', so it should not
> > be run when USERLAND="GNU".
> >
> > USERLAND="GNU" is set in make.defaults, which is parsed by Portage
> > early (at Python side) and these values are passed by Portage to
> > ebuild.sh subprocess.
>
> Hmmm, that would be nice, features/prefix/make.defaults includes setting
> that.
>
> Makes me still wonder why we do this, is it a fallback for when the
> profiles aren't available (no tree yet) or something?
>
> It exports XARGS, which only used in the ___parallel_xargs function.
> That function in turn is used in 2 places. So all this stuff is done to
> add -r to xargs if xargs isn't BSD xargs. Iow --no-run-if-empty, which
> is cool, but apparently not strictly necessary (fugly warning I guess at
> worst). We're already checking xargs to support --max-procs=, so we
> could in the same go check for support of --no-run-if-empty, and drop
> the whole XARGS thing and declutter the code quite a lot :)
Sorry, I grepped wrong. XARGS is used throughout the code, so it needs
to be initialised in isolated-functions.sh. It could be initialised by
simply probing instead of relying on uname though.
As for USERLAND, emerge-webrsync and delta-webrsync appear to use it,
but they retrieve it from portageq, so I guess USERLAND is not really
used in Portage itself.
Fabian
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] bin/isolated-functions.sh USERLAND setting
On 28-07-2022 19:57:50 +0200, Arfrever Frehtes Taifersar Arahesis wrote:
> 2022-07-28 17:47 UTCに、Fabian Groffen は書いた:
> > bin/isolated-functions.sh does the following bit:
> >
> > if [[ -z ${USERLAND} ]] ; then
> >case $(uname -s) in
> >*BSD|DragonFly)
> >export USERLAND="BSD"
> >;;
> >*)
> >export USERLAND="GNU"
> >;;
> >esac
> > fi
> >
> > (after which it uses USERLAND for a single purpose, to export XARGS to
> > either GNU '[g]xargs -r', or BSD 'xargs')
> >
> > This bit is problematic for Prefix, because Prefix may run on *BSD, but
> > use USERLAND=GNU.
>
> But this code starts with 'if [[ -z ${USERLAND} ]]', so it should not
> be run when USERLAND="GNU".
>
> USERLAND="GNU" is set in make.defaults, which is parsed by Portage
> early (at Python side) and these values are passed by Portage to
> ebuild.sh subprocess.
Hmmm, that would be nice, features/prefix/make.defaults includes setting
that.
Makes me still wonder why we do this, is it a fallback for when the
profiles aren't available (no tree yet) or something?
It exports XARGS, which only used in the ___parallel_xargs function.
That function in turn is used in 2 places. So all this stuff is done to
add -r to xargs if xargs isn't BSD xargs. Iow --no-run-if-empty, which
is cool, but apparently not strictly necessary (fugly warning I guess at
worst). We're already checking xargs to support --max-procs=, so we
could in the same go check for support of --no-run-if-empty, and drop
the whole XARGS thing and declutter the code quite a lot :)
Thanks,
Fabian
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] bin/isolated-functions.sh USERLAND setting
Hi,
bin/isolated-functions.sh does the following bit:
if [[ -z ${USERLAND} ]] ; then
case $(uname -s) in
*BSD|DragonFly)
export USERLAND="BSD"
;;
*)
export USERLAND="GNU"
;;
esac
fi
(after which it uses USERLAND for a single purpose, to export XARGS to
either GNU '[g]xargs -r', or BSD 'xargs')
This bit is problematic for Prefix, because Prefix may run on *BSD, but
use USERLAND=GNU. The problem is theoretical at this point,
occasionally people come in and want to use Prefix on *BSD again.
Now I think Gentoo/BSD is theoretical too. It officially is no longer
maintained[1].
So, question here is, do we want to retain this bit of historical
compatibility work (also check things like gsed wrapper), or can it go,
basically removing a potential problem for Gentoo Prefix?
[1] https://wiki.gentoo.org/wiki/Gentoo_FreeBSD
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-dev] Rust is here to eat your Pythonz
On 27-07-2022 13:30:32 +0200, Michał Górny wrote: > On Wed, 2022-07-27 at 12:52 +0200, Fabian Groffen wrote: > > On 26-07-2022 08:42:57 +0200, Michał Górny wrote: > > > Hi, everyone. > > > > > > Just a quick FYI: since Rust is going to be marked stable on the last > > > architecture (sparc) that it's going to support in Gentoo, we're going > > > to start cleaning up old dev-python/cryptography soon. If you don't > > > want Rust, removing cryptography entirely will be your only option. > > > > I believe this isn't always a "don't want", there's systems on which it > > simply is impossible to get Rust (ppc64-linux-musl for example). How > > much does "removing cryptography" impact an Gentoo system wrt > > availability of packages? I see pip in the list, so I expect the Python > > env will be seriously crippled? An estimation of the impact would be > > appreciated here. > > > > Check the wd40 profile, all relevant masks are included there. That helps, thanks! Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Rust is here to eat your Pythonz
On 26-07-2022 08:42:57 +0200, Michał Górny wrote: > Hi, everyone. > > Just a quick FYI: since Rust is going to be marked stable on the last > architecture (sparc) that it's going to support in Gentoo, we're going > to start cleaning up old dev-python/cryptography soon. If you don't > want Rust, removing cryptography entirely will be your only option. I believe this isn't always a "don't want", there's systems on which it simply is impossible to get Rust (ppc64-linux-musl for example). How much does "removing cryptography" impact an Gentoo system wrt availability of packages? I see pip in the list, so I expect the Python env will be seriously crippled? An estimation of the impact would be appreciated here. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] phase-functions: make ED, EROOT read-only
On 25-07-2022 19:43:21 -0400, Mike Gilbert wrote: > On Mon, Jul 25, 2022 at 1:03 PM Fabian Groffen wrote: > > > > bin/phase-functions.sh: make ED and EROOT read-only too > > > > Like D, make ED and EROOT read-only vars. > > Makes sense. https://github.com/gentoo/portage/pull/870 Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] misc-functions: Prefix fixes
On 25-07-2022 19:42:51 -0400, Mike Gilbert wrote: > On Mon, Jul 25, 2022 at 12:47 PM Fabian Groffen wrote: > > > > bin/misc-functions.sh: some Prefix fixes > > > > - ED needs not to exist, whereas D does, so ensure we check for that, > > and create ED if absent, necessary for further checks to succeed > > - use EPREFIX in INSTALL_MASK > > Seems good to me. https://github.com/gentoo/portage/pull/870 -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] world-writable: tune for Prefix
On 25-07-2022 19:39:38 -0400, Mike Gilbert wrote: > On Mon, Jul 25, 2022 at 12:41 PM Fabian Groffen wrote: > > > > bin/install-qa-check.d/90world-writable: include EPREFIX in reports > > > > It is much less confusing and consistent to report full paths including > > the leading EPREFIX. > > Makes sense to me. https://github.com/gentoo/portage/pull/869 thanks -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] multilib-strict: fix for Prefix
On 25-07-2022 19:38:57 -0400, Mike Gilbert wrote:
> On Mon, Jul 25, 2022 at 12:26 PM Fabian Groffen wrote:
> >
> > bin/install-qa-check.d/80multilib-strict: use file/find from Prefix
> >
> > diff --git a/bin/install-qa-check.d/80multilib-strict
> > b/bin/install-qa-check.d/80multilib-strict
> > index afd223250..3db4ecce3 100644
> > --- a/bin/install-qa-check.d/80multilib-strict
> > +++ b/bin/install-qa-check.d/80multilib-strict
> > @@ -1,7 +1,7 @@
> > # Strict multilib directory checks
> > multilib_strict_check() {
> > if has multilib-strict ${FEATURES} && \
> > - [[ -x /usr/bin/file && -x /usr/bin/find ]] && \
> > + [[ -x ${EPREFIX}/usr/bin/file && -x ${EPREFIX}/usr/bin/find ]]
> > && \
> >[[ -n ${MULTILIB_STRICT_DIRS} && -n ${MULTILIB_STRICT_DENY} ]]
> > then
> > rm -f "${T}/multilib-strict.log"
>
> Seems good to me.
https://github.com/gentoo/portage/pull/868
Thanks
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] 80libraries: add support for Darwin targets
On 26-07-2022 04:01:37 +0100, Sam James wrote:
>
>
> > On 26 Jul 2022, at 00:33, Mike Gilbert wrote:
> >
> > On Mon, Jul 25, 2022 at 11:38 AM Fabian Groffen wrote:
> >>
> >> bin/install-qa-check.d/80libraries: support Darwin/Mach-O objects
> >>
> >> Check for dylib on Darwin, so on everything else.
> >>
> >> Signed-off-by: Fabian Groffen
> >>
> >> diff --git a/bin/install-qa-check.d/80libraries
> >> b/bin/install-qa-check.d/80libraries
> >> index 8dc35bb87..a477ec9cb 100644
> >> --- a/bin/install-qa-check.d/80libraries
> >> +++ b/bin/install-qa-check.d/80libraries
> >> @@ -140,7 +140,9 @@ lib_check() {
> >>local abort="no"
> >>local a s
> >>for a in "${ED%/}"/usr/lib*/*.a ; do
> >> - s=${a%.a}.so
> >> + [[ ${CHOST} == *-darwin* ]] \
> >> + && s=${a%.a}.dylib \
> >> + || s=${a%.a}.so
> >
> > I would find this much easier to read if you converted it to an
> > if/else statement instead of chaining && and ||.
>
> Yes, please.
https://github.com/gentoo/portage/pull/867
Thanks
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] doins: fix D check, add EPREFIX check
On 25-07-2022 19:29:35 -0400, Mike Gilbert wrote: > Could you please create a PR at https://github.com/gentoo/portage so > that the CI system can test the changes for this patch series? https://github.com/gentoo/portage/pull/866 Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] doins: fix D check, add EPREFIX check
On 25-07-2022 19:50:41 +0200, Ulrich Mueller wrote:
> >>>>> On Mon, 25 Jul 2022, Fabian Groffen wrote:
>
> > @@ -50,6 +51,16 @@ if [[ ${_E_INSDESTTREE_#${ED}} != "${_E_INSDESTTREE_}"
> > ]]; then
> > __helpers_die "${helper} used with \${D} or \${ED}"
> > exit 1
> > fi
> > +if [[ -n ${EPREFIX} && \
> > + ${_E_INSDESTTREE_#${EPREFIX}} != "${_E_INSDESTTREE_}" ]];
>
> The semicolon is redundant.
removed, thanks
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] emake: explicitly set SHELL
On 26-07-2022 09:03:18 +0200, Florian Schmaus wrote:
> On 26.07.22 05:00, Sam James wrote:
> >> On 25 Jul 2022, at 16:28, Fabian Groffen wrote:
> >>
> >> bin/ebuild-helpers/emake: force SHELL to be set
> >>
[snip]
> >>
> >> diff --git a/bin/ebuild-helpers/emake b/bin/ebuild-helpers/emake
> >> index 60718a2e4..21da85845 100755
> >> --- a/bin/ebuild-helpers/emake
> >> +++ b/bin/ebuild-helpers/emake
> >> @@ -12,7 +12,7 @@
> >> source "${PORTAGE_BIN_PATH}"/isolated-functions.sh || exit 1
> >>
> >> cmd=(
> >> - ${MAKE:-make} ${MAKEOPTS} "$@" ${EXTRA_EMAKE}
> >> + ${MAKE:-make} SHELL="${BASH:-/bin/bash}" ${MAKEOPTS} "$@" ${EXTRA_EMAKE}
> >> )
> >>
> >> if [[ ${PORTAGE_QUIET} != 1 ]] ; then
> >>
> >
> > I don't think I agree with this as it is. Why not just ${EPREFIX}/bin/sh to
> > avoid using
> > an ancient host sh?
>
> I was about to write the same (also using EPREFIX, but EBROOT seems what
> you want, as you figured).
>
> But then I wondered if "make SHELL=$BROOT/bin/sh" wouldn't override
> explicitly set SHELL values in Makefiles. Assume a package has
>
> SHELL = /bin/zsh
>
> in one of its Makefiles. Then emake would reset this to 'sh'. Which
> appears like it could cause build issues.
>
> If this is the case, then I am not sure what we can do about it. It
> appears fragile, if not impossible, to ask 'make' which value for SHELL
> it would assume, so that emake could adjust the path. Another option
> could be that affected packages define a variable in their ebuild, e.g.
> EMAKE_SHELL="zsh", which emake could extend with BROOT before passing
> the resulting value as SHELL to make.
So, I can also envision we drop this patch, and I see if I can patch
make(1) to use $EPREFIX/bin/sh instead of /bin/sh by default. Not sure,
but this would retain the behaviour Portage is doing now for non-Prefix,
and would get the behaviour we want in Prefix.
On an alternative note, there is CONFIG_SHELL (used for setting which shell
to use with configure), which I think in many cases bleeds through to
make, but should there be a MAKE_SHELL perhaps as well? Then the
default would be pretty much ok.
(We never ran into any problems forcing SHELL to bash in Prefix, but
perhaps that's not a representative test for the whole of Gentoo.)
Thanks,
Fabian
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] [PATCH] phase-functions: make ED, EROOT read-only
bin/phase-functions.sh: make ED and EROOT read-only too Like D, make ED and EROOT read-only vars. Signed-off-by: Fabian Groffen diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh index ccf7eeea7..212b19fc1 100644 --- a/bin/phase-functions.sh +++ b/bin/phase-functions.sh @@ -12,7 +12,7 @@ PORTAGE_READONLY_METADATA="BDEPEND DEFINED_PHASES DEPEND DESCRIPTION PDEPEND RDEPEND REPOSITORY RESTRICT SLOT SRC_URI" PORTAGE_READONLY_VARS="D EBUILD EBUILD_PHASE EBUILD_PHASE_FUNC \ - EBUILD_SH_ARGS EMERGE_FROM FILESDIR MERGE_TYPE \ + EBUILD_SH_ARGS ED EMERGE_FROM EROOT FILESDIR MERGE_TYPE \ PM_EBUILD_HOOK_DIR \ PORTAGE_ACTUAL_DISTDIR PORTAGE_ARCHLIST PORTAGE_BASHRC \ PORTAGE_BINPKG_FILE PORTAGE_BINPKG_TAR_OPTS PORTAGE_BINPKG_TMPFILE \ -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH] misc-functions: Prefix fixes
bin/misc-functions.sh: some Prefix fixes
- ED needs not to exist, whereas D does, so ensure we check for that,
and create ED if absent, necessary for further checks to succeed
- use EPREFIX in INSTALL_MASK
Signed-off-by: Fabian Groffen
diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index c8bac08e7..8fcc23588 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -19,6 +19,8 @@ source "${PORTAGE_BIN_PATH}/ebuild.sh" || exit 1
install_symlink_html_docs() {
if ! ___eapi_has_prefix_variables; then
local ED=${D}
+ else
+ [[ ! -d ${ED} && -d ${D} ]] && dodir /
fi
cd "${ED}" || die "cd failed"
# Symlink the html documentation (if DOC_SYMLINKS_DIR is set in
make.conf)
@@ -83,7 +87,7 @@ install_qa_check() {
local EPREFIX= ED=${D}
fi
- cd "${ED}" || die "cd failed"
+ cd "${D}" || die "cd failed"
# Collect the paths for QA checks, highest prio first.
paths=(
@@ -367,7 +718,7 @@ preinst_mask() {
local f x
for f in man info doc; do
if has no${f} ${FEATURES}; then
- INSTALL_MASK+=" /usr/share/${f}"
+ INSTALL_MASK+=" ${EPREFIX}/usr/share/${f}"
fi
done
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] [PATCH] world-writable: tune for Prefix
bin/install-qa-check.d/90world-writable: include EPREFIX in reports
It is much less confusing and consistent to report full paths including
the leading EPREFIX.
Signed-off-by: Fabian Groffen
diff --git a/bin/install-qa-check.d/90world-writable
b/bin/install-qa-check.d/90world-writable
index 820683bd6..90b961a86 100644
--- a/bin/install-qa-check.d/90world-writable
+++ b/bin/install-qa-check.d/90world-writable
@@ -2,7 +2,7 @@
world_writable_check() {
# Now we look for all world writable files.
- local unsafe_files=$(find "${ED}" -type f -perm -2 | sed -e
"s:^${ED}:/:")
+ local unsafe_files=$(find "${ED}" -type f -perm -2 | sed -e
"s:^${D}:/:")
local OLDIFS x prev_shopts=$-
OLDIFS=$IFS
@@ -19,7 +21,7 @@ world_writable_check() {
eqawarn
fi
- local unsafe_files=$(find "${ED}" -type f '(' -perm -2002 -o -perm
-4002 ')' | sed -e "s:^${ED}:/:")
+ local unsafe_files=$(find "${ED}" -type f '(' -perm -2002 -o -perm
-4002 ')' | sed -e "s:^${D}:/:")
if [[ -n ${unsafe_files} ]] ; then
eqawarn "QA Notice: Unsafe files detected (set*id and world
writable)"
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] [PATCH] multilib-strict: fix for Prefix
bin/install-qa-check.d/80multilib-strict: use file/find from Prefix
diff --git a/bin/install-qa-check.d/80multilib-strict
b/bin/install-qa-check.d/80multilib-strict
index afd223250..3db4ecce3 100644
--- a/bin/install-qa-check.d/80multilib-strict
+++ b/bin/install-qa-check.d/80multilib-strict
@@ -1,7 +1,7 @@
# Strict multilib directory checks
multilib_strict_check() {
if has multilib-strict ${FEATURES} && \
- [[ -x /usr/bin/file && -x /usr/bin/find ]] && \
+ [[ -x ${EPREFIX}/usr/bin/file && -x ${EPREFIX}/usr/bin/find ]] && \
[[ -n ${MULTILIB_STRICT_DIRS} && -n ${MULTILIB_STRICT_DENY} ]]
then
rm -f "${T}/multilib-strict.log"
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] [PATCH] 80libraries: add support for Darwin targets
bin/install-qa-check.d/80libraries: support Darwin/Mach-O objects
Check for dylib on Darwin, so on everything else.
Signed-off-by: Fabian Groffen
diff --git a/bin/install-qa-check.d/80libraries
b/bin/install-qa-check.d/80libraries
index 8dc35bb87..a477ec9cb 100644
--- a/bin/install-qa-check.d/80libraries
+++ b/bin/install-qa-check.d/80libraries
@@ -140,7 +140,9 @@ lib_check() {
local abort="no"
local a s
for a in "${ED%/}"/usr/lib*/*.a ; do
- s=${a%.a}.so
+ [[ ${CHOST} == *-darwin* ]] \
+ && s=${a%.a}.dylib \
+ || s=${a%.a}.so
if [[ ! -e ${s} ]] ; then
s=${s%usr/*}${s##*/usr/}
if [[ -e ${s} ]] ; then
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-portage-dev] [PATCH] emake: explicitly set SHELL
bin/ebuild-helpers/emake: force SHELL to be set
On Prefix systems /bin/sh can be anything, including very ancient. So
ensure we're running with bash, since that's what Gentoo Linux is
expecting /bin/sh to be (by default, at least).
Provide a fallback for the (near impossible) case that we use a bash
that doesn't set BASH, or when we don't use bash at all. This is not
expected, though, as we explicitly require bash throughout all Portage,
so we don't really care about using a non-Prefixed one, for this really
shouldn't happen.
Signed-off-by: Fabian Groffen
diff --git a/bin/ebuild-helpers/emake b/bin/ebuild-helpers/emake
index 60718a2e4..21da85845 100755
--- a/bin/ebuild-helpers/emake
+++ b/bin/ebuild-helpers/emake
@@ -12,7 +12,7 @@
source "${PORTAGE_BIN_PATH}"/isolated-functions.sh || exit 1
cmd=(
- ${MAKE:-make} ${MAKEOPTS} "$@" ${EXTRA_EMAKE}
+ ${MAKE:-make} SHELL="${BASH:-/bin/bash}" ${MAKEOPTS} "$@" ${EXTRA_EMAKE}
)
if [[ ${PORTAGE_QUIET} != 1 ]] ; then
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] usage of /bin/bash in shebangs
On 25-07-2022 08:09:02 -0700, Zac Medico wrote: > On 7/24/22 23:17, Fabian Groffen wrote: > > On 24-07-2022 13:58:31 -0700, Zac Medico wrote: > >> On 7/24/22 12:29, Fabian Groffen wrote: > >>> Hi, > >>> > >>> Quick question, I noticed that portage uses /bin/bash hardcoded in > >>> shebang of scripts, while it uses /usr/bin/env python for python > >>> executable files. > >>> > >>> Is there anything against using /usr/bin/env bash for shell scripts? > >>> Changing this would help for Prefix. > >> > >> > >> I can't think of any reason not to do this. > > > > Do you want to see a patch, or is it OK to push this myself? > > It's OK to just push this yourself. Thanks! Pushed, thanks, b716e3591 -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH] doins: fix D check, add EPREFIX check
bin/ebuild-helpers/doins: fix D check, add EPREFIX check
ED = D/EPREFIX, so checking for ED includes EPREFIX, which when this is
absent fails to check for D. Simply check for D instead, which catches
both the case for D and ED.
Add check for usage of EPREFIX, like for using D with helpers.
Signed-off-by: Fabian Groffen
diff --git a/bin/ebuild-helpers/doins b/bin/ebuild-helpers/doins
index 24fe48121..4315a038f 100755
--- a/bin/ebuild-helpers/doins
+++ b/bin/ebuild-helpers/doins
@@ -42,7 +42,7 @@ if ! ___eapi_has_prefix_variables; then
export ED="${D}"
fi
-if [[ ${_E_INSDESTTREE_#${ED}} != "${_E_INSDESTTREE_}" ]]; then
+if [[ ${_E_INSDESTTREE_#${D}} != "${_E_INSDESTTREE_}" ]]; then
__vecho "---" 1>&2
__vecho "You should not use \${D} or \${ED} with helpers." 1>&2
__vecho " --> ${_E_INSDESTTREE_}" 1>&2
@@ -50,6 +51,16 @@ if [[ ${_E_INSDESTTREE_#${ED}} != "${_E_INSDESTTREE_}" ]];
then
__helpers_die "${helper} used with \${D} or \${ED}"
exit 1
fi
+if [[ -n ${EPREFIX} && \
+ ${_E_INSDESTTREE_#${EPREFIX}} != "${_E_INSDESTTREE_}" ]];
+then
+ __vecho "---" 1>&2
+ __vecho "You should not use \${EPREFIX} with helpers." 1>&2
+ __vecho " --> ${_E_INSDESTTREE_}" 1>&2
+ __vecho "---" 1>&2
+ __helpers_die "${helper} used with \${EPREFIX}"
+ exit 1
+fi
if ___eapi_doins_and_newins_preserve_symlinks; then
DOINS_ARGS+=( --preserve_symlinks )
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] usage of /bin/bash in shebangs
On 24-07-2022 13:58:31 -0700, Zac Medico wrote: > On 7/24/22 12:29, Fabian Groffen wrote: > > Hi, > > > > Quick question, I noticed that portage uses /bin/bash hardcoded in > > shebang of scripts, while it uses /usr/bin/env python for python > > executable files. > > > > Is there anything against using /usr/bin/env bash for shell scripts? > > Changing this would help for Prefix. > > > I can't think of any reason not to do this. Do you want to see a patch, or is it OK to push this myself? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-portage-dev] usage of /bin/bash in shebangs
Hi, Quick question, I noticed that portage uses /bin/bash hardcoded in shebang of scripts, while it uses /usr/bin/env python for python executable files. Is there anything against using /usr/bin/env bash for shell scripts? Changing this would help for Prefix. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] proposal
On 06-07-2022 15:50:30 +0200, Michał Górny wrote: > On Mon, 2022-07-04 at 16:19 +0200, Florian Schmaus wrote: > > I'd like to propose a new metadata XML element for packages: > > > > > > > > Maintainers can signal to other developers (and of course contributors > > in general) that they are happy with others to make changes to the > > ebuilds without prior consultation of the maintainer. > > I don't think adding such an element is a good idea. In my opinion, > this will strengthen the assumption that "unless otherwise noted, you > don't dare touch anything" (even though that's not your goal). "Common > sense" should really be good enough for almost everything. Right, "common sense". The problem with that one, is that "common" is not as "common" as you think it is. Ask a bunch of people, and you'll find that what they consider "common" isn't the same. So, if you do this, then please define clearly what you think is OK. For example for me: - feel free to add patches necessary for operation - feel free to fix constructs (like an if or case that should apply for something else/different/mode) - feel free to fix typos - please do not needlessly change style: if you do not "maintain" the ebuild, respect the style of the maintainer, so only add the changes you need, keep it minimal, respect the original even though you don't like it (and don't use QA as an excuse to change style) - when you make a change, make sure you check for bugs in the following days, so you can cleanup yourself should there be fallout > I mean, I do realize that 10 years ago, in the golden years of Gentoo, > it was considered normal for developers to be like "my package, my > fortress, don't you dare add systemd unit or I will retire" but today I > think we're aiming for a more welcoming developer base, and I think > we're actually going in that direction. What I'm afraid is that some > people will use this as an excuse to push back once again. Not sure I have the same memories of how it used to be 10 years ago. I actually think it is pretty much the same as it is now, as it was then. Different and fewer people, but still different preferences/opinions/common sense. > Can you really think of a case when common sense really, really wouldn't > work? I mean, sure, we all make mistakes but we should be able to learn > from them and do better next time. This also implies package > maintainers learning that they're not the only people who will ever > touch the package in question and starting to document the pitfalls. Honestly, I've never been a fan of "maintainership". It basically is some sort of "sign" that says "beware for the dog, stay away". However, it's true that sometimes people really delve into a package, and thereby know very much how it works, and what you should/should not do. Something like LLVM is a good example, maybe. Anyway, in such situation, I think extreme care should be taken by non-maintainers. Dunno how to best indicate that, and/or if that's feasible -- like you said, it quickly ends up being an excuse for declaring a package to be off-limits. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] last rites: dev-python/thrift
Dropped this mask again, I missed a dep in the tree. Fabian On 01-04-2022 13:53:24 +0200, Fabian Groffen wrote: > # Fabian Groffen (2022-04-01) > # unused package, not enabling tests, bug #796830 > # Removal in 30 days > dev-python/thrift -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] last rites: dev-python/thrift
On 01-04-2022 16:59:40 +0500, Anna Vyalkova wrote: > On 2022-04-01 13:53, Fabian Groffen wrote: > > # Fabian Groffen (2022-04-01) > > # unused package, not enabling tests, bug #796830 > > # Removal in 30 days > > dev-python/thrift > > It has a reverse dependency in GURU > dev-python/jaeger-client > > It will be needed if OpenStack packages are restored some day. Can we move/copy the package to GURU? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] last rites: dev-python/thrift
# Fabian Groffen (2022-04-01) # unused package, not enabling tests, bug #796830 # Removal in 30 days dev-python/thrift -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Policy on conditional patching
Hi, On 28-03-2022 13:05:03 +0200, Thomas Bracht Laumann Jespersen wrote: > Hi! > > I've been working on a new section in the devmanual regarding conditional > patching. In a PR [0] Sam suggested adding a section to clarify that > conditional > patching should be avoided, because it can quickly become a maintenance and > testing burden. > > The devmanual PR is here: [1] > > Ulrich points out that conditional patching is actually suggested elsewhere, > and > that actively discouraging conditional patching is a policy change, which > should > be brought up on this list. So this is what I'm doing now. He also pointed > out a > comment in eutils.eclass re [2] (the comment now lives in epatch.eclass). > > The overall policy (proposal, I guess) is something like: > > * Patches should be written such that they affect behaviour correctly based > on >e.g. build time definitions or config options, rather than USE flags >directly. > > * They should be applied unconditionally, so that, for version bumps and > other >development happening with a package, any failure to apply the patch will > be >caught by the developer. > > * Patching is ideally only done to make the package in question build > properly, >and should not be done to modify the runtime behaviour of the package. > (This >is what USE flags and configuration options of the package are for.) > > Sam made a specific point re musl: "for e.g. musl patches, we want a portable > fix, not a hack which is only applied for musl" > > Feedback on this very welcome. I'm grappling a bit with the exact wording to > go > for, so input on that is also appreciated. > > I think my question to this list is: Should it be policy that conditional > patching is to be avoided? We really don't want conditional patches, but I from my experience reality is that sometimes you have to. Therefore this should remain possible. I have no problems with highly discouraging conditional patching. About your other points, I think they are kind of debatable. Gentoo wants to be close to upstream, but with your set of rules, one can't e.g. add hpn patches to ssh, which would be really silly, as the point of Gentoo is that since you build from source, you can actually apply such patches, conditionally if they don't have a means to be disabled. In other words, I think the gist of your points is to be in an ideal world, but unfortunately reality is far from it. That said, repeating myself, nothing wrong with discouraging quick 'n' dirty, for as long as it remains a big fat warning and advice. My €0.02 Fabian > [0]: https://github.com/gentoo/gentoo/pull/24709#discussion_r832361402 > [1]: https://github.com/gentoo/devmanual/pull/281 > [2]: > https://gitweb.gentoo.org/archive/repo/gentoo-2.git/tree/eclass/eutils.eclass?id=50e8beda904760c773e5c67fdfe8242255e13495#n175 > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: www-apps/agendav
# Fabian Groffen (2022-03-16) # little activity upstream, doesn't work with PHP 8.0 # Removal in 30 days www-apps/agendav -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Changing the VDB format
Hi, I've recently been thinking about this too. On 13-03-2022 18:06:21 -0700, Matt Turner wrote: > The VDB uses a one-file-per-variable format. This has some > inefficiencies, with many file systems. For example the 'EAPI' file > that contains a single character will consume a 4K block on disk. > I recommend json and think it is the best choice because: [snip] > - json provides the smallest on-disk footprint > - json is part of Python's standard library (so is yaml, and toml will > be in Python 3.11) > - Every programming language has multiple json parsers > -- lots of effort has been spent making them extremely fast. I would like to suggest to use "tar". The reason behind this is a bit convoluted, but I try to be as clear and sound as I can: - "new style" bin-packages use tar too - tar-file allows to keep all individual files/members, e.g. for legacy tools to unpack and look at the VDB that way - tar-file allows streaming, so single file read, for efficient retrieval - single tar-file for entire VDB, allows to make it "atomic", one can modify tar archives lateron to add new vdb entries, or perform updates -- again, without inplace (e.g. memory backing) this could be done atomic) - tar-file could be used for (rsync) tree metadata (md5-cache) in the same way, e.g. re-use streaming approach, or unpack for legacy tools - tar-file could be used for Packages file, instead of flat file with keys, basically just write VDB entries with some additional keys, very similar in practise. - tar-files are slightly easier to manage from command line, tools to do so exist for a long time and are installed. (jq isn't pulled in by @system these days, I think) - tar-files can easily (optionally) be compressed retaining streaming abilities (this is for these usages very likely to pay off), a much higher dictionary benefit for a single tar vs many files. - single tar-file is much more efficient to GPG-sign (which would allow some securing of the VDB, not sure if useful though) - going back to the first point, vdb entry from binary package could simply be dropped into the vdb tar, and vice-versa - going back to metadata, dep-resolving could simply load the entire system available/installed packages with two reads in memory (if it has enough of that -- pretty common these days), which should allow for vast speedups, especially on cold(ish) filesystems. > I think we would have a significant time period for the transition. I > think I would include support for the new format in Portage, and ship > a tool with portage to switch back and forth between old and new > formats on-disk. Maybe after a year, drop the code from Portage to > support the old format? Here I believe that with tar-format, initially code could be written to instead of accessing a file directly, it could open up the tar-file, locate the member it needs, and then retrieve that instead. This is a bit naive, but probably sort of managable, and allows to having a switch that specifies which format to write. It's easy to detect which form you have automatically. E.g. nothing has to change for users unless they actively make a change for it. Like you, I think the main reason for doing this should be performance, basically allowing faster operations. I feel though that we should aim to use a single solution to maintain a number of "trees" that we have: metadata, vdb, Packages/binpkgs, for they all seem to exhibit a similar (IO) behaviour when being employed. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Re: Deprecating repoman
Just wondering, is there a "migration guide" or something? I've never used pkg* since joining in 2005. I can derive some things from the first look at the below commit, but an "expert opinion" to just map the standard things from repoman to appropriate commands would be nice. Are those pkg* keyworded *everywhere*, by the way? Thanks, Fabian On 11-03-2022 13:19:34 -0800, Matt Turner wrote: > I've filed a PR against devmanual.git to remove references to repoman > and replace them with references to pkgdev where appropriate. Most > everywhere already had pkgdev/pkgcheck text in place so there wasn't > much to do. See: https://github.com/gentoo/devmanual/pull/274 > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: sys-apps/tapi
# Fabian Groffen (2022-02-27) # Masked for removal, needs updates, significant amount of work, no # Clang toolchain available to test with # Removal on 2022-03-29. Bug #834306 sys-apps/tapi -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] last-rite: dev-perl/Mac-Pasteboard
# Fabian Groffen (2022-01-29) # Fails to compile with GCC on macOS. No revdeps. # Removal on 2022-02-28. Bug #832309. dev-perl/Mac-Pasteboard -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust
On 21-01-2022 11:29:54 +0100, Peter Böhm wrote: > Dear Developers, > > I like your goal to make Gentoo more user-friendly but Gentoo is a source > based distribution and I dont like binary versions as a default. My question > is: > > Who has problems with "big" packages like rust or firefox ? > > Only User which doesnt know there is a binary version. So, in every case we > need to describe it in our AMD64 handbook. > > Am Freitag, 21. Januar 2022, 10:22:14 CET schrieb Mart Raudsepp: > > Anyhow, my vote is to default to rust-bin - people can easily be told > > to move to dev-lang/rust at their convenience and then explicitly > > depclean rust-bin. > > I am dreaming about another solution where this is not needed: > > In our /etc/portage/make.conf we can have a new: > > MAKEBIN="rust firefox" > > ... resulting in an automatic switch to the binary version of all included > packages ... of course this is also as recommendation in our AMD64 handbook > (with a clue to delete it if not desired). or ... if we could have Portage check the requirements for building a package, and if it cannot be met, that it tries to resolve the || case, which would be the -bin variant in this case. Not sure if the information is available to Portage at dependency resolution time though. Fabian > > Kind reagards, > Peter > > > > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] portage.output: Replace darkblue colors with teal
On 04-12-2021 10:24:23 +0100, Michał Górny wrote: > On Sat, 2021-12-04 at 10:15 +0100, Fabian Groffen wrote: > > On 04-12-2021 10:13:09 +0100, Michał Górny wrote: > > > On Sat, 2021-12-04 at 09:56 +0100, Fabian Groffen wrote: > > > > Why don't you change your color.map instead of changing the default for > > > > everyone? > > > > > > Why should we keep a stupid default? Should we optimize Gentoo for > > > people who don't want to be able to read Portage's output? > > > > You're assuming everyone uses the terminal in the way you do. I simply > > don't think that's how the world looks like. > > On the other hand, you're assuming that everyone uses the terminal > in the way you do. It eludes me how you came to that conclusion. > > No need for calling things stupid, IMO. > > Using dark blue on black background is stupid. ... then don't use black background or dark blue text? Now, if you would make a supported claim that all terminals we install use a black background by default, your change becomes more valid. However, we then still don't know if people leave that default or use something else, but we could make some educated guess about the amount of people not changing the default. My point, because I think this wasn't clear to you, is and always was, how many people is this change going to be disruptive to. And should we make a hint to users when they install this version of Portage that they can revert/change this by altering color.map (and how)? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] portage.output: Replace darkblue colors with teal
On 04-12-2021 10:13:09 +0100, Michał Górny wrote: > On Sat, 2021-12-04 at 09:56 +0100, Fabian Groffen wrote: > > Why don't you change your color.map instead of changing the default for > > everyone? > > Why should we keep a stupid default? Should we optimize Gentoo for > people who don't want to be able to read Portage's output? You're assuming everyone uses the terminal in the way you do. I simply don't think that's how the world looks like. No need for calling things stupid, IMO. -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] portage.output: Replace darkblue colors with teal
Why don't you change your color.map instead of changing the default for
everyone?
Thanks,
Fabian
On 04-12-2021 05:48:49 +0100, Michał Górny wrote:
> The "darkblue" color is often barely visible on dark terminals which
> makes reading emerge output really hard (I basically have to copy-paste
> it a lot in order to be able to read it at all). Replace it with teal
> that does not seem to have any significant use in the output.
>
> Signed-off-by: Michał Górny
> ---
> lib/portage/output.py | 6 +++---
> man/color.map.5 | 6 +++---
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
>
> The before-after screenshot can be found on the PR:
> https://github.com/gentoo/portage/pull/776
>
>
> diff --git a/lib/portage/output.py b/lib/portage/output.py
> index 42f487f8a..77375b012 100644
> --- a/lib/portage/output.py
> +++ b/lib/portage/output.py
> @@ -157,7 +157,7 @@ _styles["UNMERGE_WARN"] = ("red",)
> _styles["SECURITY_WARN"] = ("red",)
> _styles["MERGE_LIST_PROGRESS"] = ("yellow",)
> _styles["PKG_BLOCKER"] = ("red",)
> -_styles["PKG_BLOCKER_SATISFIED"] = ("darkblue",)
> +_styles["PKG_BLOCKER_SATISFIED"] = ("teal",)
> _styles["PKG_MERGE"] = ("darkgreen",)
> _styles["PKG_MERGE_SYSTEM"] = ("darkgreen",)
> _styles["PKG_MERGE_WORLD"] = ("green",)
> @@ -165,8 +165,8 @@ _styles["PKG_BINARY_MERGE"] = ("purple",)
> _styles["PKG_BINARY_MERGE_SYSTEM"] = ("purple",)
> _styles["PKG_BINARY_MERGE_WORLD"] = ("fuchsia",)
> _styles["PKG_UNINSTALL"] = ("red",)
> -_styles["PKG_NOMERGE"] = ("darkblue",)
> -_styles["PKG_NOMERGE_SYSTEM"] = ("darkblue",)
> +_styles["PKG_NOMERGE"] = ("teal",)
> +_styles["PKG_NOMERGE_SYSTEM"] = ("teal",)
> _styles["PKG_NOMERGE_WORLD"] = ("blue",)
> _styles["PROMPT_CHOICE_DEFAULT"] = ("green",)
> _styles["PROMPT_CHOICE_OTHER"] = ("red",)
> diff --git a/man/color.map.5 b/man/color.map.5
> index 288bf7fbd..92a1baa91 100644
> --- a/man/color.map.5
> +++ b/man/color.map.5
> @@ -40,7 +40,7 @@ Defines color used for numbers indicating merge progress.
> \fBPKG_BLOCKER\fR = \fI"red"\fR
> Defines color used for unsatisfied blockers.
> .TP
> -\fBPKG_BLOCKER_SATISFIED\fR = \fI"darkblue"\fR
> +\fBPKG_BLOCKER_SATISFIED\fR = \fI"teal"\fR
> Defines color used for satisfied blockers.
> .TP
> \fBPKG_MERGE\fR = \fI"darkgreen"\fR
> @@ -63,10 +63,10 @@ package.
> Defines color used for world packages planned to be merged using a binary
> package.
> .TP
> -\fBPKG_NOMERGE\fR = \fI"darkblue"\fR
> +\fBPKG_NOMERGE\fR = \fI"teal"\fR
> Defines color used for packages not planned to be merged.
> .TP
> -\fBPKG_NOMERGE_SYSTEM\fR = \fI"darkblue"\fR
> +\fBPKG_NOMERGE_SYSTEM\fR = \fI"teal"\fR
> Defines color used for system packages not planned to be merged.
> .TP
> \fBPKG_NOMERGE_WORLD\fR = \fI"blue"\fR
> --
> 2.34.1
>
>
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] [RFC] LTS branch of Portage
In all fairness, there haven't been that much incidents with Portage in the past under Zac's supervision, isn't it a bit overkill to bureaucratise the release model just after one incident? It appears to me that changes to Portage need to be considered very carefully, always, since it affects everyone. Thanks, Fabian On 05-10-2021 10:31:40 +0200, Michał Górny wrote: > Hi, everyone. > > I've been thinking about this for some time already, and the recent > FILESDIR mess seems to confirm it: I'd like to start a more stable LTS > branch of Portage. > > Roughly, the idea is that: > > - master becomes 3.1.x, and primary development happens there > > - 3.0.x becomes the LTS branch and only major bugfixes are backported > there > > As things settle down in the future, master would become 3.2.x, 3.1.x > would become LTS, 3.0.x will be discontinued and so on. > > WDYT? > > -- > Best regards, > Michał Górny > > > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
On 05-10-2021 09:35:32 +0200, Michał Górny wrote: > On Thu, 2021-09-30 at 09:18 +0200, Fabian Groffen wrote: > > > > Final question, am I understanding correctly that normal lines are not > > > > prefixed with something? Would it be, for consistency, alignment, and > > > > certainty of selecting rows something to use a prefix for those lines > > > > too (assuming they aren't at this point)? > > > > > > I don't know, we've never done that. I suppose it would be possible but > > > it is even more controversial and unlike the proposed changes, it would > > > actually require mangling the process output. > > > > If I remember correctly, Portage already does. In which case, doing > > this (even if it were adding leading spaces) would not be that much > > work? > > > > I'm afraid this is not that simple. You need to account for all escape > sequences that can affect editing already output data including clean > handling of line wrapping. In the end, we'd have to have a complete > detachtty-class terminal emulator inside Portage. Fair enough, thanks for looking into it. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
On 02-10-2021 23:03:56 -0400, Ionen Wolkens wrote: > On Sat, Oct 02, 2021 at 10:53:37PM -0400, Ionen Wolkens wrote: > > Guess there's a lot of other options that could be considered as well. > > > > --- files > > >>> text > > * current, it wasn't aligned now that I look at it again > > (relying only on color to convey type feels clearly wrong to me) > > > > --- files > > >>>> text > > [QA] new based on current PR > > > > >>> text > > [QA] aligned 1 character further, maybe skipping changing >>> is fine? > > (then again that it's further is what threw me off at first) > > > > >>> text > > QA* similar to before, but aligned using only 3 chars > > > > >>> text > > [Q] kinda more obscure but it can work > > > > Guess should also add these: > >>> text > Q* Notice: > E* Some error happened > (closest to before by making use of the former leading space, thus > no alignment changes) FWIW, I like this one. Perhaps even with lowercase make[4]: leaving directory src q* soname lacks version e* failed to die Fabian > > >>> text > QA Notice: > EE Some error happened > (at least clearer than Q* Notice, but unsure about no separator.. guess > it could work?) > > > >>>> text > > QA* probably closest to how it was before alignment-wise, but meh at 4 > > > > > >>> message > > QA> not convinced about this one, but throwing it here anyway > > (other characters could be considered as well) > > > > Maybe a poll of some kind may help, personally undecided on what I > > like better beside agreeing that a change is needed. > > > -- > ionen -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
On 30-09-2021 08:44:33 +0200, Michał Górny wrote: > On Thu, 2021-09-30 at 08:40 +0200, Fabian Groffen wrote: > > Hi, > > > > Would it be possible to have some switch (e.g. --style=legacy) that > > controls this new vs. the old behaviour? Would perhaps allow > > applications that parse the output to work via setting this in the > > global opts. > > Patches welcome. It shouldn't be hard, my commit shows which files need > to be edited to alter the prefixes and how to pass them into ebd. I see. > > In addition, much like the colour map, how do you see this change in > > light of eclasses, init-scripts, etc. that also use the same scheme as > > Portage at the moment? Would you expect to change those too at some > > point? > > Eclasses are supposed to use standard einfo, elog... functions, so they > should just work™. If someone's reinventing the wheel, it's not my > problem. > > Init scripts aren't supposed to be used inside the PM, so that's out of > scope. I was just referring to the overall "feel" of Gentoo, which your work changes. It is ok that you don't plan on doing anything there. > > Final question, am I understanding correctly that normal lines are not > > prefixed with something? Would it be, for consistency, alignment, and > > certainty of selecting rows something to use a prefix for those lines > > too (assuming they aren't at this point)? > > I don't know, we've never done that. I suppose it would be possible but > it is even more controversial and unlike the proposed changes, it would > actually require mangling the process output. If I remember correctly, Portage already does. In which case, doing this (even if it were adding leading spaces) would not be that much work? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change
Hi, Would it be possible to have some switch (e.g. --style=legacy) that controls this new vs. the old behaviour? Would perhaps allow applications that parse the output to work via setting this in the global opts. In addition, much like the colour map, how do you see this change in light of eclasses, init-scripts, etc. that also use the same scheme as Portage at the moment? Would you expect to change those too at some point? Final question, am I understanding correctly that normal lines are not prefixed with something? Would it be, for consistency, alignment, and certainty of selecting rows something to use a prefix for those lines too (assuming they aren't at this point)? Thanks, Fabian On 28-09-2021 17:36:25 +0200, Michał Górny wrote: > Hi, everyone. > > I know I'm going to regret asking this... but I've prepared a change to > the Portage output format and I think it asks for a wider discussion > than internally in Portage team. > > The primary problem with the current output format is that different > kinds of messages differ only in color. This makes them > indistinguishable without colors and hard to grep. ago's been asking > for a better way to grep for QA warnings and this is pretty much what > motivated me to do this. > > The proposed new format distinguished message types both using colors > and strings. This is roughly inspired by Xorg logs. For example, > instead of: > > * some message > * other message > * hell if i know what this is > > You get: > > [WW] some message > [EE] other message > [QA] hell if i know what this is > > I've also added more colors to explicitly distinguish einfo from elog, > and ewarn from eqawarn. Then, I've replaced most of '>>>' and '!!!' > used by Portage with four-character versions to keep messages aligned. > The 'zings' used for merging files remain three-character, so now it's > easily possible to distinguish messages from installed file list. > > The PR doing this is: https://github.com/gentoo/portage/pull/759 > > Example screenshot: > https://user-images.githubusercontent.com/110765/135119090-16e9599d-1b0f-41b8-a965-a55577183ffd.png > > -- > Best regards, > Michał Górny > > > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] In what phase are file "merged"?
On 23-06-2021 08:47:58 +0200, Ulrich Mueller wrote: > >>>>> On Wed, 23 Jun 2021, Joakim Tjernlund wrote: > > > In PMS 9.2 is says: > > The call order for upgrading, downgrading or reinstalling a package is: > > > pkg_pretend (only for EAPIs listed in table 9.2), which is called > > outside of the normal call order process. > > pkg_setup > > src_unpack > > src_prepare (only for EAPIs listed in table 9.3) > > src_configure (only for EAPIs listed in table 9.4) > > src_compile > > src_test (except if RESTRICT=test) > > src_install > > pkg_preinst > > pkg_prerm for the package being replaced > > pkg_postrm for the package being replaced > > pkg_postinst > > > It does not say where in this list new files merged and old files > > unmerged, can anyone enlighten me? > > It's somewhat hidden, but it's there: > https://projects.gentoo.org/pms/8/pms.html#x1-950009.1.10 > >9.1.10 pkg_preinst >... immediately before merging the package to the live filesystem. ... > >9.1.11 pkg_postinst >... immediately after merging the package to the live filesystem. ... Aha, so does this mean pkg_prerm and pkg_postrm are run with replacing package in place, e.g. if they refer to scripts installed by the replaced package they may no longer exist or be the same? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] KeyError: 'ED' ?
On 09-06-2021 14:19:38 -0400, Mike Gilbert wrote: > On Thu, May 6, 2021 at 6:54 AM Joakim Tjernlund > wrote: > > > > On Wed, 2021-05-05 at 16:47 +, Joakim Tjernlund wrote: > > > I am upgrading an old portage(2.3.76) to 3.0.18 using qmerge(binary pkg) > > > and I get this: > > > qmerge -OK sys-apps/portage > > > [R] sys-apps/portage-3.0.18 > > > * Checking for suitable kernel configuration options... > > > [ ok ] > > > * Using python3.8 in global scope > > > FEATURES variable contains unknown value(s): disabled > > > Traceback (most recent call last): > > > File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main > > > return _run_code(code, main_globals, None, > > > File "/usr/lib/python3.8/runpy.py", line 87, in _run_code > > > exec(code, run_globals) > > > File > > > "/var/tmp/qmerge/sys-apps/portage-3.0.18/image/usr/lib/python3.8/site-packages/portage/_compat_upgrade/default_locations.py", > > > line 93, in > > > main() > > > File > > > "/var/tmp/qmerge/sys-apps/portage-3.0.18/image/usr/lib/python3.8/site-packages/portage/_compat_upgrade/default_locations.py", > > > line 63, in main > > > config_path = os.path.join(os.environ['ED'], > > > GLOBAL_CONFIG_PATH.lstrip(os.sep), 'make.globals') > > > File "/usr/lib/python3.8/os.py", line 675, in __getitem__ > > > raise KeyError(key) from None > > > KeyError: 'ED' > > > > > > so portage fails to install properly, I am guessin this is becuse I have > > > an old portage to begin with? > > > Can it be fixed? > > > > The error goes away when I do: ED=/ qmerge -OK sys-apps/portage > > Does that mean that it is qmerge(aka portage-utils) that needs to set ED > > during merge? which PHASES ? > > According to PMS, a package manager must define ED in src_install, > pkg_preinst, and pkg_postinst. Perhaps qmerge should use export for these vars. In any case this seems tracked in bug 792273. -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] last-rite: app-admin/diamond
# Fabian Groffen (2021-05-24) # Hack on hack no longer sustainable # - seemingly dead upstream # - no (official) Python 3 support # removal in 30 days, bug: https://bugs.gentoo.org/791613 migrate to something like app-metrics/collectd using the the write_graphite plugin. -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [News item review] Exim >=4.94 transports: tainted not permitted
On 06-05-2021 15:01:33 +0200, Andreas K. Huettel wrote: > > Unfortunately there is not much documentation on "tainted" data for > > Exim[1], and to resolve this, non-official sources need to be used, > > such as [2] and [3]. > > This is a safety mechanism that is part of Perl (essentially a way of > tracking data that is derived from "insecure" sources). > > So it probably would make sense to at least point towards that concept > in Perl. I think the concept is clear to most from the descriptions one can find. The big problem however is the solution, how to fix one's configuration. Luckily it seems people find their way to Exim's bugtracker to get help there. Thanks for the suggestion though, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] [News item review v2] Exim >=4.94 transports: tainted not permitted
Title: Exim>=4.94 transports: tainted not permitted Author: Fabian Groffen Posted: 2021-05-?? Revision: 1 News-Item-Format: 2.0 Display-If-Installed: mail-mta/exim The Message Transfer Agent Exim disallows tainted variables in transport configurations since version 4.94. Existing exim.conf configurations in /etc/exim need to be reviewed for breakage prior to upgrading to >=mail-mta/exim-4.94 to avoid error conditions at runtime. Since the release of Exim-4.94, transports refuse to use tainted data in constructing a delivery location. If you use this in your transports, your configuration will break, causing errors and possible downtime. Particularly, the use of $local_part in any transport, should likely be updated with $local_part_data. Check your local_delivery transport, which historically used $local_part. Unfortunately there is not much documentation on "tainted" data for Exim[1], and to resolve this, non-official sources need to be used, such as [2] and [3]. [1] https://lists.exim.org/lurker/message/20201109.222746.24ea3904.en.html [2] https://mox.sh/sysadmin/tainted-filename-errors-in-exim-4.94/ [3] https://jimbobmcgee.wordpress.com/2020/07/29/de-tainting-exim-configuration-variables/ -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [News item review] Exim >=4.94 transports: tainted not permitted
On 02-05-2021 12:23:30 +0200, Ulrich Mueller wrote: > >>>>> On Sun, 02 May 2021, Fabian Groffen wrote: > > > Title: Exim >=4.94 disallows tainted variables in transport configurations > > Title is too long (GLEP 42 allows 50 chars max). ah, missed that > I have no idea what this news item is trying to tell me. But I don't use > Exim, so probably that's the reason. :) Maybe mention at least that Exim > is a mailer? Fair point. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] [News item review] Exim >=4.94 transports: tainted not permitted
Title: Exim >=4.94 disallows tainted variables in transport configurations Author: Fabian Groffen Posted: 2021-05-?? Revision: 1 News-Item-Format: 2.0 Display-If-Installed: mail-mta/exim Since the release of Exim-4.94, transports refuse to use tainted data in constructing a delivery location. If you use this in your transports, your configuration will break, causing errors and possible downtime. Particularly, the use of $local_part in any transport, should likely be updated with $local_part_data. Check your local_delivery transport, which historically used $local_part. Unfortunately there is not much documentation on "tainted" data for Exim[1], and to resolve this, non-official sources need to be used, such as [2] and [3]. [1] https://lists.exim.org/lurker/message/20201109.222746.24ea3904.en.html [2] https://mox.sh/sysadmin/tainted-filename-errors-in-exim-4.94/ [3] https://jimbobmcgee.wordpress.com/2020/07/29/de-tainting-exim-configuration-variables/ -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Packages up for grabs: x11-misc/xbindkeys
Jeroen was sadly removed from the project. You go ahead and take it. Fabian On 22-02-2021 07:49:55 +0200, Jaco Kroon wrote: > Hi, > > Looks like Jeroen was the person mostly committing on this, Jeroen - are > you intending to keep maintaining this? > > The PR is legit but incomplete (it merely adds the patch, not use it). > > The patch has been merged upstream in 2014 already, instead we should > stable 1.8.7. > > I'll adjust the referenced bug referenced by the PR accordingly. > > Kind Regards, > Jaco > > On 2021/02/21 05:08, Jonas Stein wrote: > > Dear all > > > > the following packages are up for grabs after dropping > > desktop-misc: > > > > x11-misc/xbindkeys > > https://packages.gentoo.org/packages/x11-misc/xbindkeys > > > > There is an open PR > > https://github.com/gentoo/gentoo/pull/14706 > > > > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] [PATCH] eclass/db.eclass: use get_libname for Darwin support
Signed-off-by: Fabian Groffen
---
eclass/db.eclass | 24
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/eclass/db.eclass b/eclass/db.eclass
index 9a246d18979..52afe0b765f 100644
--- a/eclass/db.eclass
+++ b/eclass/db.eclass
@@ -23,13 +23,13 @@ db_fix_so() {
cd "${LIB}" || die
# first clean up old symlinks
- find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*so' -delete || die
- find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*so.[23]' -delete
|| die
+ find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*'"$(get_libname)"
-delete || die
+ find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*'"$(get_libname
"[23]")" -delete || die
find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*a' -delete || die
# now rebuild all the correct ones
local ext
- for ext in so a; do
+ for ext in so dylib a; do
for name in libdb libdb_{cxx,tcl,java,sql,stl}; do
target="$(find . -maxdepth 1 -type f -name
"${name}-*.${ext}" |sort -V |tail -n 1)"
[[ -n "${target}" ]] && ln -sf ${target//.\//}
${name}.${ext}
@@ -37,17 +37,17 @@ db_fix_so() {
done;
# db[23] gets some extra-special stuff
- if [[ -f libdb1.so.2 ]]; then
- ln -sf libdb1.so.2 libdb.so.2
- ln -sf libdb1.so.2 libdb1.so
- ln -sf libdb1.so.2 libdb-1.so
+ if [[ -f libdb1$(get_libname 2) ]]; then
+ ln -sf libdb1$(get_libname 2) libdb$(get_libname 2)
+ ln -sf libdb1$(get_libname 2) libdb1$(get_libname)
+ ln -sf libdb1$(get_libname 2) libdb-1$(get_libname)
fi
# what do we do if we ever get 3.3 ?
local i
for i in libdb libdb_{cxx,tcl,java,sql,stl}; do
- if [[ -f ${i}-3.2.so ]]; then
- ln -sf ${i}-3.2.so ${i}-3.so
- ln -sf ${i}-3.2.so ${i}.so.3
+ if [[ -f $i-3.2$(get_libname) ]]; then
+ ln -sf $i-3.2$(get_libname) $i-3$(get_libname)
+ ln -sf $i-3.2$(get_libname) $i$(get_libname 3)
fi
done
@@ -143,8 +143,8 @@ db_src_install_usrlibcleanup() {
mv "${LIB}/libdb_cxx.a" "${LIB}/libdb_cxx-${SLOT}.a" || die
fi
- find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*so' -delete || die
- find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*so.[23]' -delete
|| die
+ find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*'"$(get_libname)"
-delete || die
+ find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*'"$(get_libname
"[23]")" -delete || die
einfo "removing unversioned static archives"
find "${LIB}" -maxdepth 1 -type l -name 'libdb[1._-]*a' -delete || die
--
2.26.2
Re: [gentoo-dev] [RFC] Dealing with global /usr/bin/libtool use vs CC/CXX etc.
On 10-01-2021 14:34:13 +0100, Michał Górny wrote: > The vast majority of libtool-based programs use configure script to > generate libtool. However, a few non-autoconf packages also use libtool > by calling system-installed /usr/bin/libtool. The problem is that this > libtool hardcodes the values of CC/CXX at its' build time, so unless it > is rebuilt frequently, packages end up using the stale values. > The problem is known since 2005 [1] and hasn't been resolved yet. > > I can think of two ways of solving it: > > 1. We could patch system-installed libtool to respect environment > variables such as CC, CXX, etc. This will probably require carrying > a (possibly non-trivial) patch forever. On the bright side, libtool is > not exactly a package seeing frequent releases. I mean, the current > version is from 2015. > > 2. We could regenerate libtool and force local instance of libtool > in the packages needing it. The main advantage of this is that it's > a no-brainer. I could make a quick eclass that does configure a local > instance and prepends it into PATH. > > WDYT? I would prefer option 2, also because on some systems usr/bin/libtool is some entirely different tool than GNU libtool. I remember this being much more of a problem ~15 years ago, so I wonder do we have an easy way of crafting a list of affected packages, such that we can see how big the problem actually is nowadays? I'm thinking perhaps tinderbox logs or something can reveal /usr/bin/libtool usage somehow. Thanks, Fabian > [1] https://bugs.gentoo.org/88596 -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH] acct-user.eclass: don't modify existing user by default
On 04-01-2021 17:14:47 +0100, Michał Górny wrote: > Yes, I don't mind an option, as long as it spews a big fat ewarn that > the user loses the right to support. However, that's still not > the right solution to the immediate problem, and I'm currently working > on a better patch, so I'd prefer if you waited with that to avoid merge > conflicts. Could you please share your intended approach? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] _get_lock_fn: support multiprocessing spawn start method (bug 758230)
Thanks Zac!
On 04-12-2020 14:58:22 -0800, Zac Medico wrote:
> Ensure that _get_lock_fn arguments to multiprocessing.Process will
> successfully pickle, as required by the spawn start method, which
> is the default for macOS since Python 3.8.
>
> Since file descriptors are not inherited unless the fork start
> method is used, the subprocess should only try to close an
> inherited file descriptor for the fork start method.
>
> Bug: https://bugs.gentoo.org/758230
> Signed-off-by: Zac Medico
> ---
> lib/portage/locks.py | 36 +++-
> 1 file changed, 23 insertions(+), 13 deletions(-)
>
> diff --git a/lib/portage/locks.py b/lib/portage/locks.py
> index 1073343be..193045c03 100644
> --- a/lib/portage/locks.py
> +++ b/lib/portage/locks.py
> @@ -44,18 +44,7 @@ def _get_lock_fn():
> if _lock_fn is not None:
> return _lock_fn
>
> - def _test_lock(fd, lock_path):
> - os.close(fd)
> - try:
> - with open(lock_path, 'a') as f:
> - fcntl.lockf(f.fileno(),
> fcntl.LOCK_EX|fcntl.LOCK_NB)
> - except EnvironmentError as e:
> - if e.errno == errno.EAGAIN:
> - # Parent process holds lock, as expected.
> - sys.exit(0)
>
> - # Something went wrong.
> - sys.exit(1)
>
> fd, lock_path = tempfile.mkstemp()
> try:
> @@ -64,8 +53,16 @@ def _get_lock_fn():
> except EnvironmentError:
> pass
> else:
> - proc = multiprocessing.Process(target=_test_lock,
> - args=(fd, lock_path))
> + proc = multiprocessing.Process(
> + target=_subprocess_test_lock,
> + args=(
> + # Since file descriptors are not
> inherited unless the fork start
> + # method is used, the subprocess should
> only try to close an
> + # inherited file descriptor for the
> fork start method.
> + fd if
> multiprocessing.get_start_method() == "fork" else None,
> + lock_path,
> + ),
> + )
> proc.start()
> proc.join()
> if proc.exitcode == os.EX_OK:
> @@ -80,6 +77,19 @@ def _get_lock_fn():
> _lock_fn = fcntl.flock
> return _lock_fn
>
> +def _subprocess_test_lock(fd, lock_path):
> + if fd is not None:
> + os.close(fd)
> + try:
> + with open(lock_path, 'a') as f:
> + fcntl.lockf(f.fileno(), fcntl.LOCK_EX|fcntl.LOCK_NB)
> + except EnvironmentError as e:
> + if e.errno == errno.EAGAIN:
> + # Parent process holds lock, as expected.
> + sys.exit(0)
> +
> + # Something went wrong.
> + sys.exit(1)
>
> _open_fds = {}
> _open_inodes = {}
> --
> 2.26.2
>
>
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
[gentoo-dev] Last-rites: sys-kernel/xnu-headers, sys-libs/darwin-libc-headers, dev-libs/libmissing
# Fabian Groffen (2020-11-23) # No longer used, not really functional either, noone should be using # this, removal in 30 days. sys-kernel/xnu-headers sys-libs/darwin-libc-headers dev-libs/libmissing -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?
On 10-11-2020 09:21:53 +, Alexey Sokolov wrote: > > What instructed you to perform the migration? Was it the news-item? I > > don't think it should apply for Prefix profiles, and perhaps we should > > be happy the tool won't work. > > It was the big scary warning about the deprecation whenever I run > emerge. It contains list of steps. Ok. I don't know if we can do anything about that. > > Did it do anything to your system like creating a lib64 directory? Does > > anything work (because I have doubts on whether your system can still > > find the libs in there now). > > Yes. Attaching logs. The logs seem to indicate that it thinks all libs on your systems do not belong to any package. This suggests the tool cannot locate the VDB or something, as most of the things in the list are obviously owned by packages. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?
On 10-11-2020 09:49:27 +0100, Michał Górny wrote: > > > So what you're saying is that you've had the wrong value of SYMLINK_LIB > > > for ages, and now you've created a meaningless 17.1 profile that chnages > > > it but isn't actually supposed to change anything, correct? > > > > I guess, because the amd64 17.0 profile is deprecated with force, and I > > had to do something ... > > Now that's a lie. Only the regular amd64 profiles are deprecated. > There are no deprecation notices e.g. in the x32 profile or prefix > profiles. Our profiles either directly depend on the amd64/17.0 profile, or we use a sub-profile from amd64/17.0 profile, so if it's going to get removed, we are having a problem, don't we? -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?
On 10-11-2020 09:34:52 +0100, Michał Górny wrote: > On Tue, 2020-11-10 at 08:55 +0100, Fabian Groffen wrote: > > On 09-11-2020 19:38:28 +, Alexey Sokolov wrote: > > > Hi Fabian > > > I tried to migrate my prefix to 17.1, and there are issues. > > > > > > 1) unsymlink-lib requires "--root ~/gentoo" and otherwise produces an > > > error "/usr/lib is a real directory! was the migration done already?" > > > > I think unsymlink-lib doesn't have Prefix support, but in addition, > > what unsymlink-lib is trying to achieve, is not a thing perhaps on > > Prefix. > > > > A prefix system (at least all of mine) doesn't have libXX or lib/XX > > (a.k.a. multilib) directories. The /usr-split was long ago removed, > > and thus what we have is: > > > > lib -> usr/lib > > > > Now, SYMLINK_LIB=no seems to split into lib and lib64, but lib64 does > > not exist on Prefix systems. > > > > So what you're saying is that you've had the wrong value of SYMLINK_LIB > for ages, and now you've created a meaningless 17.1 profile that chnages > it but isn't actually supposed to change anything, correct? I guess, because the amd64 17.0 profile is deprecated with force, and I had to do something ... -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?
On 09-11-2020 19:38:28 +, Alexey Sokolov wrote: > Hi Fabian > I tried to migrate my prefix to 17.1, and there are issues. > > 1) unsymlink-lib requires "--root ~/gentoo" and otherwise produces an > error "/usr/lib is a real directory! was the migration done already?" I think unsymlink-lib doesn't have Prefix support, but in addition, what unsymlink-lib is trying to achieve, is not a thing perhaps on Prefix. A prefix system (at least all of mine) doesn't have libXX or lib/XX (a.k.a. multilib) directories. The /usr-split was long ago removed, and thus what we have is: lib -> usr/lib Now, SYMLINK_LIB=no seems to split into lib and lib64, but lib64 does not exist on Prefix systems. Since Prefix is non-multilib by design*, I wonder if unsymlink-lib is necessary in the best case, but going to break the Prefix system in the worst case. What instructed you to perform the migration? Was it the news-item? I don't think it should apply for Prefix profiles, and perhaps we should be happy the tool won't work. * non-multilib is a decision dating back a decade or so, which means effectively any Prefix install you encounter should be non-multilib > 2) $ unsymlink-lib --root ~/gentoo --migrate --pretend > usage: unsymlink-lib [-h] [-p] [--root ROOT] [--analyze] [--migrate] > [--rollback] [--finish] [--force-rollback] > [--resume-finish] [-P PREFIX] [--hardlink] > unsymlink-lib: error: Requested action requires root privileges > > Well, I worked it around by adding "is_root = True" to unsymlink-lib Did it do anything to your system like creating a lib64 directory? Does anything work (because I have doubts on whether your system can still find the libs in there now). > > 3) Step 9 (Rebuild gcc) fails: > configure:4372: checking whether the C compiler works > > > > configure:4394: x86_64-pc-linux-gnu-gccconftest.c >&5 > > > > /home/user/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/as: > error while loading shared libraries: > libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so: cannot open shared Something like this I was suspecting. Can you still rollback? If you can, I'd try that and hope it restores your system in working order. For any Prefix user, DO NOT run unsymlink-lib, I believe you should NOT NEED IT! Thanks, Fabian > object file: No such file or directory > configure:4398: $? = 1 > > > > configure:4436: result: no > > The file exists: > $ find ~ -name libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so > /home/user/gentoo/usr/lib/binutils/x86_64-pc-linux-gnu/2.34/libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so > > -- > Best regards, > Alexey "DarthGandalf" Sokolov > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?
On 07-11-2020 11:42:44 +, Alexey Sokolov wrote: > 22.10.2020 20:16, Andreas K. Hüttel пишет: > > Am Donnerstag, 22. Oktober 2020, 18:44:48 EEST schrieb Michał Górny: > >> On Thu, 2020-10-22 at 11:17 -0400, Brian Evans wrote: > >>> Users frequently are choosing the wrong profile versions in new installs > >>> and accidentally downgrading to 17.0 with some saying they see it first. > >>> > >>> A simple reordering could help new installs. > > > > Independent of this useful change, it's probably time to deprecate the > > amd64 > > 17.0 profiles! > > > > Prefix bootstrap script still makes new installations to use it This should be solved with b0445c0a8dd6d2f792c5bb088b154aca53868353 a9c478dc881ee18fefc7342da994b00e60eaad8e on gentoo.git and 0d7f6b6eb00d0f51f35019846b8f79048b30be93 on prefix.git. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] net-misc/asterisk fails to compile: clang/LLVM: bug 731280
On 28-08-2020 08:52:09 +0100, Sergei Trofimovich wrote: > On Fri, 28 Aug 2020 07:37:54 +0100 > Sergei Trofimovich wrote: > > > On Fri, 28 Aug 2020 08:15:47 +0200 > > Jaco Kroon wrote: > > > > > Hi All, > > > > > > https://bugs.gentoo.org/731280 > > > > > > Summary: > > > > > > This machine uses a clang/LLVM toolchain. > > > Asterisk fails to compile, ./configure fails with: > > > > > > checking for RAII support... checking for clang -fblocks... > > > configure: error: BlocksRuntime is required for clang, please install > > > libblocksruntime > > > > > > I suspect this explains the issue: > > > > > > https://github.com/mackyle/blocksruntime > > > > > > I have no idea how to actually solve this. > > > > honggfuzz also needs it as it happens to use -fblocks on clang: > > https://bugs.gentoo.org/729256 > > > > We need to package the library. I can give it a try. > > Packaged library as sys-libs/blocksruntime: > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ae376c73ef197d6c7aa619e821c436ccab0cd77e > > Usage example for app-forensics/honggfuzz: > > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fd841336dfdefbc14907e2d9b1eb1a1a3f5f8b8e This is cool, but shouldn't it be something like openmp? (e.g. blocks) There is no reason blocks have to be used if not on macOS (where system headers use blocks features). Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] euses(1) Reimplementation
706#c4 > [4] https://dl.acm.org/doi/abs/10.1145/116825.116845 > [5] > https://sourceware.org/git/?p=glibc.git;a=blob;f=string/strcasestr.c;h=d2964c5548b9ea7a68fc5b18b25ddfe7ddd6835c;hb=HEAD#l45 > [6] http://git.suugaku.co.uk/ash-euses/tree/ > [7] http://git.suugaku.co.uk/ash-euses/snapshot/ash-euses-0.3.tar.gz > > -- > > Ashley Dixon > suugaku.co.uk > > 2A9A 4117 > DA96 D18A > 8A7B B0D2 > A30E BF25 > F290 A8AA > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Speeding up Tree Verification
On 30-06-2020 13:13:29 -0500, Sid Spry wrote: > On Tue, Jun 30, 2020, at 1:20 AM, Fabian Groffen wrote: > > Hi, > > > > On 29-06-2020 21:13:43 -0500, Sid Spry wrote: > > > Hello, > > > > > > I have some runnable pseudocode outlining a faster tree verification > > > algorithm. > > > Before I create patches I'd like to see if there is any guidance on > > > making the > > > changes as unobtrusive as possible. If the radical change in algorithm is > > > acceptable I can work on adding the changes. > > > > > > Instead of composing any kind of structured data out of the portage tree > > > my > > > algorithm just lists all files and then optionally batches them out to > > > threads. > > > There is a noticeable speedup by eliding the tree traversal operations > > > which > > > can be seen when running the algorithm with a single thread and comparing > > > it to > > > the current algorithm in gemato (which should still be discussed here?). > > > > I remember something that gemato used to use multiple threads, but > > because it totally saturated disk-IO, it was brought back to a single > > thread. People were complaining about unusable systems. > > > > I think this is an argument for cgroups limits support on the portage process > or > account as opposed to an argument against picking a better algorithm. That is > something I have been working towards, but I am only one man. But this requires a) cgroups support, and b) the privileges to use it. Shouldn't be a problem in the normal case, but just saying. > > In any case, can you share your performance results? What speedup did > > you see, on warm and hot FS caches? Which type of disk do you use? > > > > I ran all tests multiple times to make them warm off of a Samsung SSD, but > nothing very precise yet. > > % gemato verify --openpgp-key signkey.asc /var/db/repos/gentoo > [...] > INFO:root:Verifying /var/db/repos/gentoo... > INFO:root:/var/db/repos/gentoo verified in 16.45 seconds > > sometimes going higher, closer to 18s, vs. > > % ./veriftree.py > 4.763171965983929 > > So roughly an order of magnitude speedup without batching to threads. That is kind of a change. Makes one wonder if you really did the same work. > > You could compare against qmanifest, which uses OpenMP-based > > paralllelism while verifying the tree. On SSDs this does help. > > > > I lost my notes -- how do I specify to either gemato or qmanifest the GnuPG > directory? My code is partially structured as it is because I had problems > doing > this. I rediscovered -K/--openpgp-key in gemato but am unsure for qmanifest. qmanifest doesn't do much magic out of the standard gnupg practices. (It is using gpgme.) If you want it to use a different gnupg dir, you may change HOME, or GNUPGHOME. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Speeding up Tree Verification
: str) -> bool:
> response = dns.resolver.query(domain, dns.rdatatype.NS)
> nsname = response.rrset[0]
> response = dns.resolver.query(str(nsname), dns.rdatatype.A)
> nsaddr = response.rrset[0].to_text()
>
> # DNSKEY
> request = dns.message.make_query(domain,
> dns.rdatatype.DNSKEY, want_dnssec=True)
> response = dns.query.udp(request, nsaddr)
> if response.rcode() != 0:
> raise Exception('Unable to request dnskey.')
>
> answer = response.answer
> if len(answer) != 2:
> raise Exception('Malformed answer to dnskey query.')
>
> name = dns.name.from_text(domain)
> try:
> dns.dnssec.validate(answer[0], answer[1], {name: answer[0]})
> except dns.dnssec.ValidationFailure as exc:
> # Validation failed. The raise causes python to abort with status 1.
> #raise exc
> return False
> except AttributeError as exc:
> # Validation may have failed; DNSKEY missing signer attribute. dig
> may report
> # domain as valid.
> #
> # TODO: Additional state where subdomain of valid domain may fail
> with 3 nested
> # KeyErrors. Avoid temptation to wildcard catch. Safer to put in
> process?
> #raise exc
> return False
> else:
> return True
>
> def hash_localpart(incoming: bytes) -> str:
> '''Z-base32 the localpart of an e-mail address
>
>
> https://tools.ietf.org/html/draft-koch-openpgp-webkey-service-08#section-3.1
> describes why this is needed.
>
> See https://tools.ietf.org/html/rfc6189#section-5.1.6 for a
> description of the z-base32 scheme.
> '''
> zb32 = "ybndrfg8ejkmcpqxot1uwisza345h769"
>
> b = hashlib.sha1(incoming).digest()
> ret = ""
> assert(len(b) * 8 == 160)
> for i in range(0, 160, 5):
> byte = i // 8
> offset = i - byte * 8
> # offset | bits remaining in k+1 | right-shift k+1
> # 3 | 0 | x
> # 4 | 1 | 7
> # 5 | 2 | 6
> # 6 | 3 | 5
> # 7 | 4 | 4
> if offset < 4:
> n = (b[byte] >> (3 - offset))
> else:
> n = (b[byte] << (offset - 3)) + (b[byte + 1] >> (11 - offset))
>
> ret += zb32[n & 0b1]
> return ret
>
> def build_web_key_uri(address: str) -> str:
> local, remote = address.split('@')
> local = hash_localpart(local.encode('utf-8'))
> return 'https://' + remote + '/.well-known/openpgpkey/hu/' + \
> local
>
> def stream_to_file(uri: str, fname: str) -> None:
> with requests.get(uri, verify=True, stream=True) as r:
> from pprint import pprint
> pprint(r.headers)
> with open(fname, 'wb') as f:
> shutil.copyfileobj(r.raw, f)
> ```
>
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] Add caching to a few commonly used functions
Hi Chun-Yu, On 26-06-2020 23:34:12 -0700, Chun-Yu Shei wrote: > Hi, > > I was recently interested in whether portage could be speed up, since > dependency resolution can sometimes take a while on slower machines. > After generating some flame graphs with cProfile and vmprof, I found 3 > functions which seem to be called extremely frequently with the same > arguments: catpkgsplit, use_reduce, and match_from_list. In the first > two cases, it was simple to cache the results in dicts, while > match_from_list was a bit trickier, since it seems to be a requirement > that it return actual entries from the input "candidate_list". I also > ran into some test failures if I did the caching after the > mydep.unevaluated_atom.use and mydep.repo checks towards the end of the > function, so the caching is only done up to just before that point. > > The catpkgsplit change seems to definitely be safe, and I'm pretty sure > the use_reduce one is too, since anything that could possibly change the > result is hashed. I'm a bit less certain about the match_from_list one, > although all tests are passing. > > With all 3 patches together, "emerge -uDvpU --with-bdeps=y @world" > speeds up from 43.53 seconds to 30.96 sec -- a 40.6% speedup. "emerge > -ep @world" is just a tiny bit faster, going from 18.69 to 18.22 sec > (2.5% improvement). Since the upgrade case is far more common, this > would really help in daily use, and it shaves about 30 seconds off > the time you have to wait to get to the [Yes/No] prompt (from ~90s to > 60s) on my old Sandy Bridge laptop when performing normal upgrades. > > Hopefully, at least some of these patches can be incorporated, and please > let me know if any changes are necessary. This sounds like a good job to me! Do you have any idea what the added memory pressure for these changes are? Thanks, Fabian > > Thanks, > Chun-Yu > > > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] Use env to find python
On 16-06-2020 23:10:28 -0500, Sid Spry wrote: > On Tue, Jun 16, 2020, at 3:57 PM, Michał Górny wrote: > > '/usr/bin/env python' (with no extra options) is the portable shebang. > > > > I added `-S` to preserve the options passed via the shebang line. It seems > they can be left off, does anyone know otherwise? -S is not supported on some platforms. In any case, I think this patch is wrong. In the Prefix case, the shebangs are set during install, in the non-Prefix case I think there is machinery in place to replace the shebangs during install. It seems the problem is already solved, why do you need these shebangs changed? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config
On 24-05-2020 10:48:47 +0100, Sergei Trofimovich wrote:
> On Sat, 23 May 2020 22:41:02 -0400
> Mike Gilbert wrote:
> > I don't think we
> > want to give people the impression that this is a well-supported
> > configuration. I would only expect people to disable this if they want
> > to break their system intentionally.
>
> Yeah, today it's certainly a way to get your system in a miserable state.
> My hope is that USE=-native-symlinks can get you a working Gentoo in a
> near future by only fixing real package problems and limitations of their
> build systems.
Portage adds PREROOTPATH, ROOTPATH and a standard set of
/usr/{local/,}{s,}bin to PATH in _doebuild_path. That means if you add
something like /usr/bin/native-toolchain to PATH only, users will get
gcc, ld, etc., while root and Portage will lack these. One can wonder
if root should have direct access to compilers, but it's easy enough to
add the compilers to PATH if really necessary.
I guess what I'm trying to say is: you can hide effect of the setup for
users if you'd like. That is, after we had buildbots point out the bulk
of packages that are wrong of course.
Thanks,
Fabian
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-dev] [RFC] Anti-spam for goose
On 22-05-2020 21:58:54 +0200, Michał Górny wrote: > Let's put it like this. This thing starts working. Package X is > broken, and we see that almost nobody is using it. We remove that > package. Now somebody is angry. He submits a lot of fake data to > render the service useless so that we don't make any future decisions > based on it. I'm affraid that has a heroic flair to me. The service should never be used for decisions like that, because it's a biased sample at most. Doing stuff like this simply destroys the soul of the distribution. I hope this isn't one of your genuine objectives with the service. If it is, I can see why you fear spam so much. Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Anti-spam for goose
alternative of using a proof-of-work algorithm was suggested to me > yesterday. The idea is that every submission has to be accompanied with > the result of some cumbersome calculation that can't be trivially run > in parallel or optimized out to dedicated hardware. > > On the plus side, it would rely more on actual physical hardware than IP > addresses provided by ISPs. While it would be a waste of CPU time > and memory, doing it just once a week wouldn't be that much harm. > > On the minus side, it would penalize people with weak hardware. > > For example, 'time hashcash -m -b 28 -r test' gives: > > - 34 s (-s estimated 38 s) on Ryzen 5 3600 > > - 3 minutes (estimated 92 s) on some old 32-bit Celeron M > > At the same time, it would still permit a lot of fake submissions. For > example, randomx [1] claims to require 2G of memory in fast mode. This > would still allow me to use 7 threads. If we adjusted the algorithm to > take ~30 seconds, that means 7 submissions every 30 s, i.e. 20k > submissions a day. > > So in the end, while this is interesting, it doesn't seem like > a workable anti-spam measure. > > > Option 3: explicit CAPTCHA > == > A traditional way of dealing with spam -- require every new system > identifier to be confirmed by solving a CAPTCHA (or a few identifiers > for one CAPTCHA). > > The advantage of this method is that it requires a real human work to be > performed, effectively limiting the ability to submit spam. > The disadvantage is that it is cumbersome to users, so many of them will > just resign from participating. > > > Other ideas > === > Do you have any other ideas on how we could resolve this? > > > [1] https://github.com/tevador/RandomX > > > -- > Best regards, > Michał Górny > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: Gentoo Identity Provider
On 18-05-2020 18:42:24 -0700, Alec Warner wrote: > TL;DR: What if we launched id.gentoo.org[1], an identity provider that > provides > authentication for Gentoo properties? Basically, 1 username / password for > wiki, > bugs, email, forums, and any other http service[0][1]. I'd be in favour of SSO for all http-, imap- and smtp-based Gentoo services. Thanks, Fabian > > Today Gentoo has numerous systems that mostly work in a segmented way. > > - To connect to hosts, we use ssh keys. > - Git is authenticated via ssh keys. > - Email uses LDAP passwords. > - Bugzilla has its own identities, with their own passwords. > - Wiki is separate, with its own passwords. > - Forums are separate. > - Infra has an additional 4 systems that use separate credentials. > > Some applications support 2FA (such as wiki.) > Some applications do not support 2FA. > Applications that require 2FA have a configuration for each app, so you have N > configurations. > > If we configured id.gentoo.org[2] you would have 1 identity across all gentoo > properties. > > Is this a thing people are interested in? > > [0] It's unlikely operations for git via ssh would change in this rollout. > [1] Its unclear if the scope is "gentoo developers" or "any community member." > The former have LDAP accounts and @gentoo.org[3] email addresses and so we can > manage them easily; managing 1000s of other accounts in the IDP remains to be > seem. > > > References >1. http://id.gentoo.org >2. http://id.gentoo.org >3. http://gentoo.org -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Last standing Python 2.7 dependency
On 02-05-2020 23:24:42 -0700, Brian Dolbec wrote: > On Sun, 3 May 2020 07:28:50 +0200 > Viktar Patotski wrote: > > > Hi all, > > > > I'd also like to clean my system and have it Python 2.7 free. Are > > there any guidelines to check which packages are still using pyton2_7 > > in my system? > > > > Thanks, > > Viktar > > > > There are both equery and enalyze commands in gentoolkit that can give > you reports about what pkgs are installed. > > equery hasuse > enalyze analyze [use|pkguse] > > for help on them: > equery -h > equery hasuse -h > enalyze -h > enalyze a -h In addition to these great tools, portage-utils' quse might also be useful: % quse python2_7 ... Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz
On 26-04-2020 21:29:42 +0200, Michał Górny wrote: > On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote: > > Bug: https://bugs.gentoo.org/715108 > > Signed-off-by: Matt Turner > > --- > > Strawman patch. Bikeshed away. > > > > xz is generally slow and doesn't do parallel good. If we want to change > this, we should go for something cool like zstd that scales better. I'd go for zstd too. It seems to be the best of both worlds, good compression at a good speed. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH] fcaps.eclass: disable fcaps() on Prefix.
On 08-03-2020 15:26:50 +0800, hero...@gentoo.org wrote:
> From: Benda Xu
>
> Gentoo Prefix runs with a normal user and cannot grant extra
> capabilities. Exit gracefully with a message.
>
> Signed-off-by: Benda Xu
> ---
> eclass/fcaps.eclass | 5 +
> 1 file changed, 5 insertions(+)
>
> diff --git a/eclass/fcaps.eclass b/eclass/fcaps.eclass
> index 467f955f5e9a..563d177c92d5 100644
> --- a/eclass/fcaps.eclass
> +++ b/eclass/fcaps.eclass
> @@ -78,6 +78,11 @@ DEPEND="filecaps? ( sys-libs/libcap )"
> fcaps() {
> debug-print-function ${FUNCNAME} "$@"
>
> + if [[ ${EUID} != 0 ]] ; then
> + einfo "Insufficient privileges to execute ${FUNCNAME[0]}"
> + return 0
Perhaps add "ignoring" or something in the message here to make it clear
this isn't treated as an error?
Thanks,
Fabian
> + fi
> +
> # Process the user options first.
> local owner='root'
> local group='0'
> --
> 2.25.0
>
>
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: PGP signature
Re: [gentoo-dev] Keywordreqs and slacking arch teams
On 28-12-2019 22:27:02 +1300, Kent Fredric wrote: > On Sat, 28 Dec 2019 08:09:33 +0100 > Michał Górny wrote: > > > How can we improve this? > > Every time this kind of issue comes up, I can't help feeling we need > some sort of tool more advanced than what we currently have. > > Something that maintains persistence of keyword demands similar to how > the current bot does, but more ... > > Its the sort of thing I get tempted to implement myself, but get too > horrified about the prospect of working with portage internals to do it. Hmmm, interested to hear what kind of things you're thinking about here. I feel it would help if we would have the ability to at least compile/test ebuilds automatically. Not sure how helpful qemu could be there, but I know things like compiling for things like arm (RPi) works reasonably well. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.
On 13-12-2019 14:24:33 -0500, Michael Orlitzky wrote: > On 12/13/19 9:28 AM, Fabian Groffen wrote: > > > > We are providing those patches, maybe. In reality very often the > > patches originate from somewhere else though. And you don't want to > > have to respin all of those just because. At least that's what I feel. > > > > Just because... the context changed? A new "!" in a line of context can > be the difference between letting someone log in with the right password > and letting them log in with the wrong password. You should at least > have to stop and verify that the patch does what you think it does when > it "gains" fuzz. And at that point, git diff will give you a clean > version of it. Counter argument is that we've been doing this for decades, and always relied on maintainers to check the contents of their patches, without problems. We didn't introduce a predictable random number generator or something. Your very specific example just illustrates the niche this proposal is targetting. As with many of the proposals lately, they just seem to aim at more work for individual maintainers, with a very low gain ratio. Better, allow this to be a FEATURE, or whatever, that devs should enable, but don't spit this in the user's face by default. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.
On 13-12-2019 15:24:40 +0100, Michał Górny wrote: > > > ...and why do we consider it correct to apply patches when the context > > > doesn't match? If our only goal is to make things 'easier' for > > > 'everyone', then we could just pass -F and ignore all the context. > > > > > > Though I don't understand why include any context in the first place if > > > you don't care about it matching. Sounds like a waste of space to me! > > > > The patch command defaults to -F2. If that makes no sense, why is it > > the upstream default? > > > > You should ask upstream, not me. But if I were to guess, the answer > would be because patch(1) is used by random people trying to apply > random patches they've found somewhere. We on the other hand are > applying patches that *we* are supposed to provide. We are providing those patches, maybe. In reality very often the patches originate from somewhere else though. And you don't want to have to respin all of those just because. At least that's what I feel. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New distfile mirror layout
On 29-10-2019 15:45:34 +0100, Michał Górny wrote: > On Tue, 2019-10-29 at 15:33 +0100, Fabian Groffen wrote: > > In addition, there are currently files there that aren't referenced from > > ebuilds. Prefix uses these files during bootstrap, local mirrors are > > often much faster than dev.g.o. > > > > If the files don't get mirrored anymore, I guess I can create a dummy > > ebuild that has the files in SRC_URI. > > Ok, this is something I wasn't aware of. I agree that dummy ebuild > should not be necessary here. However, I'm also not sure if distfiles- > local is really the proper way either, especially that I don't see such > files on woodpecker right now. There should be /space/distfiles-local and /space/distfiles-whitelist/prefix with a list of files to retain on the mirror. Thanks, Fabian > I don't think the matter is urgent right now, so let's ponder on it > a bit. In particular, I think we should have a clear indication of who > added which files, when, what for and where they came from. Those are > precisely the things that the current distfiles-local approach misses. > > > If the files get mirrored, but put in a subdir based on the filename > > hash, the original query endpoint on distfiles.g.o changes, much like > > the SRC_URI approach. > > > > Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o > > URL with subdir for most part I think, but it's sub-optimal from my > > point of view. Calculating the hash is not always feasible due to the > > lack of b2sum or other means. Hence my earlier request to have such > > official translation service on Gentoo hardware. > > > > (I just wrote a small wsgi script that calculates the hash and generates > > the redirect from Python, served via uwsgi/nginx, but there should be > > many ways to achieve the same goals, if and only if a blake2b > > implementation were available for it.) > > This is also something that needs thinking. I personally don't mind > having one but it would be nice if it was able to account for geodns > and such. > > -- > Best regards, > Michał Górny > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New distfile mirror layout
On 29-10-2019 15:17:38 +0100, Ulrich Mueller wrote: > >>>>> On Tue, 29 Oct 2019, Michał Górny wrote: > > > On Tue, 2019-10-29 at 14:09 +0100, Ulrich Mueller wrote: > >> > What if the file is hosted at a non-standard tcp port upstream > >> > (like http://example.org:8080/)? The devmanual says that it _must_ > >> > be manually uploaded to /space/distfiles-local/ in such cases. > > >> Or another example, app-emacs/vhdl-mode-3.38.1, where (incompetent, > >> or nasty?) upstream blocks wget for some reason, but other methods > >> (e.g., curl, firefox) work? How would I get the file onto the mirrors > >> there? > > > If I were you, I would've explicitly mirrored the file anyway. > > If upstream blocks wget, then users who do not use GENTOO_MIRRORS will > > also suffer due to it. > > All what I'm saying is that there can be unusual circumstances where > manual uploading of a file is useful. So please don't take that > possibility away. In addition, there are currently files there that aren't referenced from ebuilds. Prefix uses these files during bootstrap, local mirrors are often much faster than dev.g.o. If the files don't get mirrored anymore, I guess I can create a dummy ebuild that has the files in SRC_URI. If the files get mirrored, but put in a subdir based on the filename hash, the original query endpoint on distfiles.g.o changes, much like the SRC_URI approach. Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o URL with subdir for most part I think, but it's sub-optimal from my point of view. Calculating the hash is not always feasible due to the lack of b2sum or other means. Hence my earlier request to have such official translation service on Gentoo hardware. (I just wrote a small wsgi script that calculates the hash and generates the redirect from Python, served via uwsgi/nginx, but there should be many ways to achieve the same goals, if and only if a blake2b implementation were available for it.) Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New distfile mirror layout
On 29-10-2019 05:27:37 +0100, Michał Górny wrote: > On Tue, 2019-10-29 at 00:24 +0100, Chí-Thanh Christopher Nguyễn wrote: > > Hi! > > > > > Today you get chastised for using /space/distfiles-local and not > > > following policy changes. The devmanual states that it's deprecated > > > since at least 2011, and talks of using d.g.o [1]. > > > [1] > > https://devmanual.gentoo.org/general-concepts/mirrors/index.html#suitable-download-hosts > > > > Sorry I'm late to the party, but I would like to enquire about what happens > > if a file with existing filename but different b2sum gets uploaded to > > /space/distfiles-local now? > > The same as before. It gets put in top-level disfiles directory. > Hashes are calculated from filenames, so this wouldn't affect it. That > is, if it put those files in subdirectories in the first place because > it doesn't. /space/distfiles-local is no longer copied to the mirrors? or just not copied in the subdir-hierarchy? > > Doing so and updating the Manifest used to be another (not necessarily > > preferred) method to address upstream remaking release packages. > > > > It's no longer valid. Just wondering. Do you mean it isn't valid that some upstreams do this (yes horror)? We surely need a way to work around that ... Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] New distfile mirror layout
Hi, On 18-10-2019 15:41:32 +0200, Michał Górny wrote: > 3. Directly fetching files from distfiles.gentoo.org will become > a little harder. To fetch a distfile named 'foo-1.tar.gz', you'd have > to use something like: > > $ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2 > 1b > $ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz > ... > > > Alternatively, you can: > > $ wget http://distfiles.gentoo.org/distfiles/INDEX > > and grep for the right path there. This INDEX is also a more > lightweight alternative to HTML indexes generated by the servers. Would it be possible to run a service that sends a 302 for the distfiles/foo-1.tar.gz to the appropriate bucket such that manual fetching doesn't require to calculate the hash? I prototyped this myself for distfiles.prefix, and seems like a nice guesture for at least the transition period? Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Underscores in USE flags
On 21-09-2019 09:06:01 +0200, Michał Górny wrote: > On Sat, 2019-09-21 at 08:43 +0200, Fabian Groffen wrote: > > Why not teach our tools (equery, quse, etc.) to print these USE-flags > > like Portage does? (looking them up to be valid expands) > > Then users have nothing to be confused about (no distinction between > > foo_bar and FOO="bar"), and new USE_EXPANDS cannot be > > silently/accidentially introduced. > > I don't see how that solves the problem. More tools having distinct > output don't change the fact that anyone with a bit of ebuild knowledge > will say 'this looks like USE_EXPAND' while looking at it, independently > of what some tools would say. Well... someone with a bit of ebuild knowledge would see odd USE-flags. USE_EXPAND is a (bad) hack, of having some USE-flags mean something different, or resolve through something different, while in reality they really don't do anything odd. In fact, sometimes users have to use FOO="bar" (make.conf), while other times foo_bar needs to be used (e.g. use.mask, or IUSE=). Consistency would be nice, the real question is, what does USE_EXPAND actually try to achieve, and can we fix it properly in the next EAPI, such that repoman can also do the proper complaints about USE-flag (and USE_EXPAND-flag) naming by then. Back to the thread, the point is, these flags exist today, and renaming flags is not something to be considered harmless. As much as the recent renaming of lm_sensors to lm-sensors caused breakage (and still does, apparently some tools keep caches, etc.) also renaming USE-flags goes by problems, in particular for managed systems (Chef, Puppet). It's not a matter of just fixing the name for a USE-flag. This is saying nothing about whether or not we'd want to change the flag. It's about the impact of the change, and whether that is worth it for the noble aim of consistency or correctness. I believe this was the OPs point in this thread. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] Underscores in USE flags
On 20-09-2019 22:53:53 +0200, Michał Górny wrote: > On Fri, 2019-09-20 at 13:46 -0700, Zac Medico wrote: > > > > If we take this underscore rule to its logical extreme, then we should > > rename python_targets_python3_7 to python_targets_python3-7, yes? > > Believe me, I would have done that already if not the fact that with all > the dependency logic around here it would be totally destructive to all > Gentoo systems. Honestly, with this reasoning, why force other packages to go through USE-flag renaming in that case? A major consumer of USE_EXPAND isn't sticking to the rule, which makes any benefit of it moot. Tools cannot assume the last underscore separates the USE_EXPAND var from its value, users cannot see what is the value either, without knowledge. Why not teach our tools (equery, quse, etc.) to print these USE-flags like Portage does? (looking them up to be valid expands) Then users have nothing to be confused about (no distinction between foo_bar and FOO="bar"), and new USE_EXPANDS cannot be silently/accidentially introduced. > But hey, expect hyphen on 3.8. I honestly feel for consistency and not confusing users, we should either do them all or stick to the current scheme. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
[gentoo-dev] Last rites: app-portage/hashgen
# Fabian Groffen (2019-09-15) # Incorporated in >=app-portage/portage-utils-0.80 as qmanifest # Removal in 30 days. Bug #694428 app-portage/hashgen -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH] use.desc: Introduce global USE=magic
Hi Michał, It looks like you missed two from your original set? app-editors/nano[magic] Add magic file support (sys-apps/file) to automatically detect appropriate syntax highlighting www-servers/pshs[magic] Enable automatic detection of Content-Type using libmagic (sys-apps/file) Thanks, Fabian On 19-08-2019 22:04:12 +0200, Michał Górny wrote: > On Sun, 2019-08-11 at 13:21 +0200, Michał Górny wrote: > > USE=magic is currently used consistently by 12 packages: > > > > Merged. > > -- > Best regards, > Michał Górny > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes
Hi, On 25-07-2019 14:20:50 +0200, Michał Górny wrote: > Hi, > > TL;DR: I'd like to make it possible for ebuilds to define additional > variables that will be stored in md5-cache. This would be useful for CI > and other tooling that right now has to parse ebuilds for other data. Only downside I can think of, is a diskspace increase for the md5-cache. Not sure if this is going to be substantial, but given things like PYTHON_COMPAT, perhaps a quick calculation of extra "cost" can be made. Should diskspace become a problem, one could consider to use a separate file/dir, that users could rsync-exclude, since Portage won't need it to operate properly. Thanks, Fabian > > > The idea is to add a new incremental ebuild/eclass variable (technical > name: QA_EXTRA_CACHE_VARS) that would define additional data to be > stored in cache. For example, python*-r1 eclasses would define > 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc. > > When regenerating cache, the PM would read this variable, and store > the values of all defined variables into md5-cache. As a result, > programs needing those variables can get them straight from cache > without having to attempt to run or parse ebuilds (which is both slow > and prone to bugs). > > This would benefit e.g. gpyutils that right now need to attempt to parse > PYTHON_COMPAT from ebuilds. It would also benefit writing future > pkgcheck checks for user/group ID collisions. > > > Notes: > > - since md5-cache uses key-value format and allows for future > extensions, the new values can be added without breaking anything; > > - md5-cache is not specified in the PMS, and the whole thing can be > implemented without need for EAPI bump, > > - I would like to have this implemented consistently both in Portage > and pkgcore, > > - we will need to clearly define how to dump arrays. > > > What do you think? > > -- > Best regards, > Michał Górny > -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
