[gentoo-portage-dev] [PATCH] emerge: Warn on maintainer-needed pkgs
From: "Sam James (sam_c)"
Introduces new repos.conf/layout.conf variables:
* maintainer-nag (yes/no)
* maintainer-nag-url (URL to be given for volunteers to look at)
Bug: https://bugs.gentoo.org/711466
Closes: https://github.com/gentoo/portage/pull/529
Signed-off-by: Sam James (sam_c)
Signed-off-by: Zac Medico
---
lib/_emerge/depgraph.py | 48
lib/portage/repository/config.py | 26 +++--
2 files changed, 72 insertions(+), 2 deletions(-)
diff --git a/lib/_emerge/depgraph.py b/lib/_emerge/depgraph.py
index ec90e59df..216eec28e 100644
--- a/lib/_emerge/depgraph.py
+++ b/lib/_emerge/depgraph.py
@@ -53,6 +53,11 @@ from portage.util._eventloop.EventLoop import EventLoop
from portage.util._eventloop.global_event_loop import global_event_loop
from portage.versions import _pkg_str, catpkgsplit
+try:
+ from portage.xml.metadata import MetaDataXML
+except Exception:
+ MetaDataXML = None
+
from _emerge.AtomArg import AtomArg
from _emerge.Blocker import Blocker
from _emerge.BlockerCache import BlockerCache
@@ -9163,6 +9168,49 @@ class depgraph(object):
show_mask_docs()
writemsg("\n", noiselevel=-1)
+ # Build a list of problematic packages
+ maintainer_nag_packages = []
+
+ for pkg in self._dynamic_config.digraph:
+ if not isinstance(pkg, Package):
+ continue
+
+ if not pkg.type_name == 'ebuild':
+ continue
+
+ maintainer_nag =
pkg.root_config.settings.repositories[pkg.repo].maintainer_nag
+ maintainer_nag_url =
pkg.root_config.settings.repositories[pkg.repo].maintainer_nag_url
+
+ if not maintainer_nag:
+ continue
+
+ path =
os.path.dirname(pkg.root_config.trees['porttree'].dbapi.findname(pkg.cpv,
myrepo=pkg.repo))
+ path = os.path.join(path, 'metadata.xml')
+
+ if MetaDataXML is not None and os.path.isfile(path):
+ try:
+ metadata_xml = MetaDataXML(path, None)
+ maintainer =
metadata_xml.format_maintainer_string()
+ except SyntaxError:
+ continue
+
+ if not maintainer:
+ maintainer_nag_packages.append(pkg.cpv
+ "::" + pkg.repo)
+
+ if maintainer_nag_packages:
+ writemsg("\n" + colorize("WARN", "!!!") + \
+" The following packages have no maintainer.
They are at greater risk of bugs." +
+"\n!!! You can help to keep it in tree and fix
bugs (inc. possible security issues)")
+
+ if maintainer_nag_url:
+ writemsg("\n!!! Please see: " +
maintainer_nag_url)
+
+ writemsg("\n")
+ for pkg in maintainer_nag_packages:
+ writemsg("- " + pkg)
+ writemsg("\n")
+
+
for pargs, kwargs in
self._dynamic_config._unsatisfied_deps_for_display:
self._show_unsatisfied_dep(*pargs, **kwargs)
diff --git a/lib/portage/repository/config.py b/lib/portage/repository/config.py
index 50ab18026..1309cef65 100644
--- a/lib/portage/repository/config.py
+++ b/lib/portage/repository/config.py
@@ -1,4 +1,4 @@
-# Copyright 2010-2019 Gentoo Authors
+# Copyright 2010-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
from __future__ import unicode_literals
@@ -96,6 +96,8 @@ class RepoConfig(object):
'manifest_hashes',
'manifest_required_hashes',
'masters',
+ 'maintainer_nag',
+ 'maintainer_nag_url',
'missing_repo_name',
'module_specific_options',
'name',
@@ -306,6 +308,8 @@ class RepoConfig(object):
self.disable_manifest = False
self.manifest_hashes = None
self.manifest_required_hashes = None
+ self.maintainer_nag = repo_opts.get('maintainer-nag',
'no').lower() in ('true', 'yes')
+ self.maintainer_nag_url = repo_opts.get('maintainer-nag-url')
self.update_changelog = False
self.cache_formats = None
self.portage1_profiles = True
@@ -338,13 +342,23 @@ class RepoConfig(object):
self.name = layout_data['repo-name']
self.missing_repo_name = False
+ # Override some repos.conf setting with layout.conf
for value
Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz
On 4/26/20 2:12 PM, Zac Medico wrote: > On 4/26/20 1:09 PM, Matt Turner wrote: >> On Sun, Apr 26, 2020 at 12:29 PM Michał Górny wrote: >>> >>> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote: Bug: https://bugs.gentoo.org/715108 Signed-off-by: Matt Turner --- Strawman patch. Bikeshed away. >>> >>> xz is generally slow and doesn't do parallel good. If we want to change >>> this, we should go for something cool like zstd that scales better. >> >> That works for me. >> > > Let's make the default conditional on zstd USE flag as suggested in bug > 719456. It will be similar to how the ebuild enables FEATURES=xattr for > USE=xattr. In https://github.com/gentoo/gentoo/pull/15520 the issue was raised that it's probably not a good idea to change this default via the zstd USE flags, since changing the format of binary packages is kind of a big deal (it affects consumers of any binary packages that you build). I think it might be a good idea to make this new default apply only to new installations, like we did when changing default locations for bug 378603. If we're changing the default, then all architectures need keywords, so I've opened https://bugs.gentoo.org/721960. -- Thanks, Zac
[gentoo-portage-dev] [PATCH] NeededEntry: don't use scanelf -q (bug 721336)
We don't use scanelf -q, since that would omit libraries like
musl's /usr/lib/libc.so which do not have any DT_NEEDED or
DT_SONAME settings. Since we don't use scanelf -q, we have to
handle the special rpath value " - ".
Bug: https://bugs.gentoo.org/721336
Fixes: 25fbe7bc1a92 ("NeededEntry: infer implicit soname from file basename
(bug 715162)")
Signed-off-by: Zac Medico
---
bin/misc-functions.sh | 6 +-
lib/portage/util/_dyn_libs/LinkageMapELF.py | 5 -
lib/portage/util/_dyn_libs/NeededEntry.py | 5 +
3 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 9efe99b87..c2a16cbe0 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -177,7 +177,11 @@ install_qa_check() {
if type -P scanelf > /dev/null ; then
# Save NEEDED information after removing self-contained
providers
rm -f "$PORTAGE_BUILDDIR"/build-info/NEEDED{,.ELF.2}
- scanelf -qyRF '%a;%p;%S;%r;%n' "${D%/}/" | { while IFS= read -r
l; do
+ # We don't use scanelf -q, since that would omit libraries like
+ # musl's /usr/lib/libc.so which do not have any DT_NEEDED or
+ # DT_SONAME settings. Since we don't use scanelf -q, we have to
+ # handle the special rpath value " - " below.
+ scanelf -yRBF '%a;%p;%S;%r;%n' "${D%/}/" | { while IFS= read -r
l; do
arch=${l%%;*}; l=${l#*;}
obj="/${l%%;*}"; l=${l#*;}
soname=${l%%;*}; l=${l#*;}
diff --git a/lib/portage/util/_dyn_libs/LinkageMapELF.py
b/lib/portage/util/_dyn_libs/LinkageMapELF.py
index 2d4929445..473a1243d 100644
--- a/lib/portage/util/_dyn_libs/LinkageMapELF.py
+++ b/lib/portage/util/_dyn_libs/LinkageMapELF.py
@@ -272,7 +272,10 @@ class LinkageMapELF(object):
continue
plibs.update((x, cpv) for x in items)
if plibs:
- args = [os.path.join(EPREFIX or "/",
"usr/bin/scanelf"), "-qF", "%a;%F;%S;%r;%n"]
+ # We don't use scanelf -q, since that would omit
libraries like
+ # musl's /usr/lib/libc.so which do not have any
DT_NEEDED or
+ # DT_SONAME settings.
+ args = [os.path.join(EPREFIX or "/",
"usr/bin/scanelf"), "-BF", "%a;%F;%S;%r;%n"]
args.extend(os.path.join(root, x.lstrip("." + os.sep)) \
for x in plibs)
try:
diff --git a/lib/portage/util/_dyn_libs/NeededEntry.py
b/lib/portage/util/_dyn_libs/NeededEntry.py
index c52cfce3c..70ff99100 100644
--- a/lib/portage/util/_dyn_libs/NeededEntry.py
+++ b/lib/portage/util/_dyn_libs/NeededEntry.py
@@ -52,6 +52,11 @@ class NeededEntry(object):
del fields[cls._MIN_FIELDS:]
obj.arch, obj.filename, obj.soname, rpaths, needed = fields
+ # We don't use scanelf -q, since that would omit libraries like
+ # musl's /usr/lib/libc.so which do not have any DT_NEEDED or
+ # DT_SONAME settings. Since we don't use scanelf -q, we have to
+ # handle the special rpath value " - " below.
+ rpaths = "" if rpaths == " - " else rpaths
obj.runpaths = tuple(filter(None, rpaths.split(":")))
obj.needed = tuple(filter(None, needed.split(",")))
--
2.25.3
Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it
On Thu, 07 May 2020 09:29:36 +0200 Michał Górny wrote: > For example, if OCaml bindings on some package are broken and require > a lot of work, I would find useful to know how likely it is that anyone > is using it. Or if a lot of people are enabling 'frobnicate' flag, > I could consider employing USE defaults. For normal reporting, I'd suggest "counts of users" have some default presentation that encourages people to think of the data as incomplete. For example, instead of "0", it might print "<10", or say, "10: +/- 10" Or rank results in terms of relative numbers, "low", "high", etc. Or maybe incorporate time bounds with the information: "0 this month" Because even the people participating may not be participating frequently for all the niche things to turn up in every sample. Just working out a good way to calculate what the "error bars" should be is the hard part. pgpUKxGN_q6U5.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it
On Sat, 09 May 2020 15:22:52 +0200 Gerion Entrup wrote: > I'm not sure, if Portage is capable of this, but a distinction in USE > flags needed to fulfil some dependency of another package and USE flags > actively activated by the user could be useful. Presently impossible, as how portage implements the former, is by churning that information via either "plz sir, set this use flag in your config", or via auto-tweaking config to assert "I wanted this, you now want it". After that happens, the information as to /who/ specified that want is lost. At very best you can make some inferences based on the comments that get injected, but that's not anywhere near 100%, esp in turn-key approach, or, alternatively, assert that if a flag is specified in configure *and* something depends on that flag being set, then its the dependent, not the user but that really isn't true on a regular basis. For instance, uh, USE="X" (global) -> install Foo (w/ USE="X") Foo depends on Bar[X?] So is "Bar:X" required by the user, or by "Foo", or both? And does the answer to that question depend in any way on whether B (or Foo) declares IUSE="+X" or IUSE="X" ? pgpIoRLoR5lPw.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it
On Fri, 8 May 2020 09:49:18 +0200 Jaco Kroon wrote: > So we do need the full list of packages installed, filtered to ::gentoo, > but there needs to be an indicated whether it's installed because it's > in @world, as a dep of something in @world (which is possibly not in > ::gentoo), or is some form of no-longer needed dep. A dedicated report of orphans that are installed, from ::gentoo, would probably help here. Because you can't directly assume orphans are "user wanted", but they *can* be. That's why its important during --depclean to read the output and re-add any to @world you need kept. If you never depclean, then you get to skip that step. ( And I have *many* times added -1 to installation of something I wanted, out of habit, because I do so much manual hacking via emerge -1 that adding it is an impulse! ) pgpbZvnrigLQ0.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it
> > I think we shouldn't collect any data unless we have a good plan on how > we'd be able to use it. In this thread, I'd like to collect ideas > on what data to collect and how it could realistically be used. > 5) CFLAGS and possibly related variables 6) "active" version of slotted system packages (gcc, binutils, python, ???) I see this as interesting for the toolchain maintenance, but also interesting in general since we are a source-based distro. * How many users are running LTO? doing Profiling? building generic (- march=x86_64) packages? using -Os or -O3, -funroll-loop (just kidding) * How quick is gcc / binutils / ... adoption? * clang usage? -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, qa, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [gentoostats continued] Collected data and justification for it
Am Donnerstag, 7. Mai 2020, 09:29:36 CEST schrieb Michał Górny: > I'm going to start with the data and uses I can think of. Please reply > with other things you can think of. > > > 1) list of selected packages (@world) > > We would use this to determine the popularity of individual packages, > plus by scanning their dependencies we would be able to make combined > statistics for direct usage + dependencies of other selected packages. > This would allow us to judge which packages need more of our attention. > > For example, as we port Python packages to Python 3.8 the packages with > more declared users would be ported first. You may want to collect packages installed per sets, too. I mainly do what Hans mentioned in his mail to this thread but with sets. For example I have a KDE-PIM set that installs only my needed subset of the KDE PIM suite. (I also use this as common workaround for yet missing runtime dependencies / suggestions made by pkg_postinst.) Retrieval of this packages would be straight forward: Look at world_sets and collect all packages that are installed by the set. > 2) USE flags on installed packages (disabled/default/enabled) > > This would allow us to determine which flags users are most likely to > actually rely on. This could determine tested flag combinations, > defaults, and required level of support for individual flags. > > For example, if OCaml bindings on some package are broken and require > a lot of work, I would find useful to know how likely it is that anyone > is using it. Or if a lot of people are enabling 'frobnicate' flag, > I could consider employing USE defaults. I'm not sure, if Portage is capable of this, but a distinction in USE flags needed to fulfil some dependency of another package and USE flags actively activated by the user could be useful. Dependency use flags should be treated with a higher priority in my opinion, since they enable the installation of another package (tree), while use flags that enable a certain feature that is not used elsewhere are more "nice to have". Best Gerion signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] De-stabilizing and re-stabilizing (on amd64 only)
On Fri, 2020-05-08 at 13:33 +0300, Andreas K. Hüttel wrote: > > Background, I tried to locally emulate www.g.o using jekyll, and ran > into > troubles because lots of dev-ruby/* lost stable keywords. Newest > ~arch didn't > do the job, so I needed to figure out the config of www.g.o > (corresponding to > former stable) first... I'm not aware of amd64 stable keywords being lost in dev-ruby, other than dev-ruby/rails a long time ago. Which packages did you get in trouble with? And you can always file a stable bug for the maintainers :-) Hans signature.asc Description: This is a digitally signed message part
