[gentoo-portage-dev] [PATCH] _unmerge_dirs: revisit parents of removed symlinks (bug 640058)
When removal of a symlink is triggered by removal of the directory
that it points to, revisit the parent directories of the symlink.
Bug: https://bugs.gentoo.org/640058
---
pym/portage/dbapi/vartree.py | 23 +--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index 1a86940f1..43e3c4f1a 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -2753,9 +2753,13 @@ class dblink(object):
real_root = self.settings['ROOT']
dirs = sorted(dirs)
- dirs.reverse()
+ revisit = {}
- for obj, inode_key in dirs:
+ while True:
+ try:
+ obj, inode_key = dirs.pop()
+ except IndexError:
+ break
# Treat any directory named "info" as a candidate here,
# since it might have been in INFOPATH previously even
# though it may not be there now.
@@ -2818,6 +2822,7 @@ class dblink(object):
raise
if e.errno != errno.ENOENT:
show_unmerge("---",
unmerge_desc["!empty"], "dir", obj)
+ revisit[obj] = inode_key
# Since we didn't remove this directory, record
the directory
# itself for use in syncfs calls, if we have
removed another
@@ -2838,6 +2843,7 @@ class dblink(object):
# no need to protect symlinks that point to it.
unmerge_syms =
protected_symlinks.pop(inode_key, None)
if unmerge_syms is not None:
+ parents = []
for relative_path in unmerge_syms:
obj = os.path.join(real_root,
relative_path.lstrip(os.sep))
@@ -2849,6 +2855,19 @@ class dblink(object):
raise
del e
show_unmerge("!!!", "",
"sym", obj)
+ else:
+
parents.append(os.path.dirname(obj))
+
+ if parents:
+ # Revisit parents recursively
(bug 640058).
+ recursive_parents = []
+ for parent in set(parents):
+ while parent in revisit:
+
recursive_parents.append(parent)
+ parent =
os.path.dirname(parent)
+
+ for parent in
sorted(set(recursive_parents)):
+ dirs.append((parent,
revisit.pop(parent)))
def isowner(self, filename, destroot=None):
"""
--
2.13.6
Re: [gentoo-dev] [RFC] Requiring gentoo.git committers to use their @gentoo.org address
On Thu, 12 Jul 2018 08:35:57 +0200 Michał Górny wrote: > ...and the git hook would've rejected them because they aren't signed > by a Gentoo developer. Fair enough. I suspect in line with the other comments, it would be optimal if said commit hook mentioned something along the lines of: "Perhaps try reforging signature/committer data with git rebase --no-ff origin/master" In its failure message? Making the way forward obvious in this edge case would remove any residual cause for partial objections :) pgphK7IeSwSph.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, 12 Jul 2018 17:35:41 -0700 Raymond Jennings wrote: > On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec > wrote: > > > > On Thu, 12 Jul 2018 11:49:37 -0700 > > Raymond Jennings wrote: > > > > > In that case, I vote for /var/cache/portage, since that's > > > literally what purpose it serves. Namely, the cache of the > > > gentoo infra's current copy of teh portage tree. > > > > > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner > > > wrote: > > > > > > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings > > > > wrote: > > > >> > > > >> Just for the record, but would putting a setting inside > > > >> /etc/portage/make.conf be the appropriate way to handle this? > > > >> > > > > > > > > The settings already exist (and have existed for 10 years.) This > > > > bikeshed discussion is literally trying to decide what the > > > > default should be. > > > > > > > > -A > > > > > > > > > > > This is not a personal attack against you. Just picked this one to > > say something again... > > > > > > PLEASE, PLEASE stop calling it the "portage" tree. The repo name is > > "gentoo". "portage is the default package manager, but not the only > > choice. Far too often, it takes awhile to figure out what someone > > is trying to say because of that confusion between the tree and the > > package manager. > > Point of order: > > http://distfiles.gentoo.org/snapshots and numerous pieces of > documentation call it "portage" > > The confusion is ingrained by documentation. > Yes, it is, and well we can't very well change the documentation until we can get an end to this new default path bikeshed. Council will need to make the decision (soon I hope)... Then we make all the changes necessary. -- Brian Dolbec
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, Jul 12, 2018 at 12:47 PM Brian Dolbec wrote: > > On Thu, 12 Jul 2018 11:49:37 -0700 > Raymond Jennings wrote: > > > In that case, I vote for /var/cache/portage, since that's literally > > what purpose it serves. Namely, the cache of the gentoo infra's > > current copy of teh portage tree. > > > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner > > wrote: > > > > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings > > > wrote: > > >> > > >> Just for the record, but would putting a setting inside > > >> /etc/portage/make.conf be the appropriate way to handle this? > > >> > > > > > > The settings already exist (and have existed for 10 years.) This > > > bikeshed discussion is literally trying to decide what the default > > > should be. > > > > > > -A > > > > > > > This is not a personal attack against you. Just picked this one to say > something again... > > > PLEASE, PLEASE stop calling it the "portage" tree. The repo name is > "gentoo". "portage is the default package manager, but not the only > choice. Far too often, it takes awhile to figure out what someone is > trying to say because of that confusion between the tree and the > package manager. Point of order: http://distfiles.gentoo.org/snapshots and numerous pieces of documentation call it "portage" The confusion is ingrained by documentation. > PLUS, it has been decided already long ago that the directory name > should reflect the repository name. We have been enforcing that rule > for overlays for a long time. It has just been taking a long time to > get our tooling in order so that we can change our own to follow that > rule. > > So, "portage" should not be a directory name in the new default path. > > -- > Brian Dolbec > >
Re: [gentoo-dev] rfc: moving default location of portage tree
> I guess /var/portage is not a terrible choice. Well, I double that. I've already use the following structure: |- /var/portage/ | |- repos | | |- gentoo | | |- reponame1 | | |- reponame2 | |- distfiles | | |- ... | | |- ... | |- packages | | |- ... | | |- ... | |- meta | |- layman | |- ... # (used to be used for layman stuff until eselect-repository) And I think it's pretty fine and usefull, especially because distfiles/ packages does not belong to one specific repo (gentoo) as it does with /usr/ portage.
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, Jul 12, 2018 at 10:13:57PM +0200, Michał Górny wrote: > W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman > napisał: > > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec wrote: > > > > > > So, "portage" should not be a directory name in the new default path. > > > > > > > Well, in my examples I proposed it as that is the software that > > created the path, but then again in the spirit of PMS portage isn't > > the only PM. > > > > So: > > /var/lib/repos/gentoo ? > > > > Subdirectories of /var/lib should be named after the tool/package name. > There's no tool or package called 'repos'. Technically mgorny is correct here. FHS requires that everything under /var/lib be under a directory for the package or for the distro [1]. Note the comment about packaging support in 5.8.1. Based on that this is my thought: * /var/lib/portage is for portage specific stuff -- maybe even /var/db/pkg in the future should go to /var/lib/portage/pkg. * /var/lib/gentoo, on the other hand, could be where repos, distfiles and binpkgs go. William [1] http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: moving default location of portage tree
W dniu czw, 12.07.2018 o godzinie 15∶51 -0400, użytkownik Rich Freeman napisał: > On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec wrote: > > > > So, "portage" should not be a directory name in the new default path. > > > > Well, in my examples I proposed it as that is the software that > created the path, but then again in the spirit of PMS portage isn't > the only PM. > > So: > /var/lib/repos/gentoo ? > Subdirectories of /var/lib should be named after the tool/package name. There's no tool or package called 'repos'. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, Jul 12, 2018 at 3:47 PM Brian Dolbec wrote: > > So, "portage" should not be a directory name in the new default path. > Well, in my examples I proposed it as that is the software that created the path, but then again in the spirit of PMS portage isn't the only PM. So: /var/lib/repos/gentoo ? -- Rich
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, 12 Jul 2018 11:49:37 -0700 Raymond Jennings wrote: > In that case, I vote for /var/cache/portage, since that's literally > what purpose it serves. Namely, the cache of the gentoo infra's > current copy of teh portage tree. > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner > wrote: > > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings > > wrote: > >> > >> Just for the record, but would putting a setting inside > >> /etc/portage/make.conf be the appropriate way to handle this? > >> > > > > The settings already exist (and have existed for 10 years.) This > > bikeshed discussion is literally trying to decide what the default > > should be. > > > > -A > > > This is not a personal attack against you. Just picked this one to say something again... PLEASE, PLEASE stop calling it the "portage" tree. The repo name is "gentoo". "portage is the default package manager, but not the only choice. Far too often, it takes awhile to figure out what someone is trying to say because of that confusion between the tree and the package manager. PLUS, it has been decided already long ago that the directory name should reflect the repository name. We have been enforcing that rule for overlays for a long time. It has just been taking a long time to get our tooling in order so that we can change our own to follow that rule. So, "portage" should not be a directory name in the new default path. -- Brian Dolbec
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, Jul 12, 2018 at 3:34 PM konsolebox wrote: > > I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf. > Regardless of the base directory location, I might suggest a path dedicated to repositories, of which the main gentoo repo is just an initial one, and overlays could be placed as additional directories inside. /var/lib/portage/repositories/main /var/lib/gentoo/repositories/main /var/cache/gentoo/repos/main /var/cache/portage/repositories/gentoo (Something along those lines.) -- Rich
Re: [gentoo-dev] rfc: moving default location of portage tree
I have /var/lib/gentoo/portage defined in repos.conf/gentoo.conf. On Fri, Jul 13, 2018, 2:50 AM Raymond Jennings wrote: > In that case, I vote for /var/cache/portage, since that's literally > what purpose it serves. Namely, the cache of the gentoo infra's > current copy of teh portage tree. > > On Thu, Jul 12, 2018 at 11:00 AM Alec Warner wrote: > > > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings > wrote: > >> > >> Just for the record, but would putting a setting inside > >> /etc/portage/make.conf be the appropriate way to handle this? > >> > > > > The settings already exist (and have existed for 10 years.) This > bikeshed discussion is literally trying to decide what the default should > be. > > > > -A > > > >
Re: [gentoo-dev] rfc: moving default location of portage tree
In that case, I vote for /var/cache/portage, since that's literally what purpose it serves. Namely, the cache of the gentoo infra's current copy of teh portage tree. On Thu, Jul 12, 2018 at 11:00 AM Alec Warner wrote: > > On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings wrote: >> >> Just for the record, but would putting a setting inside >> /etc/portage/make.conf be the appropriate way to handle this? >> > > The settings already exist (and have existed for 10 years.) This bikeshed > discussion is literally trying to decide what the default should be. > > -A >
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thu, Jul 12, 2018 at 1:43 PM, Raymond Jennings wrote: > Just for the record, but would putting a setting inside > /etc/portage/make.conf be the appropriate way to handle this? > > The settings already exist (and have existed for 10 years.) This bikeshed discussion is literally trying to decide what the default should be. -A
Re: [gentoo-dev] rfc: moving default location of portage tree
Just for the record, but would putting a setting inside /etc/portage/make.conf be the appropriate way to handle this?
Re: [gentoo-portage-dev] [PATCH] dbapi: fix repoman implicit IUSE (bug 660982)
On Thu, 12 Jul 2018 02:59:03 -0700 Zac Medico wrote: > Account for repoman modifications of the portdbapi self.settings > reference, and treat all flags as valid for the empty profile > because it does not have any implicit IUSE settings. > > Bug: https://bugs.gentoo.org/660982 > --- > pym/_emerge/Package.py| 5 - > pym/portage/dbapi/__init__.py | 16 > 2 files changed, 20 insertions(+), 1 deletion(-) > > diff --git a/pym/_emerge/Package.py b/pym/_emerge/Package.py > index a7ce00bc9..5f34f3d27 100644 > --- a/pym/_emerge/Package.py > +++ b/pym/_emerge/Package.py > @@ -93,7 +93,10 @@ class Package(Task): > # sync metadata with validated repo (may be > UNKNOWN_REPO) self._metadata['repository'] = self.cpv.repo > > - implicit_match = db._iuse_implicit_cnstr(self.cpv, > self._metadata) > + if self.root_config.settings.local_config: > + implicit_match = > db._iuse_implicit_cnstr(self.cpv, self._metadata) > + else: > + implicit_match = > db._repoman_iuse_implicit_cnstr(self.cpv, self._metadata) usealiases > = self.root_config.settings._use_manager.getUseAliases(self) > self.iuse = self._iuse(self, self._metadata["IUSE"].split(), > implicit_match, usealiases, self.eapi) diff --git > a/pym/portage/dbapi/__init__.py b/pym/portage/dbapi/__init__.py index > d320cc75f..61d301839 100644 --- a/pym/portage/dbapi/__init__.py > +++ b/pym/portage/dbapi/__init__.py > @@ -216,6 +216,22 @@ class dbapi(object): > > yield cpv > > + def _repoman_iuse_implicit_cnstr(self, pkg, metadata): > + """ > + In repoman's version of _iuse_implicit_cnstr, > account for modifications > + of the self.settings reference between calls, and > treat all flags as > + valid for the empty profile because it does not have > any implicit IUSE > + settings. See bug 660982. > + """ > + eapi_attrs = _get_eapi_attrs(metadata["EAPI"]) > + if eapi_attrs.iuse_effective: > + iuse_implicit_match = lambda flag: (True if > not self.settings.profile_path > + else > self.settings._iuse_effective_match(flag)) > + else: > + iuse_implicit_match = lambda flag: (True if > not self.settings.profile_path > + else > self.settings._iuse_implicit_match(flag)) > + return iuse_implicit_match > + > def _iuse_implicit_cnstr(self, pkg, metadata): > """ > Construct a callable that checks if a given USE flag > should looks good thanks. Please add the test case ebuild that was supplied to the repoman gen-b0rk repo "not-broken" category. https://gitweb.gentoo.org/repo/proj/gen-b0rk.git/ -- Brian Dolbec
Re: [gentoo-dev] rfc: moving default location of portage tree
On Wed, Jul 11, 2018 at 11:16 PM William Hubbs wrote: > > That is the other part of this debate, some are saying /var/lib, and > others are saying /var/db. > > It turns out that /var/db is much more common than I thought it was > (it exists in all *bsd variants at least), so that could be an argument > for putting the repos in there. > FreeBSD does not put the package repository in /var/db. Portage cloned many of the file paths from FreeBSD along with the name/concept. FreeBSD puts their package repository in /usr/ports, so I'm not sure I'd hold them out as a great example of FHS. -- Rich
Re: [gentoo-dev] rfc: moving default location of portage tree
Am Mittwoch, 11. Juli 2018, 18:19:39 CEST schrieb Alec Warner: > [...] > > +1 to this. The challenge (in moving it) is that its been "/usr/portage" > for a long time so many tools > may have hard coded this location; as opposed to querying portage for where > the tree is, e.g.: > > PORTDIR=$(portageq get_repo_path / gentoo) > > -A Some people including myself moved the tree to /var by variable definitions (and not wild mounting) a while ago. This configuration *is* supported for a while now but not the default and if tools break they have to be fixed anyway. (Side note: At least most of the common tools like gentoolkit parts or repoman work on my machines with the moved tree.) - Nils -- GPG fingerprint: '766B 8122 1342 6912 3401 492A 8B54 D7A3 FF3C DB17' Holgersson signature.asc Description: This is a digitally signed message part.
[gentoo-portage-dev] [PATCH] dbapi: fix repoman implicit IUSE (bug 660982)
Account for repoman modifications of the portdbapi self.settings reference, and treat all flags as valid for the empty profile because it does not have any implicit IUSE settings. Bug: https://bugs.gentoo.org/660982 --- pym/_emerge/Package.py| 5 - pym/portage/dbapi/__init__.py | 16 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/pym/_emerge/Package.py b/pym/_emerge/Package.py index a7ce00bc9..5f34f3d27 100644 --- a/pym/_emerge/Package.py +++ b/pym/_emerge/Package.py @@ -93,7 +93,10 @@ class Package(Task): # sync metadata with validated repo (may be UNKNOWN_REPO) self._metadata['repository'] = self.cpv.repo - implicit_match = db._iuse_implicit_cnstr(self.cpv, self._metadata) + if self.root_config.settings.local_config: + implicit_match = db._iuse_implicit_cnstr(self.cpv, self._metadata) + else: + implicit_match = db._repoman_iuse_implicit_cnstr(self.cpv, self._metadata) usealiases = self.root_config.settings._use_manager.getUseAliases(self) self.iuse = self._iuse(self, self._metadata["IUSE"].split(), implicit_match, usealiases, self.eapi) diff --git a/pym/portage/dbapi/__init__.py b/pym/portage/dbapi/__init__.py index d320cc75f..61d301839 100644 --- a/pym/portage/dbapi/__init__.py +++ b/pym/portage/dbapi/__init__.py @@ -216,6 +216,22 @@ class dbapi(object): yield cpv + def _repoman_iuse_implicit_cnstr(self, pkg, metadata): + """ + In repoman's version of _iuse_implicit_cnstr, account for modifications + of the self.settings reference between calls, and treat all flags as + valid for the empty profile because it does not have any implicit IUSE + settings. See bug 660982. + """ + eapi_attrs = _get_eapi_attrs(metadata["EAPI"]) + if eapi_attrs.iuse_effective: + iuse_implicit_match = lambda flag: (True if not self.settings.profile_path + else self.settings._iuse_effective_match(flag)) + else: + iuse_implicit_match = lambda flag: (True if not self.settings.profile_path + else self.settings._iuse_implicit_match(flag)) + return iuse_implicit_match + def _iuse_implicit_cnstr(self, pkg, metadata): """ Construct a callable that checks if a given USE flag should -- 2.13.6
[gentoo-dev] Last rites: dev-python/promise
# Michał Górny (12 Jul 2018) # Abandoned upstream, no reverse dependencies. Collides with # dev-python/promises (that has revdeps). # Removal in 30 days. Bug #651156. dev-python/promise -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [RFC] Requiring gentoo.git committers to use their @gentoo.org address
W dniu czw, 12.07.2018 o godzinie 15∶52 +1200, użytkownik Kent Fredric napisał: > On Mon, 09 Jul 2018 10:40:22 +0200 > Michał Górny wrote: > > > Hi, > > > > We currently don't enforce any particular standard for e-mail addresses > > for developers committing to gentoo.git. FWICS, the majority of > > developers is using their @gentoo.org e-mail addresses. However, a few > > developers are using some other addresses. > > > > Using n...@gentoo.org e-mail addresses generally causes problems > > in accounting for commits. For example, our retirement scripts can't > > detect commits made using non-Gentoo e-mail address. My dev-timeline > > scripts [1] account for all emails in LDAP (which doesn't cover all > > addresses developers use). FWIK gkeys accounts for all addresses > > in the OpenPGP key UIDs. In my opinion, that's a lot of hoops to jump > > through to workaround bad practice. > > > > Therefore, I'd like to start enforcing (at the level of the hook > > verifying signatures) that all commits made to gentoo.git (and other > > repositories requiring dev signatures) are made using @gentoo.org e-mail > > address (for committer field). > > > > Is anyone opposed to that? Does anyone know of a valid reason to use > > n...@gentoo.org address when committing? > > > > [1]:https://dev.gentoo.org/~mgorny/dev-timeline.html > > > > There's one fun problem here technologically for proxy-maint, but > getting the conditions right for it to occur happen very rarely. > > 1. Assume the proxied maintainer has a git repo, where they commit > themselves. > > 2. Assume their proxy has said git repo as an alternative remote, for > which they relay work. ( That is, they work closely together directly > instead of via github pull requests and textual patches ) > > 3. ::gentoo is quiet, and the proxied maintainer has rebased their own > work on top of ::gentoo, setting Committer: metadata and signing > commits. > > Then, in that situation, it is trivial for the proxy to relay those > commits verbatim to ::gentoo, without changing either Committer: or > signature data. ...and the git hook would've rejected them because they aren't signed by a Gentoo developer. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [RFC] Requiring gentoo.git committers to use their @gentoo.org address
W dniu wto, 10.07.2018 o godzinie 09∶38 -0400, użytkownik kuzetsa napisał: > > The only issue I see is that of slight complications on handling the > > different addresses for author and commit, that's all that comes to > > mind. > > > > > > Mart > > > > I think authorship is a good point / distinction, Mart. > > Authorship was never shown in dev-timeline for addresses > which aren't @gentoo.org anyway. That's a separate issue, > so this policy change shouldn't affect proxy-maint? > For the record, dev-timeline accounts for both authors and committers. However, it only displays those who have been Gentoo developers at some point (and whose alternate e-mail addresses are in LDAP). -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] rfc: moving default location of portage tree
On Thursday, 12 July 2018 07:21:20 CEST Ulrich Mueller wrote: > > On Wed, 11 Jul 2018, Richard Yao wrote: > > This does not answer my question. Is it really a FHS violation? The > > contents of /usr changes when doing updates using the system package > > manager. When not doing updates, it really is readonly and the FHS > > says that /usr is for readonly things. I do not see how it is > > different from anything else in /usr. > > What about a system that is building binpkgs? Or an rsync mirror? There are systems building software for other systems using $ROOT, too. They need to download into distfiles and sometimes they build binpkgs, too. signature.asc Description: This is a digitally signed message part.
