Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables
Dnia 2014-12-06, o godz. 23:15:09 Zac Medico zmed...@gentoo.org napisał(a): On 12/05/2014 04:03 PM, Michał Górny wrote: Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo Portage, as fallbacks to sync-umask and sync-user repo keys. --- pym/portage/package/ebuild/_config/special_env_vars.py | 2 +- pym/portage/package/ebuild/config.py | 12 pym/portage/repository/config.py | 13 + 3 files changed, 26 insertions(+), 1 deletion(-) Why should we add backward compatibility code for something that was never supported in the master branch? I just thought it wouldn't hurt if we're backporting features. -- Best regards, Michał Górny pgpj1pzH8Pmeh.pgp Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] sync: allow overriding sync-umask for the repository
---
Changes:
* added a entry in the manpage
man/portage.5| 5 +
pym/portage/repository/config.py | 16
pym/portage/sync/controller.py | 3 +++
3 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/man/portage.5 b/man/portage.5
index 150294b..8c3d389 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -901,6 +901,11 @@ Valid non\-empty values: cvs, git, rsync
This attribute can be set to empty value to disable synchronization of given
repository. Empty value is default.
.TP
+.B sync\-umask
+Specifies umask used to synchronize the repository.
+.br
+Takes an octal permission mask, e.g. 022.
+.TP
.B sync\-uri
Specifies URI of repository used for synchronization performed by `emerge
\-\-sync`.
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index d37ce6a..678cc68 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,8 +85,9 @@ class RepoConfig(object):
'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
'name', 'portage1_profiles', 'portage1_profiles_compat',
'priority',
'profile_formats', 'sign_commit', 'sign_manifest',
'sync_cvs_repo',
- 'sync_type', 'sync_uri', 'thin_manifest', 'update_changelog',
- 'user_location', '_eapis_banned', '_eapis_deprecated',
'_masters_orig')
+ 'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+ 'update_changelog', 'user_location', '_eapis_banned',
+ '_eapis_deprecated', '_masters_orig')
def __init__(self, name, repo_opts, local_config=True):
Build a RepoConfig with options in repo_opts
@@ -154,6 +155,11 @@ class RepoConfig(object):
sync_type = sync_type.strip()
self.sync_type = sync_type or None
+ sync_umask = repo_opts.get('sync-umask')
+ if sync_umask is not None:
+ sync_umask = sync_umask.strip()
+ self.sync_umask = sync_umask or None
+
sync_uri = repo_opts.get('sync-uri')
if sync_uri is not None:
sync_uri = sync_uri.strip()
@@ -375,6 +381,8 @@ class RepoConfig(object):
repo_msg.append(indent + sync-cvs-repo: +
self.sync_cvs_repo)
if self.sync_type:
repo_msg.append(indent + sync-type: + self.sync_type)
+ if self.sync_umask:
+ repo_msg.append(indent + sync-umask: +
self.sync_umask)
if self.sync_uri:
repo_msg.append(indent + sync-uri: + self.sync_uri)
if self.masters:
@@ -464,7 +472,7 @@ class RepoConfigLoader(object):
# repos.conf is allowed to
override.
for k in ('aliases',
'auto_sync', 'eclass_overrides',
'force', 'masters',
'priority', 'sync_cvs_repo',
- 'sync_type', 'sync_uri',
+ 'sync_type',
'sync_umask', 'sync_uri',
):
v =
getattr(repos_conf_opts, k, None)
if v is not None:
@@ -915,7 +923,7 @@ class RepoConfigLoader(object):
def config_string(self):
str_or_int_keys = (auto_sync, format, location,
main_repo, priority, sync_cvs_repo,
- sync_type, sync_uri)
+ sync_type, sync_umask, sync_uri)
str_tuple_keys = (aliases, eclass_overrides, force)
repo_config_tuple_keys = (masters,)
keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 41f3830..1c8c756 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -221,6 +221,9 @@ class SyncManager(object):
if not st.st_mode 0o020:
umask = umask | 0o020
spawn_kwargs[umask] = umask
+ # override the defaults when sync_umask is set
+ if repo.sync_umask is not None:
+ spawn_kwargs[umask] = int(repo.sync_umask, 8)
self.spawn_kwargs = spawn_kwargs
if self.usersync_uid is not None:
--
2.2.0
[gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository
---
Changes:
* manpage entry
* perform username lookup before UID lookup (POSIX agrees)
* support specifying ':group' only
* fix ValueError on errors
man/portage.5| 12 ++
pym/portage/repository/config.py | 13 +++---
pym/portage/sync/controller.py | 51 +++-
3 files changed, 72 insertions(+), 4 deletions(-)
diff --git a/man/portage.5 b/man/portage.5
index 8c3d389..4a02c64 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -934,6 +934,18 @@ ssh://ssh\-user@192.168.0.1:22/\\${HOME}/portage\-storage
.TP
Note: For the ssh:// scheme, key\-based authentication might be of interest.
.RE
+.TP
+.B sync\-user
+Specifies the credentials used to perform the synchronization.
+.br
+Syntax: [user][:group]
+.br
+If only user is provided, the primary group of the user will be used.
+If only group is provided, the current user will be preserved and only
+group id will be changed.
+.br
+This key takes precedence over FEATURES=userpriv. If user or group id
+is provided, Portage no longer uses owner of the directory.
.RE
.I Example:
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index 678cc68..f45684b 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -85,7 +85,7 @@ class RepoConfig(object):
'main_repo', 'manifest_hashes', 'masters', 'missing_repo_name',
'name', 'portage1_profiles', 'portage1_profiles_compat',
'priority',
'profile_formats', 'sign_commit', 'sign_manifest',
'sync_cvs_repo',
- 'sync_type', 'sync_umask', 'sync_uri', 'thin_manifest',
+ 'sync_type', 'sync_umask', 'sync_uri', 'sync_user',
'thin_manifest',
'update_changelog', 'user_location', '_eapis_banned',
'_eapis_deprecated', '_masters_orig')
@@ -165,6 +165,11 @@ class RepoConfig(object):
sync_uri = sync_uri.strip()
self.sync_uri = sync_uri or None
+ sync_user = repo_opts.get('sync-user')
+ if sync_user is not None:
+ sync_user = sync_user.strip()
+ self.sync_user = sync_user or None
+
auto_sync = repo_opts.get('auto-sync')
if auto_sync is not None:
auto_sync = auto_sync.strip().lower()
@@ -385,6 +390,8 @@ class RepoConfig(object):
repo_msg.append(indent + sync-umask: +
self.sync_umask)
if self.sync_uri:
repo_msg.append(indent + sync-uri: + self.sync_uri)
+ if self.sync_user:
+ repo_msg.append(indent + sync-user: + self.sync_user)
if self.masters:
repo_msg.append(indent + masters: +
.join(master.name for master in self.masters))
if self.priority is not None:
@@ -472,7 +479,7 @@ class RepoConfigLoader(object):
# repos.conf is allowed to
override.
for k in ('aliases',
'auto_sync', 'eclass_overrides',
'force', 'masters',
'priority', 'sync_cvs_repo',
- 'sync_type',
'sync_umask', 'sync_uri',
+ 'sync_type',
'sync_umask', 'sync_uri', 'sync_user',
):
v =
getattr(repos_conf_opts, k, None)
if v is not None:
@@ -923,7 +930,7 @@ class RepoConfigLoader(object):
def config_string(self):
str_or_int_keys = (auto_sync, format, location,
main_repo, priority, sync_cvs_repo,
- sync_type, sync_umask, sync_uri)
+ sync_type, sync_umask, sync_uri, 'sync_user')
str_tuple_keys = (aliases, eclass_overrides, force)
repo_config_tuple_keys = (masters,)
keys = str_or_int_keys + str_tuple_keys + repo_config_tuple_keys
diff --git a/pym/portage/sync/controller.py b/pym/portage/sync/controller.py
index 1c8c756..de803b7 100644
--- a/pym/portage/sync/controller.py
+++ b/pym/portage/sync/controller.py
@@ -6,6 +6,7 @@ from __future__ import print_function
import sys
import logging
+import grp
import pwd
import portage
@@ -201,7 +202,55 @@ class SyncManager(object):
self.usersync_uid = None
spawn_kwargs = {}
spawn_kwargs[env] = self.settings.environ()
- if ('usersync' in self.settings.features and
+ if repo.sync_user is not None:
+ def get_sync_user_data(sync_user):
+ user = None
+ group = None
+
Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables
On 12/07/2014 12:22 AM, Michał Górny wrote: Dnia 2014-12-06, o godz. 23:15:09 Zac Medico zmed...@gentoo.org napisał(a): On 12/05/2014 04:03 PM, Michał Górny wrote: Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo Portage, as fallbacks to sync-umask and sync-user repo keys. --- pym/portage/package/ebuild/_config/special_env_vars.py | 2 +- pym/portage/package/ebuild/config.py | 12 pym/portage/repository/config.py | 13 + 3 files changed, 26 insertions(+), 1 deletion(-) Why should we add backward compatibility code for something that was never supported in the master branch? I just thought it wouldn't hurt if we're backporting features. Adding extra code for new redundant variables seems undesirable. Are they somehow better than using equivalent repos.conf [DEFAULT] settings for sync-user and sync-mask? -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH 4/4] sync: Add backwards compat with SYNC_{UMASK,USER} variables
Dnia 2014-12-07, o godz. 01:04:19 Zac Medico zmed...@gentoo.org napisał(a): On 12/07/2014 12:22 AM, Michał Górny wrote: Dnia 2014-12-06, o godz. 23:15:09 Zac Medico zmed...@gentoo.org napisał(a): On 12/05/2014 04:03 PM, Michał Górny wrote: Support SYNC_UMASK and SYNC_USER variables that were used in Funtoo Portage, as fallbacks to sync-umask and sync-user repo keys. --- pym/portage/package/ebuild/_config/special_env_vars.py | 2 +- pym/portage/package/ebuild/config.py | 12 pym/portage/repository/config.py | 13 + 3 files changed, 26 insertions(+), 1 deletion(-) Why should we add backward compatibility code for something that was never supported in the master branch? I just thought it wouldn't hurt if we're backporting features. Adding extra code for new redundant variables seems undesirable. Are they somehow better than using equivalent repos.conf [DEFAULT] settings for sync-user and sync-mask? They're easier to set in env than the fancy PORTAGE_REPOSITORIES. -- Best regards, Michał Górny pgpFJSwEfzAJM.pgp Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository
Dnia 2014-12-07, o godz. 10:01:23
Michał Górny mgo...@gentoo.org napisał(a):
+ if uid is not None
Missing ':' here, I won't resend the patch for it :P.
+ spawn_kwargs[uid] = uid
+ self.usersync_uid = uid
+ if gid is not None:
+ spawn_kwargs[gid] = gid
+ spawn_kwargs[groups] = [gid]
+ if home is not None:
+ spawn_kwargs[env][HOME] = home
+ elif ('usersync' in self.settings.features and
portage.data.secpass = 2 and
(st.st_uid != os.getuid() and st.st_mode 0o700 or
st.st_gid != os.getgid() and st.st_mode 0o070)):
--
Best regards,
Michał Górny
pgpfAH2D5JsIO.pgp
Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On Sat, 06 Dec 2014, Zac Medico wrote: The PMS people should be *very* interested in any changes to unpack behavior like this. It supports behavior that will lead to failures for older versions of portage and other package managers. Some remarks: - The upstream deb2targz program supports only data.tar.gz. Only the Gentoo version has been patched up to support other file types. - There is only a single package in the tree with a dependency on app-arch/deb2targz, and this only on AIX [1]. So on most users' systems deb2targz will not be installed, and ar x will be used for unpacking. - Paludis and Pkgcore unconditionally use ar x for unpacking. - PMS says [2]: deb packages (*.deb). Ebuilds must ensure that the deb2targz program is installed on those platforms where the GNU binutils ar program is not available and the installed ar program is incompatible with GNU archives. Otherwise, ebuilds must ensure that GNU binutils is installed. One could conclude that GNU binutils ar should be used on platforms where it is available. deb2targz should be used only where ar is incompatible with the GNU version. So in any case, this is a problem of Portage only. I'd suggest to change the logic in unpack() as follows: if ar is GNU; then ar x ... elif deb2targz installed; then deb2targz ... else ar x ... fi And then the deb2targz case could be patched as proposed. After all, it is only a fallback that should be compatible with ar x extraction. Ulrich [1] http://qa-reports.gentoo.org/output/genrdeps/dindex/app-arch/deb2targz [2] http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-14100011.3.3.13 pgph26utPMmUN.pgp Description: PGP signature
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/07/2014 03:04 AM, Ulrich Mueller wrote: On Sat, 06 Dec 2014, Zac Medico wrote: The PMS people should be *very* interested in any changes to unpack behavior like this. It supports behavior that will lead to failures for older versions of portage and other package managers. Some remarks: - The upstream deb2targz program supports only data.tar.gz. Only the Gentoo version has been patched up to support other file types. - There is only a single package in the tree with a dependency on app-arch/deb2targz, and this only on AIX [1]. So on most users' systems deb2targz will not be installed, and ar x will be used for unpacking. - Paludis and Pkgcore unconditionally use ar x for unpacking. - PMS says [2]: deb packages (*.deb). Ebuilds must ensure that the deb2targz program is installed on those platforms where the GNU binutils ar program is not available and the installed ar program is incompatible with GNU archives. Otherwise, ebuilds must ensure that GNU binutils is installed. One could conclude that GNU binutils ar should be used on platforms where it is available. deb2targz should be used only where ar is incompatible with the GNU version. So in any case, this is a problem of Portage only. I'd suggest to change the logic in unpack() as follows: if ar is GNU; then ar x ... elif deb2targz installed; then deb2targz ... else ar x ... fi And then the deb2targz case could be patched as proposed. After all, it is only a fallback that should be compatible with ar x extraction. Ulrich [1] http://qa-reports.gentoo.org/output/genrdeps/dindex/app-arch/deb2targz [2] http://dev.gentoo.org/~ulm/pms/5/pms.html#x1-14100011.3.3.13 Okay, so we may want to examine this AIX case more closely, because dropping support for AIX would solve our problem with the lack of xz support in deb2targz. Anyway, Brian's patch for xz support with debt2targz appears to be compatible with other package managers as well as AIX, so that seems like a good way to go. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/06/2014 04:52 PM, Brian Dolbec wrote: From 4cb661d994cf8503c4459b8f7da7ee0f739a9826 Mon Sep 17 00:00:00 2001 From: Brian Dolbec dol...@gentoo.org Date: Sat, 6 Dec 2014 14:51:13 -0800 Subject: [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LGTM, given that other package managers default to 'ar x' which already works with xz files. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On Sun, 07 Dec 2014, Zac Medico wrote: Anyway, Brian's patch for xz support with debt2targz appears to be compatible with other package managers as well as AIX, so that seems like a good way to go. I still think that ar should always be used on platforms where GNU binutils is installed. Currently unpack uses deb2targz when it finds the program, which is not a well defined behaviour. Ulrich pgppXonqP1VjG.pgp Description: PGP signature
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/07/2014 10:05 AM, Ulrich Mueller wrote: On Sun, 07 Dec 2014, Zac Medico wrote: Anyway, Brian's patch for xz support with debt2targz appears to be compatible with other package managers as well as AIX, so that seems like a good way to go. I still think that ar should always be used on platforms where GNU binutils is installed. Currently unpack uses deb2targz when it finds the program, which is not a well defined behaviour. Okay, I guess we can default to ar if [[ $(ar --version 21) == GNU ar* ]] and otherwise fall back to deb2targz. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/07/2014 09:54 AM, Zac Medico wrote: On 12/06/2014 04:52 PM, Brian Dolbec wrote: From 4cb661d994cf8503c4459b8f7da7ee0f739a9826 Mon Sep 17 00:00:00 2001 From: Brian Dolbec dol...@gentoo.org Date: Sat, 6 Dec 2014 14:51:13 -0800 Subject: [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LGTM, given that other package managers default to 'ar x' which already works with xz files. Based on discussion with Ulrich Mueller I think we should default to ar if [[ $(ar --version 21) == GNU ar* ]] and otherwise fall back to deb2targz. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository
On 12/07/2014 01:07 AM, Michał Górny wrote:
Dnia 2014-12-07, o godz. 10:01:23
Michał Górny mgo...@gentoo.org napisał(a):
+if uid is not None
Missing ':' here, I won't resend the patch for it :P.
+spawn_kwargs[uid] = uid
+self.usersync_uid = uid
+if gid is not None:
+spawn_kwargs[gid] = gid
+spawn_kwargs[groups] = [gid]
+if home is not None:
+spawn_kwargs[env][HOME] = home
+elif ('usersync' in self.settings.features and
portage.data.secpass = 2 and
(st.st_uid != os.getuid() and st.st_mode 0o700 or
st.st_gid != os.getgid() and st.st_mode 0o070)):
LGTM.
--
Thanks,
Zac
Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/07/2014 11:23 AM, Fabian Groffen wrote: On 07-12-2014 10:49:30 -0800, Zac Medico wrote: On 12/07/2014 10:47 AM, Ulrich Mueller wrote: On Sun, 07 Dec 2014, Zac Medico wrote: On 12/07/2014 10:37 AM, Ulrich Mueller wrote: It's sort of trivial, but here is a patch: From c53e7057f94728d6e0c7d16c675702ca831b9a5a Mon Sep 17 00:00:00 2001 From: Ulrich Müller u...@gentoo.org Date: Sun, 7 Dec 2014 19:33:52 +0100 Subject: [PATCH] Prefer GNU ar when unpacking .deb packages. [...] LGTM. So shall I push it? Yes, please do. FYI: % portageq envvar CHOST x86_64-apple-darwin13 % ar --version ar: illegal option -- - usage: ar -d [-TLsv] archive file ... ar -m [-TLsv] archive file ... ar -m [-abiTLsv] position archive file ... ar -p [-TLsv] archive [file ...] ar -q [-cTLsv] archive file ... ar -r [-cuTLsv] archive file ... ar -r [-abciuTLsv] position archive file ... ar -t [-TLsv] archive [file ...] ar -x [-ouTLsv] archive [file ...] % ar x $EPREFIX/usr/portage/distfiles/realpath_1.16_i386.deb % ls control.tar.gz data.tar.gz debian-binary deb2targz should work (but I don't have it installed, so not sure if tested). Fabian It's hard to whitelist it if doesn't support --version, so it seems reasonable to require deb2targz whenever ar is not GNU ar. -- Thanks, Zac
Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On Sun, 07 Dec 2014, Zac Medico wrote: On 12/07/2014 11:23 AM, Fabian Groffen wrote: FYI: % portageq envvar CHOST x86_64-apple-darwin13 % ar --version ar: illegal option -- - usage: ar -d [-TLsv] archive file ... [...] It's hard to whitelist it if doesn't support --version, so it seems reasonable to require deb2targz whenever ar is not GNU ar. unpack will still use ar if deb2targz is not installed. So on Darwin, behaviour should be the same as before. Ulrich pgpPQEfcOLwSJ.pgp Description: PGP signature
Re: [gentoo-alt] Re: [gentoo-portage-dev] [PATCH 1/2] bin/phase-helpers.shL Fix unpack() for deb2targz .xz types
On 12/07/2014 11:41 AM, Ulrich Mueller wrote: On Sun, 07 Dec 2014, Zac Medico wrote: On 12/07/2014 11:23 AM, Fabian Groffen wrote: FYI: % portageq envvar CHOST x86_64-apple-darwin13 % ar --version ar: illegal option -- - usage: ar -d [-TLsv] archive file ... [...] It's hard to whitelist it if doesn't support --version, so it seems reasonable to require deb2targz whenever ar is not GNU ar. unpack will still use ar if deb2targz is not installed. So on Darwin, behaviour should be the same as before. Ulrich Ah, that's a nice feature. That way, we can just leave it to platforms like AIX to pull in deb2targz when necessary. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH] sync: allow overriding sync-user for the repository
I would suggest to have a separate sync-group attribute and to support only user in sync-user attribute. signature.asc Description: This is a digitally signed message part.
