Re: [gentoo-portage-dev] portage user not listed in /etc/shadow ?

[M. J. Everitt]

On 16/01/19 12:59, Joakim Tjernlund wrote:
> On Wed, 2019-01-16 at 12:20 +0000, M. J. Everitt wrote:
>> On 16/01/19 11:58, Joakim Tjernlund wrote:
>>> How come portage isn't in shadow, only in passwd ?
>>> Seems wrong to me.
>>>
>>>  Jocke
>> Because the portage user never logs on .. hence has no password. That said,
> That goes for evry other system account too but these are in shadow.
>
>> it does seem an odd situation, since even users with no password do tend to
>> appear in /etc/shadow .. perhaps it was/is never "properly" added as a user
>> 
> I think/hope so. Now passwd looks like so:
> portage:x:250:250:portage:/var/tmp/portage:/bin/false
> This looks like a shadow account. Because of the missing entry passwd
> thinks this is a normal account:
> # > passwd -S portage 
> portage P
> as opposed to:
> # > passwd  -S daemon
> daemon L 10/28/1996 0 -1 -1 -1
>
>  Jocke
With the help of some devs, I have drilled this down to commit
https://gitweb.gentoo.org/proj/baselayout.git/commit/share.Linux/shadow?id=5ee3c95d2086e626247640ca35cf2da78c4c9846
in baselayout in 2016.

Some of my systems mysteriously (but predictably) are missing portage in
/etc/shadow as you describe, but these are prior to the baselayout change
above. Many have it as the commit suggests also. I suspect there may not
have been an upgrade path which added it between versions however, unless
it was pulled in via CONFIG_PROTECT somehow

Bug https://bugs.gentoo.org/show_bug.cgi?id=521970 is also referenced in
that commit.

Hope that answers your question (and my curiosity!).
Regards,

veremitz/Michael.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] portage user not listed in /etc/shadow ?

[M. J. Everitt]

On 16/01/19 11:58, Joakim Tjernlund wrote:
> How come portage isn't in shadow, only in passwd ?
> Seems wrong to me.
>
>  Jocke
Because the portage user never logs on .. hence has no password. That said,
it does seem an odd situation, since even users with no password do tend to
appear in /etc/shadow .. perhaps it was/is never "properly" added as a user


MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval

[M. J. Everitt]

On 08/01/19 19:15, Zac Medico wrote:
> On 1/8/19 5:42 AM, Fabian Groffen wrote:
>> The reporting of files remaining can look somewhat odd since the report
>> interval is hardcoded to be per 1000 objects.  Adjust this interval to
>> be time based.  This means that modern (fast) machines likely will never
>> see the countdown messages at all.  On slow setups the message will be
>> informative that there is progress, albeit rather slowly.  While at it,
>> report percentage done.
>>
>> Output before this patch:
>>
>>  * checking 6158 files for package collisions
>> 5158 files remaining ...
>> 4158 files remaining ...
>> 3158 files remaining ...
>> 2158 files remaining ...
>> 1158 files remaining ...
>> 158 files remaining ...
>>
>> Possible output after this patch on a slower machine:
>>
>>  * checking 6158 files for package collisions
>>  48% done, 3145 files remaining ...
>>  96% done, 192 files remaining ...
>> 100% done
>>
>> Signed-off-by: Fabian Groffen 
>> ---
>>  lib/portage/dbapi/vartree.py | 11 +--
>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
>> index 4b91caea8..244195fad 100644
>> --- a/lib/portage/dbapi/vartree.py
>> +++ b/lib/portage/dbapi/vartree.py
>> @@ -3475,13 +3475,18 @@ class dblink(object):
>>  symlink_collisions = []
>>  destroot = self.settings['ROOT']
>>  totfiles = len(file_list) + len(symlink_list)
>> +tnow = time.time()
>> +tinterv = 2  # seconds
>> +ninterv = tnow + tinterv
>>  showMessage(_(" %s checking %d files for package 
>> collisions\n") % \
>>  (colorize("GOOD", "*"), totfiles))
>>  for i, (f, f_type) in enumerate(chain(
>>  ((f, "reg") for f in file_list),
>>  ((f, "sym") for f in symlink_list))):
>> -if i % 1000 == 0 and i != 0:
>> -showMessage(_("%d files remaining 
>> ...\n") % (totfiles - i))
>> +if time.time() > ninterv:
>> +showMessage(_("%3d%% done, %d files 
>> remaining ...\n") %
>> +(i * 100 / totfiles, 
>> totfiles - i))
>> +ninterv = time.time() + tinterv
>>  
>>  dest_path = normalize_path(
>>  os.path.join(destroot, 
>> f.lstrip(os.path.sep)))
>> @@ -3570,6 +3575,8 @@ class dblink(object):
>>  break
>>  if stopmerge:
>>  collisions.append(f)
>> +if tnow + tinterv < ninterv:
>> +showMessage(_("100% done\n"))
>>  return collisions, dirs_ro, symlink_collisions, 
>> plib_collisions
>>  
>>  def _lstat_inode_map(self, path_iter):
>>
> Please replace time.time() with portage.util.monotonic.monotonic().
It's a bit of a nit-pick, granted, but can we ensure that the count-down's
remain padded/justified such that the numbers line up for easy at-a-glance
inspection ? The optimal standard looks somewhat like the pre-merge Sizes:

 * Final size of build directory: 2696 KiB (2.6 MiB)
 * Final size of installed tree:  5372 KiB (5.2 MiB)

Otherwise, I think this will be quite helpful. Thanks.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Error logs

[M. J. Everitt]

On 30/12/18 19:58, Pratyush Das wrote:
> Hi,
>
> In reference to this forum post
> - https://forums.gentoo.org/viewtopic-t-1091144-highlight-.html , I was
> told that several users found wrong information in the portage error logs
> pointing them to the wrong directory for distfiles to be downloaded and
> placed in for the emerge to work when the tree variable was displayed to
> the user, for the user to place the downloaded files in. 
> Instead of not using the tree variable to point to the distfiles location
> (and having a less descriptive error message), was using some other form
> of keeping a track of where the files are stored discussed? Or figuring
> out why the variable pointed to the wrong directory? 
>
> I could take a look at it if that is a feature you would be open to having. 
>
> Regards,
>
> -- 
> Pratyush Das(Reik)
Can you highlight some more examples (good & bad) as even the forum post
doesn't really go into much detail about what's present and missing? I can
see that dev-libs/cudnn is poor - usually LICENSE-restricted files give
some sensible directions, iirc. Not sure quite why this package is an
exception if it's problematic in being mirrored ...
Regards,

Michael/veremitz.


signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 0/4] Rename PORT_LOGDIR{,_CLEAN} variables to PORTAGE_LOGDIR{,_CLEAN} Bug 668538

[M. J. Everitt]

On 17/12/18 15:51, Michał Górny wrote:
> On Mon, 2018-12-17 at 15:44 +0000, M. J. Everitt wrote:
>> On 17/12/18 12:54, Michał Górny wrote:
>>>> Not only this, but as noted, unless you know the man pages for portage and
>>>> make.conf in order to recite them in your sleep, they are confusing for
>>>> users, as they do not necessarily follow an obvious pattern, and it wasn't
>>>> until I was attempting to debug something that I noticed that despite
>>>> believing I had the correct settings in my make.conf (set over a period of
>>>> YEARS) they were in fact completely useless, and it wasn't until I had to
>>>> spend time with somebody debugging WTF was happening, that this particular
>>>> issue even became apparent...
>>> I don't see how this is an argument for anything.  You have to read
>>> the manual in order to know that such variable exists and what it does. 
>>> Or, well, technically you don't since it's provided in make.conf.example
>>> already where you only need to uncomment it.
>>>
>>> Either way, the variable name is trivial.  Even if you don't follow
>>> the usual pattern of uncommenting it from make.conf.example or copying
>>> from the manual, remembering it for the time needed to retype shoudln't
>>> be a problem.
>>>
>>> So, is this a solution to a real problem?  Or is it merely a half-
>>> thought-out partial change that's going to require people to update
>>> their configuration for no long-term benefit?  And then they will have
>>> to update it again when someone decides to take another variable for
>>> a spin.
>>>
>> In the case you hadn't noticed, clearly you haven't .. the change is
>> backwards compatible.. that has already been thought out.
>>
>> But you haven't actually looked at the patch have you, Michal ?
>>
> I did look at it.  However, that doesn't change what I said.  Being
> 'backwards compatible' does not change the fact that the old variable
> becomes deprecated now.  Ergo, users are expected to eventually switch
> to the new one.
>
> Even if you don't care beyond changing this now and forgetting about it
> afterwards, someone eventually will have to clean up the old variable
> and actively force people to update.
>
Correct, but surely this doesn't apply in any other areas of Gentoo, eg.
perhaps Ebuilds? EAPIs? PMS? QA?



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 0/4] Rename PORT_LOGDIR{,_CLEAN} variables to PORTAGE_LOGDIR{,_CLEAN} Bug 668538

[M. J. Everitt]

On 17/12/18 15:44, M. J. Everitt wrote:
> On 17/12/18 12:54, Michał Górny wrote:
>>> Not only this, but as noted, unless you know the man pages for portage and
>>> make.conf in order to recite them in your sleep, they are confusing for
>>> users, as they do not necessarily follow an obvious pattern, and it wasn't
>>> until I was attempting to debug something that I noticed that despite
>>> believing I had the correct settings in my make.conf (set over a period of
>>> YEARS) they were in fact completely useless, and it wasn't until I had to
>>> spend time with somebody debugging WTF was happening, that this particular
>>> issue even became apparent...
>> I don't see how this is an argument for anything.  You have to read
>> the manual in order to know that such variable exists and what it does. 
>> Or, well, technically you don't since it's provided in make.conf.example
>> already where you only need to uncomment it.
>>
>> Either way, the variable name is trivial.  Even if you don't follow
>> the usual pattern of uncommenting it from make.conf.example or copying
>> from the manual, remembering it for the time needed to retype shoudln't
>> be a problem.
>>
>> So, is this a solution to a real problem?  Or is it merely a half-
>> thought-out partial change that's going to require people to update
>> their configuration for no long-term benefit?  And then they will have
>> to update it again when someone decides to take another variable for
>> a spin.
>>
> In the case you hadn't noticed, clearly you haven't .. the change is
> backwards compatible.. that has already been thought out.
>
> But you haven't actually looked at the patch have you, Michal ?
>
Whilst I'm here .. you won't also have noticed I've updated the
documentation too ...



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 0/4] Rename PORT_LOGDIR{,_CLEAN} variables to PORTAGE_LOGDIR{,_CLEAN} Bug 668538

[M. J. Everitt]

On 17/12/18 12:54, Michał Górny wrote:
>> Not only this, but as noted, unless you know the man pages for portage and
>> make.conf in order to recite them in your sleep, they are confusing for
>> users, as they do not necessarily follow an obvious pattern, and it wasn't
>> until I was attempting to debug something that I noticed that despite
>> believing I had the correct settings in my make.conf (set over a period of
>> YEARS) they were in fact completely useless, and it wasn't until I had to
>> spend time with somebody debugging WTF was happening, that this particular
>> issue even became apparent...
> I don't see how this is an argument for anything.  You have to read
> the manual in order to know that such variable exists and what it does. 
> Or, well, technically you don't since it's provided in make.conf.example
> already where you only need to uncomment it.
>
> Either way, the variable name is trivial.  Even if you don't follow
> the usual pattern of uncommenting it from make.conf.example or copying
> from the manual, remembering it for the time needed to retype shoudln't
> be a problem.
>
> So, is this a solution to a real problem?  Or is it merely a half-
> thought-out partial change that's going to require people to update
> their configuration for no long-term benefit?  And then they will have
> to update it again when someone decides to take another variable for
> a spin.
>
In the case you hadn't noticed, clearly you haven't .. the change is
backwards compatible.. that has already been thought out.

But you haven't actually looked at the patch have you, Michal ?



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 0/4] Rename PORT_LOGDIR{,_CLEAN} variables to PORTAGE_LOGDIR{,_CLEAN} Bug 668538

[M. J. Everitt]

On 15/12/18 08:55, Michał Górny wrote:
> On Sat, 2018-12-15 at 02:25 +0000, M. J. Everitt wrote:
>> This patchset aims to fix potential ambiguity and confusion between older 
>> PORT_LOG* variables,
>> and more recent PORTAGE_* variables - often leading to mysteriously lacking 
>> logging due to
>> incorrect variable names. Documented in Bug 668538; with thanks to Zac for 
>> diagnosis, and
>> solution assistance!
>>
> Does 'often' actually affect more than one person?  Do you have any
> evidence to support this?
>
> Given that a lot of Portage variables don't have any prefix or sane
> names, I dare say this one doesn't especially stand out.
>
Just a thought, but how about you apply your skill and wisdom to reviewing
the patches, instead of wasting it on writing snide replies?
Quite radical I know, but whadda ya think?!

As it happens, I was going for consistency here, as that often reflects
code quality, and you being a keen QA member, I'da thought perhaps you
might have spotted this!

Not only this, but as noted, unless you know the man pages for portage and
make.conf in order to recite them in your sleep, they are confusing for
users, as they do not necessarily follow an obvious pattern, and it wasn't
until I was attempting to debug something that I noticed that despite
believing I had the correct settings in my make.conf (set over a period of
YEARS) they were in fact completely useless, and it wasn't until I had to
spend time with somebody debugging WTF was happening, that this particular
issue even became apparent...



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH 4/4] Add note to NEWS/ChangeLog for PORT_LOGDIR* changes

[M. J. Everitt]

Suggested-by: Michael Everitt 
Bug: https://bugs.gentoo.org/668538
---
 NEWS | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/NEWS b/NEWS
index b430f273f..e65205c42 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,11 @@
 News (mainly features/major bug fixes)
 
+portage-next
+--
+* Renamed PORT_LOGDIR and PORT_LOGDIR_CLEAN variables to
+  PORTAGE_LOGDIR and PORTAGE_LOGDIR_CLEAN to match other existing
+  similar variables
+
 portage-2.3.42
 
 * Sync with rsync quarantines data prior to verification, using the
-- 
2.13.6




signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH 1/4] Renamed PORT_LOGDIR[_CLEAN] to PORTAGE_LOGDIR[_CLEAN]

[M. J. Everitt]

To match similar existing PORTAGE_* variables

Suggested-by: Zac Medico 
Acked-by: Michael Everitt 
Bug: https://bugs.gentoo.org/668538
---
 NEWS   |  2 +-
 cnf/make.conf.example  | 12 +-
 cnf/make.globals   |  2 +-
 lib/_emerge/post_emerge.py |  2 +-
 lib/portage/dbapi/vartree.py   |  2 +-
 lib/portage/elog/mod_echo.py   |  2 +-
 lib/portage/elog/mod_save.py   |  4 ++--
 lib/portage/elog/mod_save_summary.py   |  4 ++--
 lib/portage/emaint/modules/logs/__init__.py|  2 +-
 lib/portage/emaint/modules/logs/logs.py| 22 +-
 .../package/ebuild/_config/special_env_vars.py |  2 +-
 lib/portage/package/ebuild/prepare_build_dirs.py   | 26 +++---
 lib/portage/tests/emerge/test_simple.py|  2 +-
 man/ebuild.5   |  2 +-
 man/emaint.1   |  6 ++---
 man/emerge.1   |  2 +-
 man/make.conf.5| 18 +++
 17 files changed, 56 insertions(+), 56 deletions(-)

diff --git a/NEWS b/NEWS
index f77c310e3..b430f273f 100644
--- a/NEWS
+++ b/NEWS
@@ -389,7 +389,7 @@ portage-2.1 (ongoing via pre releases)
 * Make --emptytree only apply to ${ROOT} rather than always including /.
 * Allow packages to be upgraded that are only depended on via a
   "|| ( =cat/pkg-1* =cat/pkg-2* )" construct.
-* Ebuild output is no longer cut off early when using PORT_LOGDIR.
+* Ebuild output is no longer cut off early when using PORTAGE_LOGDIR.
 * Distfiles indirection- $DISTFILES access goes through a tmp dir to fail
   access to files not listed in SRC_URI.
 * Emerge now uses --resume to restart itself after portage upgrade.
diff --git a/cnf/make.conf.example b/cnf/make.conf.example
index 36fc9a8e4..ffebd24d4 100644
--- a/cnf/make.conf.example
+++ b/cnf/make.conf.example
@@ -130,13 +130,13 @@
 # storage. See the PORTDIR documentation for more information.
 #PKGDIR=/usr/portage/packages
 #
-# PORT_LOGDIR is the location where portage will store all the logs it
+# PORTAGE_LOGDIR is the location where portage will store all the logs it
 # creates from each individual merge. They are stored as
 # ${CATEGORY}:${PF}:MMDD-HHMMSS.log in the directory specified.
 # If the directory does not exist, it will be created automatically and
 # group permissions will be applied to it.  If the directory already
 # exists, portage will not modify its permissions.
-#PORT_LOGDIR=""
+#PORTAGE_LOGDIR=""
 #
 # PORTDIR_OVERLAY is a directory where local ebuilds may be stored without
 # concern that they will be deleted by rsync updates. Default is not
@@ -306,16 +306,16 @@
 # PORTAGE_ELOG_SYSTEM: selects the module(s) to process the log messages. 
Modules
 #  included in portage are (empty means logging is 
disabled):
 #  echo (display messages again when emerge exits)
-#  save (saves one log per package in 
$PORT_LOGDIR/elog,
-#/var/log/portage/elog if $PORT_LOGDIR is 
unset)
+#  save (saves one log per package in 
$PORTAGE_LOGDIR/elog,
+#/var/log/portage/elog if $PORTAGE_LOGDIR is 
unset)
 #  custom (passes all messages to 
$PORTAGE_ELOG_COMMAND)
 #  syslog (sends all messages to syslog)
 #  mail (send all messages to the mailserver defined
 #in $PORTAGE_ELOG_MAILURI)
 #  save_summary (like "save" but merges all messages
-#in $PORT_LOGDIR/elog/summary.log,
+#in $PORTAGE_LOGDIR/elog/summary.log,
 #/var/log/portage/elog/summary.log if
-#$PORT_LOGDIR is unset)
+#$PORTAGE_LOGDIR is unset)
 #  mail_summary (like "mail" but sends all messages in
 #a single mail when emerge exits)
 #  To use elog you should enable at least one module
diff --git a/cnf/make.globals b/cnf/make.globals
index 1bcc7ce64..5a3015ae2 100644
--- a/cnf/make.globals
+++ b/cnf/make.globals
@@ -101,7 +101,7 @@ PORTAGE_RSYNC_OPTS="--recursive --links --safe-links 
--perms --times --omit-dir-
 PORTAGE_SYNC_STALE="30"
 
 # Executed before emerge exit if FEATURES=clean-logs is enabled.
-PORT_LOGDIR_CLEAN="find \"\${PORT_LOGDIR}\" -type f ! -name \"summary.log*\" 
-mtime +7 -delete"
+PORTAGE_LOGDIR_CLEAN="find \"\${PORTAGE_LOGDIR}\" -type f ! -name 
\"summary.log*\" -mtime +7 -delete"
 
 # Minimal 

[gentoo-portage-dev] [PATCH 2/4] Provide compatibility support for old PORT_LOGDIR* variable names

[M. J. Everitt]

Suggested-by: Zac Medico 
Acked-by: Michael Everitt 
Bug: https://bugs.gentoo.org/668538
---
 lib/portage/package/ebuild/config.py | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/lib/portage/package/ebuild/config.py 
b/lib/portage/package/ebuild/config.py
index a68df5807..5c1404389 100644
--- a/lib/portage/package/ebuild/config.py
+++ b/lib/portage/package/ebuild/config.py
@@ -155,6 +155,9 @@ class config(object):
_constant_keys = frozenset(['PORTAGE_BIN_PATH', 'PORTAGE_GID',
'PORTAGE_PYM_PATH', 'PORTAGE_PYTHONPATH'])
 
+   _deprecated_keys = {'PORTAGE_LOGDIR': 'PORT_LOGDIR',
+   'PORTAGE_LOGDIR_CLEAN': 'PORT_LOGDIR_CLEAN'}
+
_setcpv_aux_keys = ('BDEPEND', 'DEFINED_PHASES', 'DEPEND', 'EAPI', 
'HDEPEND',
'INHERITED', 'IUSE', 'REQUIRED_USE', 'KEYWORDS', 'LICENSE', 
'PDEPEND',
'PROPERTIES', 'RDEPEND', 'SLOT',
@@ -2653,6 +2656,14 @@ class config(object):
except KeyError:
pass
 
+   deprecated_key = self._deprecated_keys.get(mykey)
+   if deprecated_key is not None:
+   value = self._getitem(deprecated_key)
+   warnings.warn(_("Key %s has been renamed to %s. Please 
",
+   "update your configuration") % (deprecated_key, 
mykey),
+   UserWarning)
+   return value
+
raise KeyError(mykey)
 
def get(self, k, x=None):
-- 
2.13.6





signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH 0/4] Rename PORT_LOGDIR{,_CLEAN} variables to PORTAGE_LOGDIR{,_CLEAN} Bug 668538

[M. J. Everitt]

This patchset aims to fix potential ambiguity and confusion between older 
PORT_LOG* variables,
and more recent PORTAGE_* variables - often leading to mysteriously lacking 
logging due to
incorrect variable names. Documented in Bug 668538; with thanks to Zac for 
diagnosis, and
solution assistance!

M. J. Everitt (4):
  Renamed PORT_LOGDIR[_CLEAN] to PORTAGE_LOGDIR[_CLEAN]
  Provide compatibility support for old PORT_LOGDIR* variable names
  Update documentation for PORT_LOGDIR* changes
  Add note to NEWS/ChangeLog for PORT_LOGDIR* changes

 NEWS   |  8 +++-
 cnf/make.conf.example  | 12 +++---
 cnf/make.globals   |  2 +-
 lib/_emerge/post_emerge.py |  2 +-
 lib/portage/dbapi/vartree.py   |  2 +-
 lib/portage/elog/mod_echo.py   |  2 +-
 lib/portage/elog/mod_save.py   |  4 +-
 lib/portage/elog/mod_save_summary.py   |  4 +-
 lib/portage/emaint/modules/logs/__init__.py|  2 +-
 lib/portage/emaint/modules/logs/logs.py| 22 +--
 .../package/ebuild/_config/special_env_vars.py |  2 +-
 lib/portage/package/ebuild/config.py   | 11 ++
 lib/portage/package/ebuild/prepare_build_dirs.py   | 26 ++--
 lib/portage/tests/emerge/test_simple.py|  2 +-
 man/ebuild.5   |  2 +-
 man/emaint.1   |  6 +--
 man/emerge.1   |  2 +-
 man/make.conf.5| 46 --
 18 files changed, 90 insertions(+), 67 deletions(-)

-- 
2.13.6




signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v2 3/3] Introduce a tiny init replacement for inside pid namespace

[M. J. Everitt]

On 18/11/18 09:18, Michał Górny wrote:
> On Sun, 2018-11-18 at 09:04 +0000, M. J. Everitt wrote:
>> On 18/11/18 08:53, Michał Górny wrote:
>>> Signed-off-by: Michał Górny 
>>> ---
>>>  bin/pid-ns-init| 25 +
>>>  lib/portage/process.py | 11 ++-
>>>  2 files changed, 31 insertions(+), 5 deletions(-)
>>>  create mode 100644 bin/pid-ns-init
>>>
>>> diff --git a/bin/pid-ns-init b/bin/pid-ns-init
>>> new file mode 100644
>>> index 0..90660571a
>>> --- /dev/null
>>> +++ b/bin/pid-ns-init
>>> @@ -0,0 +1,25 @@
>>> +#!/usr/bin/env python
>>> +# Copyright 2018 Gentoo Authors
>>> +# Distributed under the terms of the GNU General Public License v2
>>> +
>>> +import os
>>> +import sys
>>> +
>>> +
>>> +def main(argv):
>>> +   if len(argv) < 2:
>>> +   return 'Usage: {} '.format(argv[0])
>>> +   main_child_pid = int(argv[1])
>>> +
>>> +   # wait for child processes
>>> +   while True:
>>> +   pid, status = os.wait()
>>> +   if pid == main_child_pid:
>>> +   return os.WEXITSTATUS(status)
>>> +
>>> +   # this should never be reached
>>> +   return 127
>>> +
>>> +
>>> +if __name__ == '__main__':
>>> +   sys.exit(main(sys.argv))
>>> diff --git a/lib/portage/process.py b/lib/portage/process.py
>>> index dee126c3c..75ec299f0 100644
>>> --- a/lib/portage/process.py
>>> +++ b/lib/portage/process.py
>>> @@ -544,13 +544,14 @@ def _exec(binary, mycommand, opt_name, fd_pipes, env, 
>>> gid, groups, uid, umask,
>>> else:
>>> if unshare_pid:
>>> # pid namespace 
>>> requires us to become init
>>> -   # TODO: do init-ty stuff
>>> -   # therefore, fork() ASAP
>>> fork_ret = os.fork()
>>> if fork_ret != 0:
>>> -   pid, status = 
>>> os.waitpid(fork_ret, 0)
>>> -   assert pid == 
>>> fork_ret
>>> -   os._exit(status)
>>> +   
>>> os.execv(portage._python_interpreter, [
>>> +   
>>> portage._python_interpreter,
>>> +   
>>> os.path.join(portage._bin_path,
>>> +   
>>> 'pid-ns-init'),
>>> +   '%s' % 
>>> fork_ret,
>>> +   ])
>>> if unshare_mount:
>>> # mark the whole 
>>> filesystem as slave to avoid
>>> # mounts escaping the 
>>> namespace
>> Why in python?! Surely a small C app would be significantly more efficient ..
>>
> Surely adding a new build system for C apps would be entirely justified
> by the necessity of premature optimization of program that spends ~100%
> of its time in wait().
>
Forgive my ignorance then .. what purpose does a process have which simply
wait()s ?! .. no, on second thoughts, don't answer that question, I'm
unlikely to like the answer anyway ...



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v2 3/3] Introduce a tiny init replacement for inside pid namespace

[M. J. Everitt]

On 18/11/18 08:53, Michał Górny wrote:
> Signed-off-by: Michał Górny 
> ---
>  bin/pid-ns-init| 25 +
>  lib/portage/process.py | 11 ++-
>  2 files changed, 31 insertions(+), 5 deletions(-)
>  create mode 100644 bin/pid-ns-init
>
> diff --git a/bin/pid-ns-init b/bin/pid-ns-init
> new file mode 100644
> index 0..90660571a
> --- /dev/null
> +++ b/bin/pid-ns-init
> @@ -0,0 +1,25 @@
> +#!/usr/bin/env python
> +# Copyright 2018 Gentoo Authors
> +# Distributed under the terms of the GNU General Public License v2
> +
> +import os
> +import sys
> +
> +
> +def main(argv):
> + if len(argv) < 2:
> + return 'Usage: {} '.format(argv[0])
> + main_child_pid = int(argv[1])
> +
> + # wait for child processes
> + while True:
> + pid, status = os.wait()
> + if pid == main_child_pid:
> + return os.WEXITSTATUS(status)
> +
> + # this should never be reached
> + return 127
> +
> +
> +if __name__ == '__main__':
> + sys.exit(main(sys.argv))
> diff --git a/lib/portage/process.py b/lib/portage/process.py
> index dee126c3c..75ec299f0 100644
> --- a/lib/portage/process.py
> +++ b/lib/portage/process.py
> @@ -544,13 +544,14 @@ def _exec(binary, mycommand, opt_name, fd_pipes, env, 
> gid, groups, uid, umask,
>   else:
>   if unshare_pid:
>   # pid namespace 
> requires us to become init
> - # TODO: do init-ty stuff
> - # therefore, fork() ASAP
>   fork_ret = os.fork()
>   if fork_ret != 0:
> - pid, status = 
> os.waitpid(fork_ret, 0)
> - assert pid == 
> fork_ret
> - os._exit(status)
> + 
> os.execv(portage._python_interpreter, [
> + 
> portage._python_interpreter,
> + 
> os.path.join(portage._bin_path,
> + 
> 'pid-ns-init'),
> + '%s' % 
> fork_ret,
> + ])
>   if unshare_mount:
>   # mark the whole 
> filesystem as slave to avoid
>   # mounts escaping the 
> namespace
Why in python?! Surely a small C app would be significantly more efficient ..



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Re: [RFC] Improving Gentoo package format

[M. J. Everitt]

On 11/11/18 19:20, Rich Freeman wrote:
> On Sun, Nov 11, 2018 at 1:02 PM M. J. Everitt  wrote:
>> If you can really present a decent argument for replicating the
>> functionality of other distros like Debian, Arch, Ubuntu etc then let's
>> here it. For now, the strength of Gentoo is being able to fully customise a
>> system to your own requirements, not being trapped by some distro
>> maintainer's arbitrary choices. Play to your USP's and strengths rather
>> than chasing rainbows ..
>>
> Why do we support binary packages at all?  Simple: compiling packages
> is expensive, and if you happen to already have them compiled, fully
> customized to your own requirements, then there is no point in
> recompiling them.  You're just spending a ton of resources to build
> the exact same files you already have.
>
> The only change I'm suggesting is that portage could take all the
> configuration you're already supplying, and then optionally go see if
> somebody you trust has already built the package that meets your
> requirements.  If so, then it would be downloaded and installed,
> otherwise it would just compile from source.
>
> You get the exact same files installed on your system either way.
>
Ok so I get the principle, but who's gonna provide the tools to make this
feasible, and perhaps more interestingly, who's going to curate, provide,
host and maintain the binpkg repos you propose? We barely have enough
developers to maintain a working source package repository, let alone
adding new distro "features" .. unless perhaps you have a few hours every
week to spare?

I see no sense in reinventing the wheel here, besides #thegentooway



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Re: [RFC] Improving Gentoo package format

[M. J. Everitt]

On 11/11/18 18:41, Rich Freeman wrote:
> On Sun, Nov 11, 2018 at 12:31 PM M. J. Everitt  wrote:
>> Binpkgs are also a popular component of a few downstream distro's based on
>> Gentoo (thinking pentoo right now as an easy example).
>>
>> So we don't want to break existing users of this format without considering
>> the ramifications for these scenarios, as you'll have some very grumpy 
>> devs...
>>
> I'd argue that they'd be more important for Gentoo if they were more
> useful.  IMO the main limitation with them is the inability to
> auto-download them from a repository, detecting the binpkg USE flags
> BEFORE downloading.  This is why I suggested hashing the USE flags or
> similar and sticking that in the filename.
>
> Obviously you can't host a repository with all the USE combinations.
> However, you could have a reference repo and the package manager could
> check it before doing a build.  If you get a hit then you can install
> the binpkg.  If you don't then you can do a source build.
>
> Portage already checks the USE flags inside the binpkg before merging
> it and by default doesn't use a non-matching binpkg.  The problem with
> the current approach is:
> 1.  You have to download the package to check this (could be a big file).
> 2.  You can't host multiple versions of a binpkg with different USE
> flags since the filenames collide.
>
> I suggested a content hash because you can use it for an arbitrary
> amount of metadata, vs having to cram arch/USE/multilib and I'm sure
> something I'm missing into a filename.  Make the hash as short as is
> economical - it isn't like we have THAT many permutations, the PM can
> still check the internal metadata, and this isn't a security feature.
>
If you can really present a decent argument for replicating the
functionality of other distros like Debian, Arch, Ubuntu etc then let's
here it. For now, the strength of Gentoo is being able to fully customise a
system to your own requirements, not being trapped by some distro
maintainer's arbitrary choices. Play to your USP's and strengths rather
than chasing rainbows ..



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Re: [RFC] Improving Gentoo package format

[M. J. Everitt]

On 11/11/18 18:15, Duncan wrote [excerpted]:
> Is there any interest at all in binpkgs, perhaps when improved, from the 
> other PMs?  Or are they effectively dead now or not interested in binpkgs 
> even if the format were to be improved, or simply too hard to work with?  
> Because "it'd be nice" (aka MAY level) to have this formally standardized 
> to PMS... if there's any interest from the other PMs.
>
Binpkgs are an important part of catalyst/releng stage-building runs, as it
allows portage to 'cache' a lot of the packages needed/used.

Binpkgs are also a popular component of a few downstream distro's based on
Gentoo (thinking pentoo right now as an easy example).

So we don't want to break existing users of this format without considering
the ramifications for these scenarios, as you'll have some very grumpy devs...



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] XARGS: use gxargs for USERLAND=BSD (bug 663256)

[M. J. Everitt]

On 10/08/18 00:17, M. J. Everitt wrote:
> On 10/08/18 00:04, Zac Medico wrote:
>> For USERLAND=BSD, set XARGS="gxargs -r" if gxargs is available,
>> so the code from bug 630292 works for USERLAND=BSD.
>>
>> Fixes: 50283f1abb77 (install-qa-check.d/60pngfix: parallel support (bug 
>> 630292))
>> Reported-by: Michał Górny 
>> Bug: https://bugs.gentoo.org/663256
>> ---
>>  bin/isolated-functions.sh | 6 +-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
>> index 28ca94532..cac42a4c5 100644
>> --- a/bin/isolated-functions.sh
>> +++ b/bin/isolated-functions.sh
>> @@ -448,7 +448,11 @@ fi
>>  if [[ -z ${XARGS} ]] ; then
>>  case ${USERLAND} in
>>  BSD)
>> -export XARGS="xargs"
>> +if type -P gxargs > /dev/null; then
>> +export XARGS="gxargs -r"
>> +else
>> +export XARGS="xargs"
>> +fi
>>  ;;
>>  *)
>>  export XARGS="xargs -r"
> Isn't the else clause there redundant?
>
Oops, sorry missed the minus before the first export .. my bad .. 



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] XARGS: use gxargs for USERLAND=BSD (bug 663256)

[M. J. Everitt]

On 10/08/18 00:04, Zac Medico wrote:
> For USERLAND=BSD, set XARGS="gxargs -r" if gxargs is available,
> so the code from bug 630292 works for USERLAND=BSD.
>
> Fixes: 50283f1abb77 (install-qa-check.d/60pngfix: parallel support (bug 
> 630292))
> Reported-by: Michał Górny 
> Bug: https://bugs.gentoo.org/663256
> ---
>  bin/isolated-functions.sh | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/bin/isolated-functions.sh b/bin/isolated-functions.sh
> index 28ca94532..cac42a4c5 100644
> --- a/bin/isolated-functions.sh
> +++ b/bin/isolated-functions.sh
> @@ -448,7 +448,11 @@ fi
>  if [[ -z ${XARGS} ]] ; then
>   case ${USERLAND} in
>   BSD)
> - export XARGS="xargs"
> + if type -P gxargs > /dev/null; then
> + export XARGS="gxargs -r"
> + else
> + export XARGS="xargs"
> + fi
>   ;;
>   *)
>   export XARGS="xargs -r"
Isn't the else clause there redundant?



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v3 0/2] Two insecure ownership and group-writability QA checks.​

[M. J. Everitt]

On 07/08/18 17:46, Michael Orlitzky wrote:
> Changes in v3:
>
>   * Undo the setguid exception from v2, and add a comment explaining why.
>   * Add line breaks for readability in two comments.
>   * Try to put back the leading "/" in the output list.
>   * Remove a superfluous comment mentioning the "prefix."
>
> Michael Orlitzky (2):
>   bin/install-qa-check.d: add new 90bad-bin-owner QA check.
>   bin/install-qa-check.d: add new 90bad-bin-group-write QA check.
>
>  bin/install-qa-check.d/90bad-bin-group-write | 55 
> 
>  bin/install-qa-check.d/90bad-bin-owner   | 48 
>  2 files changed, 103 insertions(+)
>  create mode 100644 bin/install-qa-check.d/90bad-bin-group-write
>  create mode 100644 bin/install-qa-check.d/90bad-bin-owner
>
May I just briefly complement you on your patch mail format ..

It's much easier to see, when you make updates to a patchset, that you
have added a summary in your 0/x mail for what changed from the previous
iteration.

I don't think git has an easy way to both describe the patch diffs from
the original *and* the changes from last iteration to current - that
really would be the cherry on the icing on the cake! (something perhaps
the avid perl users may be able to graft together!).

Keep up the good work :)
Best regards,

Michael.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 2/2] Update /usr/portage references (bug 378603)

[M. J. Everitt]

On 06/08/18 19:35, Zac Medico wrote:
> On 08/06/2018 12:30 AM, Brian Dolbec wrote:
>> All these changes as well as the catlayst changes need to be
>> co-ordinated so that snapshots and portage and stages don't precede the
>> docs changes.
> I suppose all of the docs changes can be made in advance, with mention
> of both old a new locations. These commands can be used to check the
> local configuration:
>
> portageq get_repo_path / gentoo
> portageq distdir
> portageq pkgdir
Sounds like a plan .. do we need a tracker bug?! :P

Let's do this! :D

MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] depgraph.autounmask_breakage_detected: ignore REQUIRED_USE violations (bug 654782)

[M. J. Everitt]

On 04/05/18 06:22, Zac Medico wrote:
> When autounmask USE changes violate REQUIRED_USE, rather than
> recalculate with autounmask disabled, display the autounmask USE
> changes along with the REQUIRED_USE violation. Adjust unit tests
> to allow for the autounmask USE changes that were previously
> discarded. For the case reported in bug 654782, the output will
> appear as follows:
>
> The following keyword changes are necessary to proceed:
>  (see "package.accept_keywords" in the portage(5) man page for more details)
> # required by =retext-7.0.1-r1 (argument)
> =app-editors/retext-7.0.1-r1 ~amd64
>
> The following USE changes are necessary to proceed:
>  (see "package.use" in the portage(5) man page for more details)
> # required by app-editors/retext-7.0.1-r1::gentoo
> # required by =retext-7.0.1-r1 (argument)
>> =dev-python/PyQt5-5.9.2 printsupport webengine network gui widgets
> !!! The ebuild selected to satisfy 
> "dev-python/PyQt5[gui,network,printsupport,webengine,widgets,python_targets_python3_4(-)?,python_targets_python3_5(-)?,python_targets_python3_6(-)?,-python_single_target_python3_4(-),-python_single_target_python3_5(-),-python_single_target_python3_6(-)]"
>  has unmet requirements.
> - dev-python/PyQt5-5.9.2::gentoo USE="-bluetooth -dbus -debug -declarative 
> -designer -examples -gles2 -gui -help -location -multimedia -network -opengl 
> -positioning -printsupport -sensors -serialport -sql -svg -testlib 
> -webchannel -webengine -webkit -websockets -widgets -x11extras -xmlpatterns" 
> PYTHON_TARGETS="python2_7 python3_4 python3_6 -python3_5"
>
>   The following REQUIRED_USE flag constraints are unsatisfied:
> webengine? ( widgets? ( webchannel ) )
>
>   The above constraints are a subset of the following complete expression:
> any-of ( python_targets_python2_7 python_targets_python3_4 
> python_targets_python3_5 python_targets_python3_6 ) bluetooth? ( gui ) 
> declarative? ( gui network ) designer? ( widgets ) help? ( gui widgets ) 
> location? ( positioning ) multimedia? ( gui network ) opengl? ( gui widgets ) 
> positioning? ( gui ) printsupport? ( gui widgets ) sensors? ( gui ) 
> serialport? ( gui ) sql? ( widgets ) svg? ( gui widgets ) testlib? ( widgets 
> ) webchannel? ( network ) webengine? ( network widgets? ( printsupport 
> webchannel ) ) webkit? ( gui network printsupport widgets ) websockets? ( 
> network ) widgets? ( gui ) xmlpatterns? ( network )
>
> (dependency required by "app-editors/retext-7.0.1-r1::gentoo" [ebuild])
> (dependency required by "=retext-7.0.1-r1" [argument])
>
> Bug: https://bugs.gentoo.org/654782
> ---
>  pym/_emerge/depgraph.py|  8 ++--
>  pym/portage/tests/resolver/test_autounmask.py  | 48 
> --
>  pym/portage/tests/resolver/test_slot_collisions.py | 20 +
>  3 files changed, 61 insertions(+), 15 deletions(-)
>
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index fbd16ad98..429d8871c 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -3009,6 +3009,10 @@ class depgraph(object):
>   {"myparent" : dep.parent, 
> "show_req_use" : pkg}))
>   self._dynamic_config._required_use_unsatisfied 
> = True
>   self._dynamic_config._skip_restart = True
> + # Add pkg to digraph in order to enable 
> autounmask messages
> + # for this package, which is useful when 
> autounmask USE
> + # changes have violated REQUIRED_USE.
> + self._dynamic_config.digraph.add(pkg, 
> dep.parent, priority=priority)
>   return 0
>  
>   if not pkg.onlydeps:
> @@ -9428,10 +9432,6 @@ class depgraph(object):
>   return self._dynamic_config._need_config_reload
>  
>   def autounmask_breakage_detected(self):
> - # Check for REQUIRED_USE violations.
> - for changes in 
> self._dynamic_config._needed_use_config_changes.values():
> - if getattr(changes, 'required_use_satisfied', None) is 
> False:
> - return True
>   try:
>   for pargs, kwargs in 
> self._dynamic_config._unsatisfied_deps_for_display:
>   self._show_unsatisfied_dep(
> diff --git a/pym/portage/tests/resolver/test_autounmask.py 
> b/pym/portage/tests/resolver/test_autounmask.py
> index 9042349ea..809d42104 100644
> --- a/pym/portage/tests/resolver/test_autounmask.py
> +++ b/pym/portage/tests/resolver/test_autounmask.py
> @@ -251,15 +251,42 @@ class AutounmaskTestCase(TestCase):
>   use_changes={ "dev-util/R-1": { "bar": 
> True } }),
>  
>   #Test interaction with REQUIRED_USE.
> + # Some of these cases trigger USE change(s) 
> that violate
> +

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

[M. J. Everitt]

On 21/03/18 23:26, M. J. Everitt wrote:
> n 20/03/18 04:49, Manuel Rüger wrote:
>> Hi Zac,
>>
>> alternatively could --exclude be extended to support sets?
>> So users could --exclude @world or @profile.
>>
>> Cheers,
>> Manuel
>>
> The idea is sound enough, but I fear the syntax would be too confusing.
>
> Reading a potential command-line as "emerge   world-file unless --no-replace specified>  puts my head into
> a spin! I can see it might be clearer for an unmerge perhaps ..
>
> Unless I'm missing something fundamental ...
>
On a related note, it would be quite handy to enumerate an
"exclude-from" option like rsync/tar(?) to specify a file with the
--exclude options in. There could be some way to append @world to this
option, perhaps?

MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] emerge: add --ignore-world [ y | n ] option (bug 608564)

[M. J. Everitt]

n 20/03/18 04:49, Manuel Rüger wrote:
> Hi Zac,
>
> alternatively could --exclude be extended to support sets?
> So users could --exclude @world or @profile.
>
> Cheers,
> Manuel
>
The idea is sound enough, but I fear the syntax would be too confusing.

Reading a potential command-line as "emergeputs my head into
a spin! I can see it might be clearer for an unmerge perhaps ..

Unless I'm missing something fundamental ...



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] emerge --search: use slash to auto-detect category (bug 647940)

[M. J. Everitt]

On 18/02/18 21:27, Zac Medico wrote:
> Since search strings containing a slash do no work unless category
> match mode is enabled, use slash to auto-detect category match mode,
> so that users do not have to prefix the search string with the special
> @ symbol.
>
> Bug: https://bugs.gentoo.org/647940
> ---
>  pym/_emerge/search.py | 4 
>  1 file changed, 4 insertions(+)
>
> diff --git a/pym/_emerge/search.py b/pym/_emerge/search.py
> index 36897fb15..eb52b2ca3 100644
> --- a/pym/_emerge/search.py
> +++ b/pym/_emerge/search.py
> @@ -256,6 +256,10 @@ class search(object):
>   if self.searchkey.startswith('@'):
>   match_category = 1
>   self.searchkey = self.searchkey[1:]
> + # Auto-detect category match mode (@ symbol can be deprecated
> + # after this is available in a stable version of portage).
> + if '/' in self.searchkey:
> + match_category = 1
>   fuzzy = False
>   if regexsearch:
>   self.searchre=re.compile(self.searchkey,re.I)
+1 long overdue! :D



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v2 1/3] repoman: Add commit message verification

[M. J. Everitt]

On 18/02/18 16:28, Brian Dolbec wrote:
> On Sat, 17 Feb 2018 13:18:03 +
> "M. J. Everitt" <m.j.ever...@iee.org> wrote:
>
>> Might I suggest breaking out checks into a separate module? I think
>> that hard-coding it all is likely to become a pain as time goes on,
>> more checks get added, etc  ...
>>
> So, you would like the commit message checks to be  a plug-in system
> like the ebuild checks are now.
>
> Yes that is possible,  that would also allow overlays and sub-distros
> to customize it to their liking.  But you do realize it would be a
> module system with only the one plug-in.  It would add a slight amount
> of overhead and subsequently a tiny bit more time.  Although it is not
> much as I remember when I first developed the plug-in system for emaint.
>
> I will consider that for the stage3 work in progress, but for the
> master branch of repoman, the hard-coded version above would be fine.
>
That's probably a better implementation, but for the time being I was
thinking more of an 'actions_commit' module to break out the extra code
from the generic 'actions' module.
Regards,
Michael/veremitz.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v2 1/3] repoman: Add commit message verification

[M. J. Everitt]

On 17/02/18 12:56, Michał Górny wrote:
> Add a check for common mistakes in commit messages. For now, it is
> pretty rough and works only for -m/-F. It will be extended to work
> in the interactive mode in the future.
> ---
>  repoman/pym/repoman/actions.py  | 74 
> -
>  repoman/pym/repoman/tests/commit/__init__.py|  2 +
>  repoman/pym/repoman/tests/commit/__test__.py|  1 +
>  repoman/pym/repoman/tests/simple/test_simple.py |  8 +--
>  4 files changed, 80 insertions(+), 5 deletions(-)
>  create mode 100644 repoman/pym/repoman/tests/commit/__init__.py
>  create mode 100644 repoman/pym/repoman/tests/commit/__test__.py
>
> diff --git a/repoman/pym/repoman/actions.py b/repoman/pym/repoman/actions.py
> index b76a6e466..91603865c 100644
> --- a/repoman/pym/repoman/actions.py
> +++ b/repoman/pym/repoman/actions.py
> @@ -119,7 +119,16 @@ class Actions(object):
>   if commitmessage[:9].lower() in ("cat/pkg: ",):
>   commitmessage = self.msg_prefix() + 
> commitmessage[9:]
>  
> - if not commitmessage or not commitmessage.strip():
> + if commitmessage is not None and commitmessage.strip():
> + res, expl = self.verify_commit_message(commitmessage)
> + if not res:
> + print(bad("RepoMan does not like your commit 
> message:"))
> + print(expl)
> + if self.options.force:
> + print('(but proceeding due to --force)')
> + else:
> + sys.exit(1)
> + else:
>   commitmessage = self.get_new_commit_message(qa_output)
>  
>   commitmessage = commitmessage.rstrip()
> @@ -585,3 +594,66 @@ class Actions(object):
>   print("* no commit message?  aborting commit.")
>   sys.exit(1)
>   return commitmessage
> +
> + footer_re = re.compile(r'^[\w-]+:')
> +
> + @classmethod
> + def verify_commit_message(cls, msg):
> + """
> + Check whether the message roughly complies with GLEP66. Returns
> + (True, None) if it does, (False, ) if it does not.
> + """
> +
> + problems = []
> + paras = msg.strip().split('\n\n')
> + summary = paras.pop(0)
> +
> + if ':' not in summary:
> + problems.append('summary line must start with a logical 
> unit name, e.g. "cat/pkg:"')
> + if '\n' in summary.strip():
> + problems.append('commit message must start with a 
> *single* line of summary, followed by empty line')
> + # accept 69 overall or unit+50, in case of very long package 
> names
> + elif len(summary.strip()) > 69 and len(summary.split(':', 
> 1)[-1]) > 50:
> + problems.append('summary line is too long (max 69 
> characters)')
> +
> + multiple_footers = False
> + gentoo_bug_used = False
> + bug_closes_without_url = False
> + body_too_long = False
> +
> + found_footer = False
> + for p in paras:
> + lines = p.splitlines()
> + # if all lines look like footer material, we assume 
> it's footer
> + # else, we assume it's body text
> + if all(cls.footer_re.match(l) for l in lines if 
> l.strip()):
> + # multiple footer-like blocks?
> + if found_footer:
> + multiple_footers = True
> + found_footer = True
> + for l in lines:
> + if l.startswith('Gentoo-Bug'):
> + gentoo_bug_used = True
> + elif l.startswith('Bug:') or 
> l.startswith('Closes:'):
> + if 'http://' not in l and 
> 'https://' not in l:
> + bug_closes_without_url 
> = True
> + else:
> + for l in lines:
> + # we recommend wrapping at 72 but 
> accept up to 80;
> + # do not complain if we have single 
> word (usually
> + # it means very long URL)
> + if len(l.strip()) > 80 and 
> len(l.split()) > 1:
> + body_too_long = True
> +
> + if multiple_footers:
> + problems.append('multiple footer-style blocks found, 
> please combine them into one')
> + if gentoo_bug_used:
> + 

Re: [gentoo-portage-dev] [PATCH v2 6/9] rsync: Issue an explicit warning if Manifest timestamp is >24hr old

[M. J. Everitt]

On 02/02/18 20:42, Michał Górny wrote[excerpted]:
> index 27a2548c0..cb80f6d66 100644
> --- a/pym/portage/sync/modules/rsync/__init__.py
> +++ b/pym/portage/sync/modules/rsync/__init__.py
> @@ -109,6 +110,20 @@ class RsyncSync(NewBase):
>   writemsg_level("!!! sync-rsync-verify-jobs not 
> a positive integer: %s\n" % (self.verify_jobs,),
>   level=logging.WARNING, noiselevel=-1)
>   self.verify_jobs = None
> + # Support overriding max age.
> + self.max_age = self.repo.module_specific_options.get(
> + 'sync-rsync-verify-max-age', '')
> + if self.max_age:
> + try:
> + self.max_age = int(self.max_age)
> + if self.max_age < 0:
> + raise ValueError(self.max_age)
> + except ValueError:
> + writemsg_level("!!! sync-rsync-max-age not a 
> non-negative integer: %s\n" % (self.max_age,),
A beautiful double-negative .. but would read better as "not a positive
integer" or "is a negative integer" ..

MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] Rewrite doins in python (bug 624526)

[M. J. Everitt]

On 14/08/17 08:39, Zac Medico wrote:
> From: Hidehiko Abe 
>
> doins is written in bash. However, specifically in case that
> too many files are installed, it is very slow.
> This CL rewrites the script in python for performance.
>
> BUG=chromium:712659
> TEST=time (./setup_board --forace && \
>  ./build_package --withdev && \
>  ./build_image --noenable_rootfs_verification test)
> ===Before===
> real21m35.445s
> user93m40.588s
> sys 21m31.224s
>
> ===After===
> real17m30.106s
> user94m1.812s
> sys 20m13.468s
>
I know I'm gonna get chewed out on this one, but here goes anyway ...

Surely for a package like chromium, who's build time is already in the
'hours' range anyway, surely a couple of minutes gain for the install
phase is neither here nor there?! If there were some genuine filesystem
iop gains/etc for this change, I think I'd likely support it further ..

On this basis, what do the performance differences look like on an
'average' package .. and are there any regressions in this regard?!

I take issue with the copyright assignment, as I believe the legal ..
err .. IANAL devs are campaigning for full rights to be owned and
enforced by Gentoo Inc LLC in the US .. even if they have no idea what
that means or does .. :]

MJE



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] emerge: add --autounmask-keep-keywords option (bug 622480)

[M. J. Everitt]

On 14/08/17 01:19, Zac Medico wrote [excerpted]:
> You'll get the same result as --autounmask-use-only if you use
> --autounmask-keep-keywords together with --autounmask-keep-masks.
>
> This way, we can also add --autounmask-keep-license and
> --autounmask-keep-use options if we want, and the meaning is very easy
> to explain. The meaning of --autounmask-use-only would have required
> more explanation, since it allowed license changes in addition to USE
> changes.
I guess that's the more flexible option, and logical to all users of all
locales ;)

Does portage ignore missing command-line options, if I set them in my
make.conf in anticipation!?!?!

Thanks for the clear explanation, extra coding, and look forward to the
push to 'live' :)

Michael.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] emerge: add --autounmask-keep-keywords option (bug 622480)

[M. J. Everitt]

On 12/08/17 19:08, Zac Medico wrote:
> The option prevents --autounmask from making changes to
> package.accept_keywords. This option does not imply
> --autounmask-keep-masks, so --autounmask is still allowed
> to create package.unmask changes unless the
> --autounmask-keep-masks is also specified.
>
> X-Gentoo-bug: 622480
> X-Gentoo-bug-url: https://bugs.gentoo.org/622480
> ---
>  man/emerge.1   |  7 +++
>  pym/_emerge/depgraph.py| 12 ++--
>  pym/_emerge/main.py|  9 +++
>  .../resolver/test_autounmask_keep_keywords.py  | 68 
> ++
>  4 files changed, 92 insertions(+), 4 deletions(-)
>  create mode 100644 
> pym/portage/tests/resolver/test_autounmask_keep_keywords.py
>
> diff --git a/man/emerge.1 b/man/emerge.1
> index ffb453efb..12a0db166 100644
> --- a/man/emerge.1
> +++ b/man/emerge.1
> @@ -395,6 +395,13 @@ using the \'=\' operator will be written. With this
>  option, \'>=\' operators will be used whenever possible.
>  USE and license changes always use the latter behavior.
>  .TP
> +.BR "\-\-autounmask\-keep\-keywords [ y | n ]"
> +If \-\-autounmask is enabled, no package.accept_keywords changes will
> +be created. This leads to unsatisfied dependencies if any keyword
> +changes are required. This option does not imply \-\-autounmask\-keep\-masks,
> +so \-\-autounmask is still allowed to create package.unmask changes unless
> +the \-\-autounmask\-keep\-masks is also specified.
> +.TP
>  .BR "\-\-autounmask\-keep\-masks [ y | n ]"
>  If \-\-autounmask is enabled, no package.unmask or ** keyword changes
>  will be created. This leads to unsatisfied dependencies if
> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
> index abe2cb1bd..b4fc5f297 100644
> --- a/pym/_emerge/depgraph.py
> +++ b/pym/_emerge/depgraph.py
> @@ -5707,6 +5707,7 @@ class depgraph(object):
>   if self._dynamic_config._autounmask is not True:
>   return
>  
> + autounmask_keep_keywords = 
> self._frozen_config.myopts.get("--autounmask-keep-keywords", "n") != "n"
>   autounmask_keep_masks = 
> self._frozen_config.myopts.get("--autounmask-keep-masks", "n") != "n"
>   autounmask_level = self._AutounmaskLevel()
>  
> @@ -5716,14 +5717,16 @@ class depgraph(object):
>   autounmask_level.allow_license_changes = True
>   yield autounmask_level
>  
> - autounmask_level.allow_unstable_keywords = True
> - yield autounmask_level
> -
> - if not autounmask_keep_masks:
> + if not autounmask_keep_keywords:
> + autounmask_level.allow_unstable_keywords = True
> + yield autounmask_level
>  
> + if not (autounmask_keep_keywords or autounmask_keep_masks):
> + autounmask_level.allow_unstable_keywords = True
>   autounmask_level.allow_missing_keywords = True
>   yield autounmask_level
>  
> + if not autounmask_keep_masks:
>   # 4. USE + license + masks
>   # Try to respect keywords while discarding
>   # package.mask (see bug #463394).
> @@ -5732,6 +5735,7 @@ class depgraph(object):
>   autounmask_level.allow_unmasks = True
>   yield autounmask_level
>  
> + if not (autounmask_keep_keywords or autounmask_keep_masks):
>   autounmask_level.allow_unstable_keywords = True
>  
>   for missing_keyword, unmask in ((False, True), (True, 
> True)):
> diff --git a/pym/_emerge/main.py b/pym/_emerge/main.py
> index 2132aa63c..d3a415b91 100644
> --- a/pym/_emerge/main.py
> +++ b/pym/_emerge/main.py
> @@ -129,6 +129,7 @@ def insert_optional_args(args):
>   '--autounmask'   : y_or_n,
>   '--autounmask-continue'  : y_or_n,
>   '--autounmask-only'  : y_or_n,
> + '--autounmask-keep-keywords' : y_or_n,
>   '--autounmask-keep-masks': y_or_n,
>   '--autounmask-unrestricted-atoms' : y_or_n,
>   '--autounmask-write' : y_or_n,
> @@ -348,6 +349,11 @@ def parse_opts(tmpcmdline, silent=False):
>   "choices" : true_y_or_n
>   },
>  
> + "--autounmask-keep-keywords": {
> + "help": "don't add package.accept_keywords entries",
> + "choices" : true_y_or_n
> + },
> +
>   "--autounmask-keep-masks": {
>   "help": "don't add package.unmask entries",
>   "choices" : true_y_or_n
> @@ -797,6 +803,9 @@ def parse_opts(tmpcmdline, silent=False):
>   if myoptions.autounmask_unrestricted_atoms in true_y:
>   myoptions.autounmask_unrestricted_atoms = True
>  
> + if