[gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

[Doug Freed]

This is the email I get when a Manifest is missing DIST entries; it's
more verbose than it needs to be, but I'd rather have more than less.
In this particular case, the developer that made the bad commit likely
had something they were working on in sys-cluster/torque added to the
git index (ie, they did git add), and then needed to make an unrelated
change, and didn't stash their changes beforehand.  You should always
check 'git status' output before running repoman commit and/or git
commit.  It's probably best to check before you start on a change, and
then you can 'git stash -u' right away (the -u includes untracked
files, which is useful if your in progress change is adding something
new), and then after you've committed the change you wanted to get
done right away, you can 'git stash pop' to get back to the state you
were in before.

This particular issue has already been fixed, but I'll be forwarding
all these emails to the list from now on (I have to do that manually,
because there are some that aren't anybody's fault, and I don't need
to spam you about those).

-Doug


-- Forwarded message --
From: (Cron Daemon) 
Date: Thu, Jan 26, 2017 at 5:39 PM
Subject: Cron  /usr/local/bin/pidlock -s rsync-gen
/bin/bash /usr/local/bin/mastermirror/rsync-gen.sh
To: infra-gmir...@gentoo.org


[ERROR/ForkPoolWorker-7] sys-cluster/torque is missing DIST entries!
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
23, in _open_file
encoding=_encodings['fs'], errors='strict'), 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
b'/var/empty/torque-6.1.0.tar.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/mastermirror/thicken-manifests.py", line 122,
in maybe_thicken_manifest
manifest.create(assumeDistHashesAlways=True)
  File "/usr/lib64/python3.4/site-packages/portage/manifest.py", line
506, in create
self.fhashdict["DIST"][f] = perform_multiple_checksums(fname, self.hashes)
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
426, in perform_multiple_checksums
rVal[x] = perform_checksum(filename, x, calc_prelink)[0]
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
390, in perform_checksum
myhash, mysize = hashfunc_map[hashname](myfilename)
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
52, in __call__
with _open_file(filename) as f:
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
31, in _open_file
raise portage.exception.FileNotFound(filename)
portage.exception.FileNotFound: b'/var/empty/torque-6.1.0.tar.gz'

!!! A file listed in the Manifest could not be found:
/var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque/torque-6.0.1.ebuild
/usr/local/bin/mastermirror/rsync-gen.sh: A Manifest has a failure!
/var/tmp/gmirror-rsync/logs/regen/regen-run-20170126-2238.log.validate:

RepoMan scours the neighborhood...
  digest.missing [fatal]1
   
/var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque::torque-6.1.0.tar.gz
  digest.unused 1
  file.size 69
  manifest.bad [fatal]  1
   sys-cluster/torque/Manifest
Please fix these important QA issues first.
RepoMan sez: "Make your QA payment on time and you'll never see the
likes of me."

[gentoo-dev] Re: Common rsync-gen errors, why they happen, and what you can do about it

[Doug Freed]

Oh, I missed some things.  First off, a general timeline of rsync-gen
events (all times occur every hour at roughly specified minute after
the hour):

:00 / :30 - current state of git tree is checked out on git server and
then rsynced to private master rsync server - this is your cutoff time
for getting changes into the rsync-gen run
:06 / :36 - rsync-gen script starts
:09 / :39 (ish) - md5-cache generation and manifest thickening have
usually completed by this time; rsync-gen runs repoman manifest-check
across the whole tree at this time
:15 / :45 (ish) - By this time any failure in the repoman
manifest-check has surfaced, and if so, I (as well as others, mostly
infra) have been emailed about it
:20 / :50 (ish) - The final transfer from the staging directory to the
final directory has usually finished by this time, though that depends
heavily on how many changes were made since the last transfer (if you
break things and don't fix them for a while, this can be a lot)

At some point, the repoman manifest-check, or some variation of it,
will probably get added to a post-receive hook, which will then abort
your push if you try to push something that would break the conversion
process.  That said, you should still be doing your due diligence to
ensure that eventual hook doesn't yell at you.

And in case you're wondering why I get emailed about these failures, I
wrote the script that thickens Manifests, which does the entire tree
as parallel as possible in under a minute.  In addition, I improved
sizable portions of the rsync-gen script itself.  The code isn't
absolutely perfect (no code is), so I get emails about any failures so
I can see if it's something I need to fix with my code.  But it's been
a while since that's been the case, so all the failures these days are
primarily for the previously mentioned issues.

-Doug

[gentoo-dev] Common rsync-gen errors, why they happen, and what you can do about it

[Doug Freed]

The two most common errors that occur in the rsync-gen process are
missing DIST entries in the Manifest committed to git, and
metadata.xml that fails to validate against the DTD.  Two much less
common errors that have occurred in the last 24 hours which prompted
this email are FILESDIR items that are way too large (>60 KiB), and
completely missing metadata.xml.  All four of these errors are fatal
to the rsync-gen process, and for good reason.

Anything fatal to rsync-gen completely stops the git to rsync
conversion, and the rsync tree remains at the state it was before the
failure until the failure is resolved.  If it isn't apparent, this
completely blocks all updates, GLSA distribution, news item
distribution, and anything else distributed by the rsync tree either
now or in the future, until the problem is fixed.  If you're curious
about the current age of the rsync tree due to failures, you can
retrieve the metadata/timestamp or metadata/timestamp.x (not
timestamp.chk) file from rsync.gentoo.org and inspect its contents (or
do a verbose listing of the file, and look at the mtime).  The
rsync.gentoo.org rotation only contains servers managed by infra,
which sync against the private master rsync server very frequently, so
you can be reasonably sure they're about as up to date as possible.
Explanation of the various timestamp files:

* timestamp: created by rsync-gen script in the staging directory
immediately after md5-cache generation and manifest thickening is
complete.
* timestamp.x: created by cron job periodically (script comment says
every 5 minutes, but I think it's less often than that) in the staging
directory
* timestamp.chk: created by cron job periodically (again, script
comment says every 5 minutes, but I think it's every 15 minutes) in
the *final* directory

The final directory is what is served by the private master rsync
server, so you can use timestamp.chk to see the age of your local
mirror relative to the private master rsync server.  The staging
directory is transferred to the final directory as part of successful
completion of the rsync-gen script (it's the last step), so if
anything bails before that, it doesn't happen (thus, timestamp and
timestamp.x are mostly equivalent in meaning).

Why do these happen?  The answer is very simple: you forgot to run
repoman in the package directory before pushing your changes.
Alternately, the metadata.xml failing to validate error can happen if
your repoman is old.

So what can you do about it?  First, make sure repoman is installed
and up to date.  As of portage 2.3.0, repoman is a separate package
(app-portage/repoman) with its own release schedule, so once you've
updated portage past that version, you need to install repoman
yourself.  Repoman needs to be at least version 2.3.0 as well (which
it will be if you've installed portage 2.3.0 or newer), because the
metadata.xml validation check does not exist in any previous version,
and as mentioned, that check is fatal to the rsync-gen process.
Second, and I cannot stress this enough, *run repoman.*  Before you
push any changes to a package, run repoman manifest followed by at
least repoman manifest-check.  Repoman manifest will ensure that all
your DIST entries are there, and repoman manifest-check will cover the
other three issues I mentioned.  Repoman's scan and full commands also
include the checks done by manifest-check, so if you're running one of
those already prior to pushing your changes, you don't need to do
repoman manifest-check as well.  Note: pkgcore's repoman-alike (I
believe it's called pcheck) does not behave the same as repoman as of
this writing, and some checks it does not consider fatal are fatal to
repoman (eg the FILESDIR item larger than 60 KiB), so if you're
relying on pcheck for your QA checks, you should also do a repoman
manifest-check for good measure.  This should go without saying, but
if you modify an eclass in a way that affects SRC_URI for any package,
you should go through the packages that use that eclass and run
repoman manifest on them.

Remember, QA includes YOU!  (This also applies to all of you
submitting pull requests or otherwise proxy maintaining packages, and
all members of proxy-maint project; you should all be doing these QA
steps before you submit or push changes.)

Sincerely,

The guy who gets emailed every half hour because you broke the rsync
conversion until it's fixed, also known as Doug or dwfreed.

[gentoo-dev] YATP (Yet Another Tinderbox Project)

[Doug Freed]

I'm in the early design phases of my own tinderbox project, and I'd
appreciate some input on the features people would like to see.  The
basic idea is that it'll be a web interface where people can submit
their own build requests, and then a cluster of VMs will go off and
build them, all while collecting useful information for later use,
such as build logs, binpkgs of the build products, memory, disk, and
CPU usage, etc.  I've created an issue tracker to collect ideas here:

https://bitbucket.org/dwfreed-gentoo/tinderbox-meta/issues

You do not need a Bitbucket account to post to the issue tracker, but
in that case you'll need to pass a recaptcha (javascript will probably
be required, you've been warned), and you won't get emails about
updates.  Please post your suggestions there and not to this list.

Please note, any comments about use of Bitbucket instead of X,
requirements for javascript, or anything else based on your own
personal beliefs will be unceremoniously deleted.  I've already
selected most of the software and libraries I'll be using to implement
this, and attempts to bikeshed those choices will likewise be deleted.

Thanks in advance for any suggestions you may have.

-dwfreed

Re: [gentoo-dev] gpg: signing failed: Inappropriate ioctl for device

[Doug Freed]

On Wed, Dec 14, 2016 at 10:52 AM, M. J. Everitt  wrote:
> I would tend to agree that in a sysadmin role this would be very helpful.
>
> At the risk of repeating myself, is there any einfo/elog displayed when
> portage is (first) updated, to alert users of this functionality? A link
> to the wiki page explaining the features [poke wiki team!] mentioned
> would probably be most useful.
>

The PORTAGE_ELOG_* variables are documented in
/usr/share/portage/config/make.conf.example, which is referenced by
the make.conf man page.

-Doug

Re: [gentoo-dev] [PATCH 2/3] multiprocessing.eclass: Introduce get_nproc() to get no of CPUs

[Doug Freed]

On Wed, Dec 14, 2016 at 12:48 PM, Nathan Zachary
<nathanzach...@gentoo.org> wrote:
> On 14/12/16 10:11, Doug Freed wrote:
>> On Wed, Dec 14, 2016 at 11:04 AM, Michał Górny <mgo...@gentoo.org> wrote:
>>> On Wed, 14 Dec 2016 15:27:25 +0300
>>> Andrew Savchenko <birc...@gentoo.org> wrote:
>>>
>>>> On Tue, 13 Dec 2016 10:36:15 +0100 Michał Górny wrote:
>>>>> +   nproc=$(python -c 'import multiprocessing; 
>>>>> print(multiprocessing.cpu_count());' 2>/dev/null)
>>>> This is not portable. E.g. paludis users can have python-less
>>>> system. Adding dev-lang/python to DEPEND will be also a bad idea,
>>>> since this is quite heavy dependency.
>>> You can bikeshed potential circumstances where it wouldn't work for
>>> the next year. Which doesn't change that it would work quite reliably
>>> for the most of Gentoo users.
>>>
>>>> Since on Linux boxes nproc is from coreutils, which is in @system,
>>>> so looks like only *bsd setups are the problem. On FreeBSD
>>> ...and all other operating systems (see: Prefix).
>>>
>>>>   sysctl -a | egrep -i 'hw.ncpu'
>>> I somehow doubt that would give me the expected number only, and I lack
>>> a BSD install handy to test it.
>> $(sysctl -n hw.ncpu)
>>
> I don't know that the sysctl command works universally:
>
> # sysctl -n hw.ncpu
> sysctl: cannot stat /proc/sys/hw/ncpu: No such file or directory
>

It's BSD-specific (Darwin may have it too, but I'm not in OS X at the
moment, so I can't check), which pretty much describes this branch in
the codepath as well.  Linux users will have the nproc command from
coreutils.

-Doug

[gentoo-dev] Handling ebuild logging output in Portage (Was: gpg: signing failed: Inappropriate ioctl for device)

[Doug Freed]

On Wed, Dec 14, 2016 at 10:45 AM, Rich Freeman  wrote:
> On Wed, Dec 14, 2016 at 10:27 AM, M. J. Everitt  wrote:
>>
>> I do, but only usually if its the last package of an emerge because
>> otherwise its lost many many thousands of lines upwards. Thank goodness
>> for portage's savelog feature. - Actually that reminds me .. someone
>> mentioned a useful tweak to that, with an appropriate FEATURES switch,
>> it would categorise the output of the logging system .. must look that
>> one up again, or poke the wiki team ...
>>
>
> IMO, emailing elogs to root should probably be the default.  By all
> means let people turn it off, but I bet a lot of people don't realize
> it is an option.

Having just looked at portage's mail code, you can't use the mail
subsystem if you have no mailserver or sendmail binary.  Instead,
it'll just throw an error, so you couldn't have any default Portage
configuration to mail them somewhere.  The current default
configuration appends to ${PORT_LOGDIR}/elog/summary.log with qa, log,
warn, and error levels for every package that outputs any of these
classes.  If PORT_LOGDIR is unset, the target is
/var/log/portage/elog/summary.log.

>
> This goes in all my make.conf files:
> PORTAGE_ELOG_CLASSES="warn error log"
> PORTAGE_ELOG_SYSTEM="save mail"
> PORTAGE_ELOG_MAILURI="a...@a.com smtp.server.address"
> PORTAGE_ELOG_MAILSUBJECT="package \${PACKAGE} merged on \${HOST} with notice"
>
> Yes, some packages are a bit spammy and this should be fixed, but in
> general it has prevented more headaches than it has caused.

This can't really be fixed in a good way, and I'd really rather it
wasn't anyway.  A [ -z "$REPLACING_VERSIONS" ] test is only valid in
pkg_* phases (and by PMS, REPLACING_VERSIONS doesn't have to be
defined in pkg_pretend or pkg_setup), and in many cases, using
REPLACING_VERSIONS in any way can be difficult to do correctly.  Using
the readme.gentoo eclass wouldn't be an awful way to go if it really
bothers you, but you could also use mail_summary instead of mail to
reduce the mail spam to 1 email per emerge run.

-Doug

Re: [gentoo-dev] [PATCH 2/3] multiprocessing.eclass: Introduce get_nproc() to get no of CPUs

[Doug Freed]

On Wed, Dec 14, 2016 at 11:04 AM, Michał Górny  wrote:
> On Wed, 14 Dec 2016 15:27:25 +0300
> Andrew Savchenko  wrote:
>
>> On Tue, 13 Dec 2016 10:36:15 +0100 Michał Górny wrote:
>> > +   nproc=$(python -c 'import multiprocessing; 
>> > print(multiprocessing.cpu_count());' 2>/dev/null)
>>
>> This is not portable. E.g. paludis users can have python-less
>> system. Adding dev-lang/python to DEPEND will be also a bad idea,
>> since this is quite heavy dependency.
>
> You can bikeshed potential circumstances where it wouldn't work for
> the next year. Which doesn't change that it would work quite reliably
> for the most of Gentoo users.
>
>> Since on Linux boxes nproc is from coreutils, which is in @system,
>> so looks like only *bsd setups are the problem. On FreeBSD
>
> ...and all other operating systems (see: Prefix).
>
>>   sysctl -a | egrep -i 'hw.ncpu'
>
> I somehow doubt that would give me the expected number only, and I lack
> a BSD install handy to test it.

$(sysctl -n hw.ncpu)

Re: [gentoo-dev] [PATCH 1/5] depend.apache.eclass: Replace build_with_use with has_version

[Doug Freed]

On Fri, Dec 9, 2016 at 1:09 AM, Michał Górny <mgo...@gentoo.org> wrote:
> On Thu,  8 Dec 2016 21:36:28 +0100
> Andreas K. Hüttel <dilfri...@gentoo.org> wrote:
>
>> From: Doug Freed <dwfr...@mtu.edu>
>>
>> ---
>>  eclass/depend.apache.eclass | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/eclass/depend.apache.eclass b/eclass/depend.apache.eclass
>> index b69c2ec..a7d206f 100644
>> --- a/eclass/depend.apache.eclass
>> +++ b/eclass/depend.apache.eclass
>> @@ -290,7 +290,7 @@ has_apache() {
>>  has_apache_threads() {
>>   debug-print-function $FUNCNAME $*
>>
>> - if ! built_with_use www-servers/apache threads; then
>> + if ! has_version 'www-servers/apache[threads]'; then
>>   return
>>   fi
>>
>> @@ -313,14 +313,14 @@ has_apache_threads() {
>>  has_apache_threads_in() {
>>   debug-print-function $FUNCNAME $*
>>
>> - if ! built_with_use www-servers/apache threads; then
>> + if ! has_version 'www-servers/apache[threads]'; then
>>   return
>>   fi
>>
>>   local myforeign="$1"
>>   local myflag="${2:-threads}"
>>
>> - if ! built_with_use ${myforeign} ${myflag}; then
>> + if ! has_version "${myforeign}[${myflag}]"; then
>>   echo
>>   eerror "You need to enable USE flag '${myflag}' in 
>> ${myforeign} to"
>>   eerror "build a thread-safe version of ${CATEGORY}/${PN} for 
>> use"
>
> I don't think it's valid for EAPI 0/1. You should probably move the
> EAPI 1 ban first, to avoid keeping half-broken state between commits,
> and add an explicit die call here for EAPI 0. It's better if ebuild
> dies with explanation rather than unreliably fails with invalid
> has_version syntax.

Correct, USE dependencies don't exist until EAPI 2 (which is why
people used built_with_use instead of has_version).  This is why I
banned both EAPI 0 and 1 in my version of the patch.  There are
presently 19 EAPI 0 consumers of this eclass, though.

-Doug

Re: [gentoo-dev] rdp vs rdesktop vs freerdp USE flags

[Doug Freed]

On Thu, Dec 8, 2016 at 7:38 AM, Andrew Savchenko  wrote:
> Hi,
>
> On Thu, 08 Dec 2016 11:29:51 +0100 Pacho Ramos wrote:
>> When looking at freerdp reverse deps I noticed we are using three different
>> names for USE flags enabling freerdp support: rdp, rdesktop and freerdp
>>
>> rdesktop is the only one that is a global USE flag, even if it's used only by
>> two packages, the others are local USE flags that are enabling similar
>> supports.
>>
>> What should we do? Move all to rdesktop?
>
> Move everything to rdp, since this one is most common; add it to
> global flags and remove rdesktop from the list.

+1; RDP is the protocol, whereas freerdp/rdesktop is the
implementation.  This allows one to later replace the dependency with
an any-of or virtual, without needing to change the useflag.

-Doug
dwfreed

[gentoo-dev] [PATCH] depend.apache.eclass: fix for EAPI=6

[Doug Freed]

This also fixes various other code smells.  I wrote this back in the start of
October, and never got around to sending it to the list for review.  A note:
given how extensively this patch changes the eclass, it is suggested that it
actually become depend.apache-r1 and only support EAPI=6, and the EAPI=6
ebuilds that presently inherit depend.apache can inherit this instead.
---
 eclass/depend.apache.eclass | 156 ++--
 1 file changed, 93 insertions(+), 63 deletions(-)

diff --git a/eclass/depend.apache.eclass b/eclass/depend.apache.eclass
index b69c2ec..2ef18cf 100644
--- a/eclass/depend.apache.eclass
+++ b/eclass/depend.apache.eclass
@@ -36,23 +36,21 @@
 # want_apache2 server
 #
 # pkg_setup() {
-#  depend.apache_pkg_setup server
+#  depend.apache_pkg_setup
 # }
 # @CODE
-
-inherit multilib
+#
+# NOTE: Unless you have an indirect dependency on Apache, if you have your
+# own pkg_setup, you must call depend.apache_pkg_setup to get the variables
+# provided by this eclass.
 
 case ${EAPI:-0} in
-   0|1|2|3|4|5)
+   2|3|4|5)
+   inherit multilib
;;
6)
-   ewarn
-   ewarn "EAPI=${EAPI} is not supported by depend.apache.eclass."
-   ewarn "This means that ${CATEGORY}/${PF} is most likely buggy."
-   ewarn "Please file a report on https://bugs.gentoo.org/;
-   ewarn
;;
-   *)
+   0|1|*)
die "EAPI=${EAPI} is not supported by depend.apache.eclass"
;;
 esac
@@ -64,47 +62,47 @@ esac
 # @ECLASS-VARIABLE: APACHE_VERSION
 # @DESCRIPTION:
 # Stores the version of apache we are going to be ebuilding.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APXS
 # @DESCRIPTION:
 # Path to the apxs tool.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_BIN
 # @DESCRIPTION:
 # Path to the apache binary.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_CTL
 # @DESCRIPTION:
 # Path to the apachectl tool.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_BASEDIR
 # @DESCRIPTION:
 # Path to the server root directory.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_CONFDIR
 # @DESCRIPTION:
 # Path to the configuration file directory.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_MODULES_CONFDIR
 # @DESCRIPTION:
 # Path where module configuration files are kept.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_VHOSTS_CONFDIR
 # @DESCRIPTION:
 # Path where virtual host configuration files are kept.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_MODULESDIR
 # @DESCRIPTION:
 # Path where we install modules.
-# This variable is set by the want/need_apache functions.
+# This variable is set by depend.apache-pkg_setup.
 
 # @ECLASS-VARIABLE: APACHE_DEPEND
 # @DESCRIPTION:
@@ -126,6 +124,17 @@ APACHE2_2_DEPEND="=www-servers/apache-2.2*"
 # Dependencies for Apache 2.4.x
 APACHE2_4_DEPEND="=www-servers/apache-2.4*"
 
+# @ECLASS-VARIABLE: APACHE_DEPENDENCY
+# @INTERNAL
+# @DESCRIPTION:
+# Whether we want or need Apache (used in pkg_setup)
+APACHE_DEPENDENCY="none"
+
+# @ECLASS-VARIABLE: APACHE_USEFLAG
+# @INTERNAL
+# @DESCRIPTION:
+# What useflag indicates we want apache2, defaults to apache2
+APACHE_USEFLAG="apache2"
 
 # 
==
 # INTERNAL FUNCTIONS
@@ -134,8 +143,6 @@ APACHE2_4_DEPEND="=www-servers/apache-2.4*"
 _init_apache2() {
debug-print-function $FUNCNAME $*
 
-   # WARNING: Do not use these variables with anything that is put
-   # into the dependency cache (DEPEND/RDEPEND/etc)
APACHE_VERSION="2"
APXS="/usr/sbin/apxs2"
APACHE_BIN="/usr/sbin/apache2"
@@ -153,16 +160,42 @@ _init_no_apache() {
APACHE_VERSION="0"
 }
 
+# @FUNCTION: _apache_dependency
+# @USAGE: { want | need } dependency [myiuse]
+# @DESCRIPTION:
+# Sets up apache dependencies. First parameter is the literal string "want" or
+# "need" to indicate the type of dependency. Second parameter is the dependency
+# string for {R,}DEPEND. Third parameter overrides the default useflag
+# indicating apache2 is wanted.
+_apache_dependency() {
+   debug-print-function $FUNCNAME $*
+
+   case $1 in
+   want)
+  

Re: [gentoo-dev] rsync.gentoo.org rsync modules: ChangeLogs dropped from gentoo-portage

[Doug Freed]

On Wed, Nov 16, 2016 at 3:26 AM, Kent Fredric  wrote:
> On Tue, 15 Nov 2016 01:11:48 +
> "Robin H. Johnson"  wrote:
>
>> They will continue to be included in the Manifest until the next phase
>> of the change.
>>
>> rsync.gentoo.org::gentoo-portage
>> - tree without ChangeLogs
>>
>> rsync.gentoo.org::gentoo-repo
>> - same as gentoo-portage, new name for consistency.
>>
>> rsync.gentoo.org::gentoo-repo-changelog
>> - ChangeLogs only (as previously announced).
>>
>> $SOON (aiming for 2016/Nov/19):
>> - ChangeLog generation order will be switching back to newest-first, and
>>   will be DROPPED from Manifests by this point at the latest.
>
> So are we going to have a broken rsync tree for 4 days due to the manifest
> validation failing on every package, resulting in no working upgrades for
> 4 days?
>
> Or has that problem been fixed?

Portage does not care if ChangeLog is missing on disk during Manifest
checking, because its existence is completely irrelevant to the
functioning of the package.

[gentoo-portage-dev] [PATCH] Change how the tmp file for the commit msg is made (bug 571546)

[Doug Freed]

From: Doug Goldstein 

Changes the way the temporary file for the commit message is generated
to make a temporary directory and create a file called COMMIT_EDITMSG
inside of it. This is to cause various editors to treat this as a git
style commit message by default.

Signed-off-by: Doug Goldstein 
X-Gentoo-Bug: 571546
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=571546
---
 repoman/pym/repoman/actions.py   | 21 +++--
 repoman/pym/repoman/utilities.py | 39 ---
 2 files changed, 31 insertions(+), 29 deletions(-)

diff --git a/repoman/pym/repoman/actions.py b/repoman/pym/repoman/actions.py
index 4144b45..822a538 100644
--- a/repoman/pym/repoman/actions.py
+++ b/repoman/pym/repoman/actions.py
@@ -6,6 +6,7 @@ import errno
 import io
 import logging
 import platform
+import shutil
 import signal
 import sys
 import tempfile
@@ -446,15 +447,15 @@ class Actions(object):
myfiles += myremoved
myfiles.sort()
 
-   fd, commitmessagefile = tempfile.mkstemp(".repoman.msg")
-   mymsg = os.fdopen(fd, "wb")
-   mymsg.write(_unicode_encode(commitmessage))
-   mymsg.close()
+   commitmessagedir = tempfile.mkdtemp(".repoman.msg")
+   commitmessagefile = os.path.join(commitmessagedir, 
"COMMIT_EDITMSG")
+   with open(commitmessagefile, "wb") as mymsg:
+   mymsg.write(_unicode_encode(commitmessage))
 
retval = self.vcs_settings.changes.commit(myfiles, 
commitmessagefile)
# cleanup the commit message before possibly exiting
try:
-   os.unlink(commitmessagefile)
+   shutil.rmtree(commitmessagedir)
except OSError:
pass
if retval != os.EX_OK:
@@ -467,10 +468,10 @@ class Actions(object):
 
def priming_commit(self, myupdates, myremoved, commitmessage):
myfiles = myupdates + myremoved
-   fd, commitmessagefile = tempfile.mkstemp(".repoman.msg")
-   mymsg = os.fdopen(fd, "wb")
-   mymsg.write(_unicode_encode(commitmessage))
-   mymsg.close()
+   commitmessagedir = tempfile.mkdtemp(".repoman.msg")
+   commitmessagefile = os.path.join(commitmessagedir, 
"COMMIT_EDITMSG")
+   with open(commitmessagefile, "wb") as mymsg:
+   mymsg.write(_unicode_encode(commitmessage))
 
separator = '-' * 78
 
@@ -489,7 +490,7 @@ class Actions(object):
retval = self.vcs_settings.changes.commit(myfiles, 
commitmessagefile)
# cleanup the commit message before possibly exiting
try:
-   os.unlink(commitmessagefile)
+   shutil.rmtree(commitmessagedir)
except OSError:
pass
if retval != os.EX_OK:
diff --git a/repoman/pym/repoman/utilities.py b/repoman/pym/repoman/utilities.py
index 8a757dc..c204faa 100644
--- a/repoman/pym/repoman/utilities.py
+++ b/repoman/pym/repoman/utilities.py
@@ -30,7 +30,7 @@ import sys
 import time
 import textwrap
 import difflib
-from tempfile import mkstemp
+import tempfile
 
 # import our initialized portage instance
 from repoman._portage import portage
@@ -187,23 +187,24 @@ def get_commit_message_with_editor(editor, message=None, 
prefix=""):
@rtype: string or None
@return: A string on success or None if an error occurs.
"""
-   fd, filename = mkstemp()
+   commitmessagedir = tempfile.mkdtemp(".repoman.msg")
+   filename = os.path.join(commitmessagedir, "COMMIT_EDITMSG")
try:
-   os.write(
-   fd, _unicode_encode(_(
-   prefix +
-   "\n\n# Please enter the commit message "
-   "for your changes.\n# (Comment lines starting "
-   "with '#' will not be included)\n"),
-   encoding=_encodings['content'], 
errors='backslashreplace'))
-   if message:
-   os.write(fd, b"#\n")
-   for line in message:
-   os.write(
-   fd, _unicode_encode(
-   "#" + line, 
encoding=_encodings['content'],
-   errors='backslashreplace'))
-   os.close(fd)
+   with open(filename, "wb") as mymsg:
+   mymsg.write(
+   _unicode_encode(_(
+   prefix +
+   "\n\n# Please enter the commit message "
+   "for your