Re: [gentoo-dev] berkdb and gdbm in global USE defaults

2017-01-26 Thread Mart Raudsepp 
Ühel kenal päeval, N, 26.01.2017 kell 22:33, kirjutas Mike Gilbert:
> I recently ran into a REQUIRED_USE constraint that required I select
> between berkdb and gdbm for an email client.

There shouldn't be a REQUIRED_USE constraint that forces you to select
one or the other. The maintainer should be giving the choice of both,
but if only one can be chosen, the maintainer should make the choice
for you by preferring one of them. Likely gdbm, given berkdb licensing
saga.
Though I guess this is a little bit more problematic when that DB is
long living, but I think it should still work good enough with this
guideline.

Then there is no need to think about what is enabled globally or not.
Point being, use REQUIRED_USE sparingly, and rarely a good idea to
block things with common global USE flags, or demand a local USE flag
based on a default enabled global USE flag without locally USE
defaulting that global flag too - and other such cases.

I'd talk to the maintainer(s) of such package(s) via bugzilla or other
means and discuss such REQUIRED_USE potential overuse.

> Looking through our profiles, I see we have both berkdb and gdbm
> enabled "globally".
> 
> default/linux/make.defaults:USE="berkdb crypt ipv6 ncurses nls pam
> readline ssl tcpd zlib"
> releases/make.defaults:USE="acl gdbm nptl unicode"
> 
> Is there any reason to have these USE flags enabled globally?
> 
> These USE seem pretty package-specific in scope. On my system, they
> are used by around a dozen of 1000+ installed packages. I think it
> might make sense to migrate them to appropriate IUSE defaults, or
> leave them disabled where they do not provide critical functionality.

Re: [gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

2017-01-26 Thread Michał Górny 
On Thu, 26 Jan 2017 18:48:39 -0500
Doug Freed  wrote:

> This particular issue has already been fixed, but I'll be forwarding
> all these emails to the list from now on (I have to do that manually,
> because there are some that aren't anybody's fault, and I don't need
> to spam you about those).

I should point out that:

1) CI is detecting this kind of issues much faster than you are,
and reporting them both to the committer and to a *dedicated* mailing
list, so your mail is completely redundant and delayed.

2) Spamming the developer mailing list is completely unprofessional
here. If you are unhappy about those mails, just disable them, and stop
blaming people for your misery. Trying to prove others incompetent
helps nobody.

That said, if you forward yet another mail with the same purpose,
I will file a formal complaint at ComRel and request suspending your
mailing list access.

-- 
Best regards,
Michał Górny



pgpAN0k4LmfPd.pgp
Description: OpenPGP digital signature

[gentoo-dev] berkdb and gdbm in global USE defaults

2017-01-26 Thread Mike Gilbert 
I recently ran into a REQUIRED_USE constraint that required I select
between berkdb and gdbm for an email client.

Looking through our profiles, I see we have both berkdb and gdbm
enabled "globally".

default/linux/make.defaults:USE="berkdb crypt ipv6 ncurses nls pam
readline ssl tcpd zlib"
releases/make.defaults:USE="acl gdbm nptl unicode"

Is there any reason to have these USE flags enabled globally?

These USE seem pretty package-specific in scope. On my system, they
are used by around a dozen of 1000+ installed packages. I think it
might make sense to migrate them to appropriate IUSE defaults, or
leave them disabled where they do not provide critical functionality.

[gentoo-dev] Last rites: x11-drivers/ati-drivers, x11-libs/amd-adl-sdk, x11-libs/xvba-video

2017-01-26 Thread Matt Turner 

# Matt Turner  (26 Jan 2017)
# Dead and replaced by media-libs/mesa[video_cards_radeonsi]
# (or the proprietary amdgpu-pro, which is not in tree).
# Masked for removal in 30 days.
# Bug #582406
x11-drivers/ati-drivers
x11-libs/amd-adl-sdk
x11-libs/xvba-video


signature.asc
Description: Digital signature

Re: [gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

2017-01-26 Thread Ian Stakenvicius 
On 26/01/17 06:48 PM, Doug Freed wrote:
> This is the email I get when a Manifest is missing DIST entries; it's
> more verbose than it needs to be, but I'd rather have more than less.
> In this particular case, the developer that made the bad commit likely
> had something they were working on in sys-cluster/torque added to the
> git index (ie, they did git add), and then needed to make an unrelated
> change, and didn't stash their changes beforehand.  You should always
> check 'git status' output before running repoman commit and/or git
> commit.  It's probably best to check before you start on a change, and
> then you can 'git stash -u' right away (the -u includes untracked
> files, which is useful if your in progress change is adding something
> new), and then after you've committed the change you wanted to get
> done right away, you can 'git stash pop' to get back to the state you
> were in before.
> 

A maintainer/committer directed email would definitely help with this,
yes; otherwise maybe an RSS feed we could subscribe to would be more
fitting than list emails?

The problem was definitely me, however i'm not sure what I messed up
in this particular case.  What I had done was 'git reset *' and 'git
checkout -- *' within sys-cluster/torque/ to trash all of the
non-committed work.  I'll definitely be more careful in the future.





signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

2017-01-26 Thread M. J. Everitt 
On 27/01/17 00:55, NP-Hardass wrote:
> On 01/26/2017 06:48 PM, Doug Freed wrote:
>> This is the email I get when a Manifest is missing DIST entries; it's
>> more verbose than it needs to be, but I'd rather have more than less.
>> In this particular case, the developer that made the bad commit likely
>> had something they were working on in sys-cluster/torque added to the
>> git index (ie, they did git add), and then needed to make an unrelated
>> change, and didn't stash their changes beforehand.  You should always
>> check 'git status' output before running repoman commit and/or git
>> commit.  It's probably best to check before you start on a change, and
>> then you can 'git stash -u' right away (the -u includes untracked
>> files, which is useful if your in progress change is adding something
>> new), and then after you've committed the change you wanted to get
>> done right away, you can 'git stash pop' to get back to the state you
>> were in before.
>>
>> This particular issue has already been fixed, but I'll be forwarding
>> all these emails to the list from now on (I have to do that manually,
>> because there are some that aren't anybody's fault, and I don't need
>> to spam you about those).
>>
>> -Doug
>>
>>
>> -- Forwarded message --
>> From: (Cron Daemon) <gmir...@gentoo.org>
>> Date: Thu, Jan 26, 2017 at 5:39 PM
>> Subject: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen
>> /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh
>> To: infra-gmir...@gentoo.org
>>
>>
>> [ERROR/ForkPoolWorker-7] sys-cluster/torque is missing DIST entries!
>> Traceback (most recent call last):
>>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
>> 23, in _open_file
>> encoding=_encodings['fs'], errors='strict'), 'rb')
>> FileNotFoundError: [Errno 2] No such file or directory:
>> b'/var/empty/torque-6.1.0.tar.gz'
>>
>> During handling of the above exception, another exception occurred:
>>
>> Traceback (most recent call last):
>>   File "/usr/local/bin/mastermirror/thicken-manifests.py", line 122,
>> in maybe_thicken_manifest
>> manifest.create(assumeDistHashesAlways=True)
>>   File "/usr/lib64/python3.4/site-packages/portage/manifest.py", line
>> 506, in create
>> self.fhashdict["DIST"][f] = perform_multiple_checksums(fname, 
>> self.hashes)
>>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
>> 426, in perform_multiple_checksums
>> rVal[x] = perform_checksum(filename, x, calc_prelink)[0]
>>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
>> 390, in perform_checksum
>> myhash, mysize = hashfunc_map[hashname](myfilename)
>>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
>> 52, in __call__
>> with _open_file(filename) as f:
>>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
>> 31, in _open_file
>> raise portage.exception.FileNotFound(filename)
>> portage.exception.FileNotFound: b'/var/empty/torque-6.1.0.tar.gz'
>>
>> !!! A file listed in the Manifest could not be found:
>> /var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque/torque-6.0.1.ebuild
>> /usr/local/bin/mastermirror/rsync-gen.sh: A Manifest has a failure!
>> /var/tmp/gmirror-rsync/logs/regen/regen-run-20170126-2238.log.validate:
>>
>> RepoMan scours the neighborhood...
>>   digest.missing [fatal]1
>>
>> /var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque::torque-6.1.0.tar.gz
>>   digest.unused 1
>>   file.size 69
>>   manifest.bad [fatal]  1
>>sys-cluster/torque/Manifest
>> Please fix these important QA issues first.
>> RepoMan sez: "Make your QA payment on time and you'll never see the
>> likes of me."
>>
> While increasing QA is a laudable goal, I'm not sure spamming the list
> every time you detect a QA issue is the right way to proceed with
> improving QA.  I'm much rather you adopt a strategy similar to mgorny's
> QA tool, and email the dev responsible for the commit that they broke
> the repo.   Find a package with an issue, extract the last commit on
> that directory, get the committer, and send them the email.  In all
> likelihood, these kinds of errors aren't intentional, and public shaming
> isn't really necessary in this case.  Simply notifying the developer so
> that they can rectify the issue should be sufficient, unless I'm missing
> something.
>
I think this is a useful tool, but I made the suggestion on a previous
thread to post to the "gentoo-automated-testing" ML where anyone
interested can subscribe. There may be potential in the suggestion of
mailing the dev concerned, but perhaps just having somewhere 'visible'
where it can be checked, or devs can be 'pointed' would be a reasonable
'first step'.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

2017-01-26 Thread NP-Hardass 
On 01/26/2017 06:48 PM, Doug Freed wrote:
> This is the email I get when a Manifest is missing DIST entries; it's
> more verbose than it needs to be, but I'd rather have more than less.
> In this particular case, the developer that made the bad commit likely
> had something they were working on in sys-cluster/torque added to the
> git index (ie, they did git add), and then needed to make an unrelated
> change, and didn't stash their changes beforehand.  You should always
> check 'git status' output before running repoman commit and/or git
> commit.  It's probably best to check before you start on a change, and
> then you can 'git stash -u' right away (the -u includes untracked
> files, which is useful if your in progress change is adding something
> new), and then after you've committed the change you wanted to get
> done right away, you can 'git stash pop' to get back to the state you
> were in before.
> 
> This particular issue has already been fixed, but I'll be forwarding
> all these emails to the list from now on (I have to do that manually,
> because there are some that aren't anybody's fault, and I don't need
> to spam you about those).
> 
> -Doug
> 
> 
> -- Forwarded message --
> From: (Cron Daemon) <gmir...@gentoo.org>
> Date: Thu, Jan 26, 2017 at 5:39 PM
> Subject: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen
> /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh
> To: infra-gmir...@gentoo.org
> 
> 
> [ERROR/ForkPoolWorker-7] sys-cluster/torque is missing DIST entries!
> Traceback (most recent call last):
>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
> 23, in _open_file
> encoding=_encodings['fs'], errors='strict'), 'rb')
> FileNotFoundError: [Errno 2] No such file or directory:
> b'/var/empty/torque-6.1.0.tar.gz'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/usr/local/bin/mastermirror/thicken-manifests.py", line 122,
> in maybe_thicken_manifest
> manifest.create(assumeDistHashesAlways=True)
>   File "/usr/lib64/python3.4/site-packages/portage/manifest.py", line
> 506, in create
> self.fhashdict["DIST"][f] = perform_multiple_checksums(fname, self.hashes)
>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
> 426, in perform_multiple_checksums
> rVal[x] = perform_checksum(filename, x, calc_prelink)[0]
>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
> 390, in perform_checksum
> myhash, mysize = hashfunc_map[hashname](myfilename)
>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
> 52, in __call__
> with _open_file(filename) as f:
>   File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
> 31, in _open_file
> raise portage.exception.FileNotFound(filename)
> portage.exception.FileNotFound: b'/var/empty/torque-6.1.0.tar.gz'
> 
> !!! A file listed in the Manifest could not be found:
> /var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque/torque-6.0.1.ebuild
> /usr/local/bin/mastermirror/rsync-gen.sh: A Manifest has a failure!
> /var/tmp/gmirror-rsync/logs/regen/regen-run-20170126-2238.log.validate:
> 
> RepoMan scours the neighborhood...
>   digest.missing [fatal]1
>
> /var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque::torque-6.1.0.tar.gz
>   digest.unused 1
>   file.size 69
>   manifest.bad [fatal]  1
>sys-cluster/torque/Manifest
> Please fix these important QA issues first.
> RepoMan sez: "Make your QA payment on time and you'll never see the
> likes of me."
> 

While increasing QA is a laudable goal, I'm not sure spamming the list
every time you detect a QA issue is the right way to proceed with
improving QA.  I'm much rather you adopt a strategy similar to mgorny's
QA tool, and email the dev responsible for the commit that they broke
the repo.   Find a package with an issue, extract the last commit on
that directory, get the committer, and send them the email.  In all
likelihood, these kinds of errors aren't intentional, and public shaming
isn't really necessary in this case.  Simply notifying the developer so
that they can rectify the issue should be sufficient, unless I'm missing
something.

-- 
NP-Hardass



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Re: [PATCH] spawn: instantiate userpriv_groups before fork (bug 582098)

2017-01-26 Thread Brian Dolbec 
On Thu, 26 Jan 2017 16:05:47 -0800
Zac Medico  wrote:

> On 01/25/2017 04:21 PM, Zac Medico wrote:
> > Make spawn force instantiation of portage.data.userpriv_groups in
> > the main process, in order to avoid redundant instantiation in child
> > processes. This mitigates the impact of "Bad file descriptor" errors
> > reported in bug 582098, by avoiding redundant instantiation of
> > userpriv_groups in child processes. It may even solve the problem
> > completely, if the "Bad file descriptor" errors are triggered by
> > interactions between garbage collection and the file descriptor
> > operations performed in the _exec function by the _setup_pipes call.
> > 
> > X-Gentoo-Bug: 582098
> > X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=582098
> > ---
> >  pym/portage/process.py | 4 
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/pym/portage/process.py b/pym/portage/process.py
> > index ba41ea8..bc4efb5 100644
> > --- a/pym/portage/process.py
> > +++ b/pym/portage/process.py
> > @@ -305,6 +305,10 @@ def spawn(mycommand, env={}, opt_name=None,
> > fd_pipes=None, returnpid=False, if unshare_net or unshare_ipc:
> > find_library("c")
> >  
> > +   # Force instantiation of portage.data.userpriv_groups
> > before the
> > +   # fork, so that the result is cached in the main process.
> > +   bool(groups)
> > +
> > parent_pid = os.getpid()
> > pid = None
> > try:
> >   
> 
> Pushed:
> 
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=ccf975296daec92d376c4989e5ffb2a6cdbe8a2d

Sorry, it's been hard to keep up with email last few days, but yeah,
this is a trivial patch.  I know it's been a bastard to track down the
source of the problem.  Thanks

-- 
Brian Dolbec

[gentoo-dev] Fwd: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen /bin/bash /usr/local/bin/mastermirror/rsync-gen.sh

2017-01-26 Thread Doug Freed 
This is the email I get when a Manifest is missing DIST entries; it's
more verbose than it needs to be, but I'd rather have more than less.
In this particular case, the developer that made the bad commit likely
had something they were working on in sys-cluster/torque added to the
git index (ie, they did git add), and then needed to make an unrelated
change, and didn't stash their changes beforehand.  You should always
check 'git status' output before running repoman commit and/or git
commit.  It's probably best to check before you start on a change, and
then you can 'git stash -u' right away (the -u includes untracked
files, which is useful if your in progress change is adding something
new), and then after you've committed the change you wanted to get
done right away, you can 'git stash pop' to get back to the state you
were in before.

This particular issue has already been fixed, but I'll be forwarding
all these emails to the list from now on (I have to do that manually,
because there are some that aren't anybody's fault, and I don't need
to spam you about those).

-Doug


-- Forwarded message --
From: (Cron Daemon) <gmir...@gentoo.org>
Date: Thu, Jan 26, 2017 at 5:39 PM
Subject: Cron <gmirror@dipper> /usr/local/bin/pidlock -s rsync-gen
/bin/bash /usr/local/bin/mastermirror/rsync-gen.sh
To: infra-gmir...@gentoo.org


[ERROR/ForkPoolWorker-7] sys-cluster/torque is missing DIST entries!
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
23, in _open_file
encoding=_encodings['fs'], errors='strict'), 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
b'/var/empty/torque-6.1.0.tar.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/mastermirror/thicken-manifests.py", line 122,
in maybe_thicken_manifest
manifest.create(assumeDistHashesAlways=True)
  File "/usr/lib64/python3.4/site-packages/portage/manifest.py", line
506, in create
self.fhashdict["DIST"][f] = perform_multiple_checksums(fname, self.hashes)
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
426, in perform_multiple_checksums
rVal[x] = perform_checksum(filename, x, calc_prelink)[0]
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
390, in perform_checksum
myhash, mysize = hashfunc_map[hashname](myfilename)
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
52, in __call__
with _open_file(filename) as f:
  File "/usr/lib64/python3.4/site-packages/portage/checksum.py", line
31, in _open_file
raise portage.exception.FileNotFound(filename)
portage.exception.FileNotFound: b'/var/empty/torque-6.1.0.tar.gz'

!!! A file listed in the Manifest could not be found:
/var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque/torque-6.0.1.ebuild
/usr/local/bin/mastermirror/rsync-gen.sh: A Manifest has a failure!
/var/tmp/gmirror-rsync/logs/regen/regen-run-20170126-2238.log.validate:

RepoMan scours the neighborhood...
  digest.missing [fatal]1
   
/var/tmp/gmirror-rsync/gentoo-x86-stage/sys-cluster/torque::torque-6.1.0.tar.gz
  digest.unused 1
  file.size 69
  manifest.bad [fatal]  1
   sys-cluster/torque/Manifest
Please fix these important QA issues first.
RepoMan sez: "Make your QA payment on time and you'll never see the
likes of me."

Re: [gentoo-portage-dev] [PATCH] sync.py: set returncode when task failed in SyncScheduler._task_exit()

2017-01-26 Thread Zac Medico 
On 01/26/2017 11:00 AM, Alexandru Elisei wrote:
> If task.returncode is not os.EX_OK then returncode isn't defined when
> trying to append the (repo, returncode) tuple to self.retvals. This will
> raise an UnboundLocalError exception.
> 
> This reverts the changes to the function made by commit f143e58.
> ---
>  pym/portage/emaint/modules/sync/sync.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/pym/portage/emaint/modules/sync/sync.py
> b/pym/portage/emaint/modules/sync/sync.py
> index b190b3c..076297a 100644
> --- a/pym/portage/emaint/modules/sync/sync.py
> +++ b/pym/portage/emaint/modules/sync/sync.py
> @@ -362,6 +362,7 @@ class SyncScheduler(AsyncScheduler):
>   # that hooks will be called in a backward-compatible manner
>   # even if all sync tasks have failed.
>   hooks_enabled = True
> + returncode = task.returncode
>   if task.returncode == os.EX_OK:
>   returncode, message, updatecache_flg, hooks_enabled = 
> task.result
>   if message:
> 

Thanks, applied:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9305bccfb29807f54a9dc514d3aea70639eb2c49
-- 
Thanks,
Zac

Re: [gentoo-dev] Unused profiles

2017-01-26 Thread Fabian Groffen 
On 22-01-2017 19:56:58 +0100, Fabian Groffen wrote:
> On 19-01-2017 20:47:46 +0100, Michał Górny wrote:
> > prefix/bsd/freebsd/arch
> > prefix/darwin/macos/arch
> > prefix/sunos/solaris/arch

These profiles are there just for reuse by other profiles.  They should
not be removed, but they shouldn't be pointed to either.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature

[gentoo-dev] Re: Unused profiles

2017-01-26 Thread Michael Haubenwallner 
On 01/22/2017 07:56 PM, Fabian Groffen wrote:
> On 19-01-2017 20:47:46 +0100, Michał Górny wrote:

>> prefix/aix/7.1.0.0/ppc

This one I've added to profiles.desc,

>> prefix/hpux/B.11.11/hppa2.0

while just dropping this one, silently.

I've completely given up on HP-UX here already, and one would
hardly find anything working in current tree at all. Unless we
see some interest in HP-UX by someone within some time, all hpux
profiles (and keywords) may eventually go.

> Please don't remove these.  Most seem missing profile.desc ptrs to me,
> but some are odd, which needs investigation.
> 
> Thanks,
> Fabian


Thanks for notifying!
/haubi/

[gentoo-portage-dev] [PATCH] man/portage.5: replace repos.conf sync-type 'websync' with 'webrsync'

2017-01-26 Thread Alexandru Elisei 
---
 man/portage.5 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/man/portage.5 b/man/portage.5
index 2cacafc..7d40b95 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -989,7 +989,7 @@ repository has changed since the previous sync operation.
 .B sync\-type
 Specifies type of synchronization performed by `emerge \-\-sync`.
 .br
-Valid non\-empty values: cvs, git, rsync, svn, websync (emerge-webrsync)
+Valid non\-empty values: cvs, git, rsync, svn, webrsync (emerge-webrsync)
 .br
 This attribute can be set to empty value to disable synchronization of given
 repository. Empty value is default.
-- 
2.10.2