Re: [gentoo-portage-dev] [PATCH 2/2] pym/portage/util/locale.py: add a C module to check locale

2016-05-19 Thread Michał Górny
;A'] = i; >+ rlc[i-'A'] = tolower(i); >+ } >+ >+ for (i = 'a'; i <= 'z'; i++) >+ if(lc[i-'a'] != rlc[i-'a']) { >+ lower = 0; >+ break; >+ } >+ >+ for (i = 'A'; i <= 'Z'; i++) >+ if(uc[i-'A'] != ruc[i-'A']) { >+ upper = 0; >+ break; >+ } >+ >+ if (lower == 0 || upper == 0) { >+ strcpy(msg, >+ "!!! WARNING: The LC_CTYPE variable is set to a locale >that >specifies\n" >+ "!!! transformation between lowercase and uppercase >ASCII >characters that\n" >+ "!!! is different than the one specified by POSIX >locale. This can >break\n" >+ "!!! ebuilds and cause issues in programs that rely on >the common >character\n" >+ "!!! conversion scheme. Please consider enabling >another locale >(such as\n" >+ "!!! en_US.UTF-8) in /etc/locale.gen and setting it as >LC_CTYPE >in\n" >+ "!!! make.conf.\n\n" >+ ); >+ >+ if (lower == 0) >+ error_msg(msg, lc, ruc, uc); >+ >+ if (upper == 0) >+ error_msg(msg, uc, rlc, lc); >+ } >+ >+#if PY_MAJOR_VERSION >= 3 >+ return Py_BuildValue("iy", lower && upper, msg); >+#else >+ return Py_BuildValue("is", lower && upper, msg); >+#endif >+} -- Best regards, Michał Górny (by phone)

Re: [gentoo-portage-dev] [PATCH 2/2] pym/portage/util/locale.py: add a C module to check locale

2016-05-20 Thread Michał Górny
On Fri, 20 May 2016 06:59:14 -0400 "Anthony G. Basile" wrote: > On 5/19/16 9:38 AM, Michał Górny wrote: > > Dnia 19 maja 2016 14:43:38 CEST, "Anthony G. Basile" > > napisał(a): > >> From: "Anthony G. Basile" > >> > >&

[gentoo-portage-dev] [PATCH 1/2] portage.package.ebuild.config: Rename iuse_implicit -> iuse_effective

2016-05-20 Thread Michał Górny
Rename the iuse_implicit variable used in USE_EXPAND handling to iuse_effective, since that is what is actually passed there. Correct naming makes figuring out what the function does much easier. --- pym/portage/package/ebuild/config.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)

[gentoo-portage-dev] [PATCH 2/2] ebuild.config: Fix filtering all USE_EXPAND variables in EAPI 5+

2016-05-20 Thread Michał Górny
Ensure that all USE_EXPAND variables are properly filtered and exported in EAPI 5 and newer, as required by the PMS. This includes exporting an empty value if no matching flag is provided in IUSE. Bug: https://bugs.gentoo.org/show_bug.cgi?id=582140 --- pym/portage/eapi.py | 6 ++

Re: [gentoo-portage-dev] [PATCH] ebuild: Extend helper-in-global-scope ban to all EAPIs

2016-05-20 Thread Michał Górny
On Thu, 5 May 2016 23:10:43 +0200 Michał Górny wrote: > Make helper calls in global scope fatal in all supported EAPIs since > this is the behavior required by PMS and all major offenders are fixed > already. > --- > bin/eapi.sh | 4 >

Re: [gentoo-portage-dev] [PATCH 2/2] ebuild.config: Fix filtering all USE_EXPAND variables in EAPI 5+

2016-05-20 Thread Michał Górny
On Sat, 21 May 2016 00:26:40 +0200 Michał Górny wrote: > Ensure that all USE_EXPAND variables are properly filtered and exported > in EAPI 5 and newer, as required by the PMS. This includes exporting > an empty value if no matching flag is provided in IUSE. > > Bug: https://

[gentoo-portage-dev] [PATCH v2 1/2] portage.package.ebuild.config: Rename iuse_implicit -> iuse_effective

2016-05-21 Thread Michał Górny
Rename the iuse_implicit variable used in USE_EXPAND handling to iuse_effective, since that is what is actually passed there. Correct naming makes figuring out what the function does much easier. --- pym/portage/package/ebuild/config.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)

[gentoo-portage-dev] [PATCH v2 2/2] portage.package.ebuild.config: Always export filtered USE_EXPAND vars

2016-05-21 Thread Michał Górny
Ensure that all USE_EXPAND variables are always exported with filtered USE flags inside, even if none of those flags are declared in IUSE. This is the behavior required for EAPI 5+ by the PMS. Since the behavior for earlier EAPIs is left undefined and having different behavior would be confusing t

[gentoo-portage-dev] [PATCH 3/3] portage.dbapi.vartree: Support exclusions in INSTALL_MASK

2016-05-21 Thread Michał Górny
Allow INSTALL_MASK patterns to start with '-' to indicate that a specific match is to be excluded from being masked. In this case, the last matching pattern determines whether the file is actually filtered out or kept. --- pym/portage/dbapi/vartree.py | 10 ++ 1 file changed, 6 insertions(

[gentoo-portage-dev] [PATCH 2/3] Move INSTALL_MASK handling into merging

2016-05-21 Thread Michał Górny
Introduce a new logic for INSTALL_MASK handling in merging code, replacing the old code that removed matching files and directories from imagedir in bash. The new code actually ignores matching files on-the-fly while testing for file collisions and merging files. The files are still written to CONT

[gentoo-portage-dev] [PATCH 1/3] portage.package.ebuild.config: Move FEATURES=no* handling there

2016-05-21 Thread Michał Górny
Move the code responsible for adding additional paths to INSTALL_MASK into portage.package.ebuild.config. --- bin/misc-functions.sh| 13 - pym/portage/package/ebuild/config.py | 10 ++ 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/bin/misc-fun

[gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK redesign, part I

2016-05-21 Thread Michał Górny
will be able to use them. In a few days, I'll try to provide a part II that would implement the actual mask groups. However, I may need some help adding support for the configuration files. I'm not touching PKG_INSTALL_MASK for now since it's harder and outsi

Re: [gentoo-portage-dev] [PATCH 2/3] Move INSTALL_MASK handling into merging

2016-05-22 Thread Michał Górny
On Sun, 22 May 2016 08:56:03 +0200 Michał Górny wrote: > Introduce a new logic for INSTALL_MASK handling in merging code, > replacing the old code that removed matching files and directories > from imagedir in bash. The new code actually ignores matching files > on-the-fly while test

[gentoo-portage-dev] [PATCH v2] Move INSTALL_MASK handling into merging

2016-05-22 Thread Michał Górny
Introduce a new logic for INSTALL_MASK handling in merging code, replacing the old code that removed matching files and directories from imagedir in bash. The new code actually ignores matching files on-the-fly while testing for file collisions and merging files. The files are still written to CONT

[gentoo-portage-dev] [PATCH 1/2] portage.util.configparser: Provide common code to handle cp imports

2016-05-22 Thread Michał Górny
Provide a common code unit to handle portable *ConfigParser imports for all supported Python versions. --- pym/portage/_sets/__init__.py | 10 ++ pym/portage/repository/config.py | 9 + pym/portage/util/_desktop_entry.py | 7 ++- pym/portage/util/configparser.py | 22

[gentoo-portage-dev] [PATCH 2/2] portage.util.configparser: Commonize portable config file reading routine

2016-05-22 Thread Michał Górny
--- pym/portage/_sets/__init__.py | 29 ++- pym/portage/repository/config.py | 38 ++--- pym/portage/util/_desktop_entry.py | 20 ++--- pym/portage/util/configparser.py | 57 +- 4 files changed, 64 insertions(

Re: [gentoo-portage-dev] [PATCH 2/2] pym/portage/util/locale.py: add a C module to help check locale

2016-05-22 Thread Michał Górny
urn; > +#endif > +} To be honest, I think we'd be happier having two big #ifdefs for init funcs rather than one function with a lot of #ifdefs, and the common ConvertCaseError part in a separate static function. > + > + > +static PyObject * > +portage_c_tolower(PyObject *self, PyObject *args)

Re: [gentoo-portage-dev] [PATCH 2/2] pym/portage/util/locale.py: add a C module to help check locale

2016-05-23 Thread Michał Górny
On Mon, 23 May 2016 08:08:18 -0400 "Anthony G. Basile" wrote: > On 5/23/16 2:44 AM, Michał Górny wrote: > > On Sun, 22 May 2016 13:04:40 -0400 > > "Anthony G. Basile" wrote: > > > >> From: "Anthony G. Basile" > >> > &g

Re: [gentoo-portage-dev] [PATCH 2/2] portage.util.configparser: Commonize portable config file reading routine

2016-05-23 Thread Michał Górny
On Sun, 22 May 2016 11:04:51 -0700 Zac Medico wrote: > On 05/22/2016 01:41 AM, Michał Górny wrote: > > --- > > pym/portage/_sets/__init__.py | 29 ++- > > pym/portage/repository/config.py | 38 ++--- > > pym/portage/

[gentoo-portage-dev] [PATCH] Remove unneeded native_kwargs wrapper

2016-05-24 Thread Michał Górny
Remove the native_kwargs wrapper and all uses of it. As pointed out by Zac Medico, this was only needed for Python < 2.6.5, and we no longer support those versions for quite some time. --- bin/portageq | 2 +- pym/_emerge/MiscFunctionsProcess.py

Re: [gentoo-portage-dev] [PATCH 3/3] pym/portage/util/locale.py: add a C module to help check locale

2016-05-28 Thread Michał Górny
t; +} > +#else > +initlibc(void) > +{ > + Py_InitModule("libc", LibcMethods); > +} > +#endif > + > + > +static PyObject * > +_libc_tolower(PyObject *self, PyObject *args) > +{ > + int c; > + > + if (!PyArg_ParseTuple(args, "i", &c)) > + return NULL; > + > + return Py_BuildValue("i", tolower(c)); > +} > + > + > +static PyObject * > +_libc_toupper(PyObject *self, PyObject *args) > +{ > + int c; > + > + if (!PyArg_ParseTuple(args, "i", &c)) > + return NULL; > + > + return Py_BuildValue("i", toupper(c)); > +} Aside from that, it look nice and shiny now. Thanks a lot! -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> pgpMDaEA9fng0.pgp Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 3/3] pym/portage/util/locale.py: add a C module to help check locale

2016-05-29 Thread Michał Górny
On Sun, 29 May 2016 02:37:50 -0400 "Anthony G. Basile" wrote: > On 5/29/16 2:30 AM, Michał Górny wrote: > > On Fri, 27 May 2016 10:26:44 -0400 > > "Anthony G. Basile" wrote: > > > >> From: "Anthony G. Basile" > >> > &g

Re: [gentoo-portage-dev] [PATCH] Remove unneeded native_kwargs wrapper

2016-05-31 Thread Michał Górny
Merged now. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> pgpWLd3EwKFeo.pgp Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH 4/3] portage.package.ebuild.config: Support path groups from install-mask.conf

2016-05-31 Thread Michał Górny
--- .../package/ebuild/_config/InstallMaskManager.py | 59 ++ pym/portage/package/ebuild/config.py | 34 - pym/portage/util/configparser.py | 19 ++- 3 files changed, 110 insertions(+), 2 deletions(-) create mode 100644 pym/por

Re: [gentoo-portage-dev] New meeting

2016-06-07 Thread Michał Górny
LMhLmFyiJftThMV7bnGmuc6EvtLl72oM/Ca/4WI8ZDO4XqkuWIth1vdZKyq3wm >3xI1Cwsow2Fjg0kN8e2nkKgZHw/3ulEZnvaC9QijkCjJpjxD013kHtd3xhndBmAi >gJMtQf4MOtoluiq7wKVQMUQb6yty8jyWjiHrO9YUwwhXFodfZ10/aXwx3aK20e2U >BgyVxpG+gxwkBJbuEQWIQbgvQHxyuwOlRvaE7KyRvsTLmrbMQjaf8ya7VX/Tzl6H >EJxQpEIwqARgVMY6WluPRSAg6vqB1f/ozLntpjqCwInyHYO1y2AS2++VPLusoZkC >Mtr9L1PTgtVfGWf2edy8 >=PUFC >-END PGP SIGNATURE- -- Best regards, Michał Górny (by phone)

Re: [gentoo-portage-dev] [PATCH 4/3] portage.package.ebuild.config: Support path groups from install-mask.conf

2016-06-10 Thread Michał Górny
On Tue, 31 May 2016 17:58:34 +0200 Michał Górny wrote: > --- > .../package/ebuild/_config/InstallMaskManager.py | 59 > ++ > pym/portage/package/ebuild/config.py | 34 - > pym/portage/util/configparser.py | 19 +++

Re: [gentoo-portage-dev] [PATCH v2] Move INSTALL_MASK handling into merging

2016-06-12 Thread Michał Górny
Dnia 12 czerwca 2016 11:10:55 CEST, Zac Medico napisał(a): >On 05/22/2016 01:21 AM, Michał Górny wrote: >> Introduce a new logic for INSTALL_MASK handling in merging code, >> replacing the old code that removed matching files and directories >> from imagedir in bash. The new

Re: [gentoo-portage-dev] [PATCH 3/3] portage.dbapi.vartree: Support exclusions in INSTALL_MASK

2016-06-12 Thread Michał Górny
Dnia 12 czerwca 2016 11:20:36 CEST, Zac Medico napisał(a): >On 05/21/2016 11:56 PM, Michał Górny wrote: >> Allow INSTALL_MASK patterns to start with '-' to indicate that >> a specific match is to be excluded from being masked. In this case, >> the last matching patt

Re: [gentoo-portage-dev] [PATCH 1/3] portage.package.ebuild.config: Move FEATURES=no* handling there

2016-06-12 Thread Michał Górny
Dnia 12 czerwca 2016 09:28:14 CEST, Zac Medico napisał(a): >On 06/12/2016 12:19 AM, Zac Medico wrote: >> On 05/21/2016 11:56 PM, Michał Górny wrote: >>> diff --git a/pym/portage/package/ebuild/config.py >b/pym/portage/package/ebuild/config.py >>> index 45b7d08

Re: [gentoo-portage-dev] [PATCH 3/3] portage.dbapi.vartree: Support exclusions in INSTALL_MASK

2016-06-12 Thread Michał Górny
Dnia 12 czerwca 2016 11:43:35 CEST, Zac Medico napisał(a): >On 06/12/2016 02:31 AM, Michał Górny wrote: >> Dnia 12 czerwca 2016 11:20:36 CEST, Zac Medico >napisał(a): >>> On 05/21/2016 11:56 PM, Michał Górny wrote: >>>> Allow INSTALL_MASK patterns to start

Re: [gentoo-portage-dev] [PATCH v2] Move INSTALL_MASK handling into merging

2016-06-12 Thread Michał Górny
Dnia 12 czerwca 2016 11:49:26 CEST, Zac Medico napisał(a): >On 06/12/2016 02:28 AM, Michał Górny wrote: >> Dnia 12 czerwca 2016 11:10:55 CEST, Zac Medico >napisał(a): >>> On 05/22/2016 01:21 AM, Michał Górny wrote: >>>> Introduce a new logic for INST

Re: [gentoo-portage-dev] [PATCH] ebuild: Extend helper-in-global-scope ban to all EAPIs

2016-06-26 Thread Michał Górny
On Sun, 26 Jun 2016 17:07:09 -0700 Zac Medico wrote: > On 05/20/2016 03:27 PM, Michał Górny wrote: > > On Thu, 5 May 2016 23:10:43 +0200 > > Michał Górny wrote: > > > >> Make helper calls in global scope fatal in all supported EAPIs since > >> this

[gentoo-portage-dev] [PATCH] repoman: Make LIVEVCS.* checks fatal

2016-06-28 Thread Michał Górny
Make LIVEVCS.* checks fatal to prevent people from committing ebuilds using live eclasses instead of package.masking them afterwards by QA. --- repoman/pym/repoman/qa_data.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/repoman/pym/repoman/qa_data.py b/repoman/pym/repoman/qa_data.py index

Re: [gentoo-portage-dev] [PATCH] repoman: Make LIVEVCS.* checks fatal

2016-06-30 Thread Michał Górny
On Tue, 28 Jun 2016 12:40:07 -0700 Zac Medico wrote: > On 06/28/2016 11:30 AM, Michał Górny wrote: > > Make LIVEVCS.* checks fatal to prevent people from committing ebuilds > > using live eclasses instead of package.masking them afterwards by QA. > > --- > > repoman

Re: [gentoo-portage-dev] Leadership election

2016-10-11 Thread Michał Górny
On Tue, 11 Oct 2016 01:59:38 -0700 Zac Medico wrote: > On 10/11/2016 12:06 AM, Alexander Berntsen wrote: > > Right, voting. I vote for... Brian? Yeah. Brian! > > > > I vote for Brian too. Brian! :) ++Brian. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgor

[gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
Increate the default git sync-depth from 1 to 10, to ensure that gentoo-mirror git clones retain at least a single original signed commit. --- pym/portage/sync/modules/git/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pym/portage/sync/modules/git/__init__.py b/py

[gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
Run `git update-index --refresh` to force proper index recheck before running `git reset --merge` on a shallow pull. This fixes syncing on some filesystem configurations including overlayfs on squashfs. --- pym/portage/sync/modules/git/git.py | 21 - 1 file changed, 16 insertio

Re: [gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:22:51 -0700 Zac Medico wrote: > On Sun, Oct 30, 2016 at 1:19 PM, Zac Medico wrote: > > On 10/30/2016 12:08 PM, Michał Górny wrote: > >> Increate the default git sync-depth from 1 to 10, to ensure that > >> gentoo-mirror git clones retain

Re: [gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:39:53 -0700 Zac Medico wrote: > On 10/30/2016 12:23 PM, Michał Górny wrote: > > Run `git update-index --refresh` to force proper index recheck before > > running `git reset --merge` on a shallow pull. This fixes syncing on > > some filesystem con

[gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
Increate the default git sync-depth from 1 to 10, to ensure that gentoo-mirror git clones retain at least a single original signed commit. --- man/portage.5| 10 -- pym/portage/sync/modules/git/__init__.py | 2 +- 2 files changed, 9 insertions(+), 3 deletions(-

Re: [gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:53:05 -0700 Zac Medico wrote: > On 10/30/2016 01:44 PM, Michał Górny wrote: > > On Sun, 30 Oct 2016 13:39:53 -0700 > > Zac Medico wrote: > > > >> On 10/30/2016 12:23 PM, Michał Górny wrote: > >>> Run `git update-index --ref

Re: [gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 14:44:26 -0700 Zac Medico wrote: > On 10/30/2016 02:34 PM, Michał Górny wrote: > > Increate the default git sync-depth from 1 to 10, to ensure that > > s/Increate/Increase/ Fixed. > > gentoo-mirror git clones retain at least a single origi

Re: [gentoo-portage-dev] Re: [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-31 Thread Michał Górny
On Mon, 31 Oct 2016 07:16:24 + (UTC) Martin Vaeth wrote: > Michał Górny wrote: > > + if quiet: # -q needs to go first > > + update_index_cmd.append('-q') > > The options -q --unmerged (which are implicitly passed

Re: [gentoo-portage-dev] portage-2.3.2 stable request?

2016-11-04 Thread Michał Górny
emerge-webrsync now). Wouldn't it be better to have a really working version of gkeys before it's stabilized? Like one that could be used without having to create custom configuration files and/or run it as root? -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> pgpVR3redBmlG.pgp Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-05 Thread Michał Górny
git', 'prune'], > + cwd=portage._unicode_encode(self.repo.location)) > + > git_cmd_opts += " --depth %d" % self.repo.sync_depth > git_cmd = "%s fetch %s%s" % (self.bin_command, > remote_bran

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-05 Thread Michał Górny
On Sat, 5 Nov 2016 15:11:10 -0700 Zac Medico wrote: > On 11/05/2016 02:50 PM, Michał Górny wrote: > > On Sat, 5 Nov 2016 13:43:15 -0700 > > Zac Medico wrote: > > > >> This is necessary in order to avoid "There are too many unreachable > >> loose o

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-06 Thread Michał Górny
On Sat, 5 Nov 2016 15:56:20 -0700 Zac Medico wrote: > On 11/05/2016 03:22 PM, Michał Górny wrote: > > On Sat, 5 Nov 2016 15:11:10 -0700 > > Zac Medico wrote: > > > >> On 11/05/2016 02:50 PM, Michał Górny wrote: > >>> On Sat, 5 Nov 2

Re: [gentoo-portage-dev] [PATCH v2] parse_metadata_use: prefer first language found (bug 599060)

2016-11-06 Thread Michał Górny
continue > + > # emulate the Element.itertext() method from > python-2.7 > inner_text = [] > stack = [] This really doesn't fix the bug at hand. We should always prefer English descriptions, either thr

Re: [gentoo-portage-dev] Vote to revert all changes related to shallow git fetch from bug 552814

2016-11-07 Thread Michał Górny
oner it is done, the less likely it is that users have too many loose objects to not be able to clean up the mess automatically during periodic 'git gc'. -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> pgpqTwH99fREO.pgp Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] Run distcc-pump server throughout src_configure() to src_install()

2017-01-31 Thread Michał Górny
Dnia 2014-09-07, nie o godzinie 12:16 +0200, Michał Górny pisze: > Start distcc-pump server in all phases throughout src_configure() to > src_install() rather than in src_compile() alone. Since each of those > phases may involve some compilations, we should try to take full > advanta

Re: [gentoo-portage-dev] [PATCH] sys-apps/portage: add native-extensions USE flag (bug 571444)

2017-02-01 Thread Michał Górny
es=true" >> \ > + setup.cfg || die > + fi > + >   if ! use ipc ; then >   einfo "Disabling ipc..." >   sed -e "s:_enable_ipc_daemon = > True:_enable_ipc_daemon = False:" \ -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCH] sys-apps/portage: add native-extensions USE flag (bug 571444)

2017-02-04 Thread Michał Górny
W dniu 01.02.2017, śro o godzinie 20∶46 -0500, użytkownik Michael Orlitzky napisał: > On 02/01/2017 04:03 PM, Michał Górny wrote: > > > SLOT="0" > > > -IUSE="build doc epydoc +ipc linguas_ru selinux xattr" > > > +IUSE="build doc epydo

Re: [gentoo-portage-dev] [PATCH 08/18] Add tentative support for EAPI6 eapply_user function

2017-02-10 Thread Michał Górny
W dniu 09.02.2017, czw o godzinie 14∶39 -0800, użytkownik Zac Medico napisał: > On 12/01/2014 01:28 PM, Michał Górny wrote: > > Add support for the user patch applying function. > > --- > > bin/eapi.sh | 4 > > bin/phase-helpers.sh | 22

[gentoo-portage-dev] [PATCH 1/3] repoman: Sort deprecated eclasses

2017-02-22 Thread Michał Górny
--- repoman/pym/repoman/modules/scan/ebuild/checks.py | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py b/repoman/pym/repoman/modules/scan/ebuild/checks.py index 15e225156..286021524 100644 --- a/repoman/pym/repoman/mo

[gentoo-portage-dev] [PATCH 3/3] repoman: Deprecate confutils.eclass

2017-02-22 Thread Michał Górny
--- repoman/pym/repoman/modules/scan/ebuild/checks.py | 1 + 1 file changed, 1 insertion(+) diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py b/repoman/pym/repoman/modules/scan/ebuild/checks.py index d091a92b7..06afad5f7 100644 --- a/repoman/pym/repoman/modules/scan/ebuild/checks.py

[gentoo-portage-dev] [PATCH 2/3] repoman: Deprecate gpe.eclass

2017-02-22 Thread Michał Górny
--- repoman/pym/repoman/modules/scan/ebuild/checks.py | 1 + 1 file changed, 1 insertion(+) diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py b/repoman/pym/repoman/modules/scan/ebuild/checks.py index 286021524..d091a92b7 100644 --- a/repoman/pym/repoman/modules/scan/ebuild/checks.py

Re: [gentoo-portage-dev] [PATCH 3/3] repoman: Deprecate confutils.eclass

2017-02-22 Thread Michał Górny
W dniu 22.02.2017, śro o godzinie 01∶15 -0800, użytkownik Zac Medico napisał: > This whole series looks good. Pushed, thanks. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

[gentoo-portage-dev] [PATCH 2/3] checksum: Remove redundant internal fallbacks

2017-02-28 Thread Michał Górny
Remove the internal digest fallbacks since they have no real use nowadays. The hashlib versions are preferred later in the logic anyway, and they are available since Python 2.5. --- pym/portage/checksum.py | 19 ++- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/pym

[gentoo-portage-dev] [PATCH 1/3] checksum: Add summary on top to help following the logic

2017-02-28 Thread Michał Górny
Add a summary of all supported digest algorithms on top of the file along with the supported implementations and their order of preference. This will help people follow the crazy logic below. --- pym/portage/checksum.py | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --

[gentoo-portage-dev] [PATCHES] Little cleanup of checksum + BLAKE2*/SHA3

2017-02-28 Thread Michał Górny
x27;t able to find any other (fallback) Python provider of the latter at the moment, so it will take a while before we can start using it anywhere. -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 3/3] checksum: Add blake2* and sha3 hashes from hashlib 3.6+

2017-02-28 Thread Michał Górny
Add initial support for using the new SHA3_256 and SHA3_512, as well as competetive BLAKE2b and BLAKE2s hashes that are now provided in hashlib module of Python 3.6. --- pym/portage/checksum.py | 14 +- pym/portage/const.py| 6 ++ 2 files changed, 15 insertions(+), 5 deletions

Re: [gentoo-portage-dev] [PATCH 2/3] checksum: Remove redundant internal fallbacks

2017-02-28 Thread Michał Górny
W dniu 28.02.2017, wto o godzinie 10∶01 +0100, użytkownik Michał Górny napisał: > Remove the internal digest fallbacks since they have no real use > nowadays. The hashlib versions are preferred later in the logic anyway, > and they are available since Python 2.5. > --- > pym/port

[gentoo-portage-dev] [PATCHES] Fix md5 refs in vartree

2017-02-28 Thread Michał Górny
shlib directly. I've replaced the use of portage.checksum implementation detail with another use of hashlib, and removed the compatibility for Python < 2.5. I think it's a reasonable temporary measure until someone on friend terms with the code reworks it not to use MD5. -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 1/2] vartree: Use hashlib lazy-import, kill py<2.5 compat

2017-02-28 Thread Michał Górny
--- pym/portage/dbapi/vartree.py | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py index 5053801e6..2a612d162 100644 --- a/pym/portage/dbapi/vartree.py +++ b/pym/portage/dbapi/vartree.py @@ -1,4 +1,4 @@ -# Copyr

[gentoo-portage-dev] [PATCH 2/2] vartree: Replace uses of internal portage.checksum._new_md5 var

2017-02-28 Thread Michał Górny
Use the public hashlib.md5 method that is required for other code in the module already. --- pym/portage/dbapi/vartree.py | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py index 2a612d162..c421dc50b 100644 --- a/py

[gentoo-portage-dev] [PATCH 1/2] checksum: Fix overriding fallbacks on broken pycrypto

2017-02-28 Thread Michał Górny
The pycrypto override used the same variables as actual hash functions before determining whether its functions are useful. As a result, if pycrypto had a broken module and no hash function was generated, the possible previous implementation was replaced by None. --- pym/portage/checksum.py | 12 +

[gentoo-portage-dev] [PATCH 2/2] checksum: Add pycryptodome fallbacks for SHA3 and BLAKE2

2017-02-28 Thread Michał Górny
--- pym/portage/checksum.py | 22 ++ 1 file changed, 22 insertions(+) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index fc38417a7..042a0a745 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -116,6 +116,28 @@ try: except ImportError:

Re: [gentoo-portage-dev] [PATCHES] Fix md5 refs in vartree

2017-03-01 Thread Michał Górny
W dniu 28.02.2017, wto o godzinie 11∶03 -0800, użytkownik Zac Medico napisał: > On 02/28/2017 02:12 AM, Michał Górny wrote: > > Hi, > > > > Here's a prequel to my other patch set. It cleans up the use of MD5 > > in vartree. > > > > Currently, MD5 is l

Re: [gentoo-portage-dev] [PATCH 1/2] checksum: Fix overriding fallbacks on broken pycrypto

2017-03-01 Thread Michał Górny
W dniu 28.02.2017, wto o godzinie 23∶57 -0800, użytkownik Zac Medico napisał: > On 02/28/2017 11:34 PM, Michał Górny wrote: > > The pycrypto override used the same variables as actual hash functions > > before determining whether its functions are useful. As a result, if > >

Re: [gentoo-portage-dev] [PATCH] movefile: support in-kernel file copying on Linux (bug 607868)

2017-03-02 Thread Michał Górny
break; > +} > +} > +else { > +error = 1; > +break; > + } > +} > +} > +#else > +/* Less than Linux 3.1 does not support SEEK_DATA or copy_file_range, > + * so just use sendfile for in-kernel copy. This will fail for Linux > + * versions from 2.6.0 to 2.6.32, because sendfile does not support > + * writing to regular files. The caller should perform a runtime > + * test to verify that this function works with the running kernel. > + */ > +struct stat sb; > +int stat_acquired; > + > +if (!PyArg_ParseTuple(args, "ii", &fd_in, &fd_out)) > +return NULL; > + > +eintr_retry = 1; > +offset_out = 0; > +stat_acquired = 0; > + > +while (eintr_retry) { > + > +error = 0; > + > +Py_BEGIN_ALLOW_THREADS > + > +if (!stat_acquired && fstat(fd_in, &sb) < 0) { > +error = 1; > +} > +else { > +stat_acquired = 1; > +while (offset_out < sb.st_size) { > +copyfunc_ret = sendfile(fd_out, > +fd_in, > +&offset_out, > +sb.st_size - offset_out); > + > +if (copyfunc_ret < 0) { Likewise, especially that old kernels may refuse file-to-file copy here. > +error = 1; > +break; > +} > +} > +} > +#endif > +Py_END_ALLOW_THREADS > + > +if (!(error && errno == EINTR && PyErr_CheckSignals() == 0)) > +eintr_retry = 0; > +} > + > +if (error) > +return PyErr_SetFromErrno(PyExc_OSError); > + > +return Py_BuildValue("i", offset_out); > +} -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCH v2] movefile: support in-kernel file copying on Linux (bug 607868)

2017-03-03 Thread Michał Górny
/* Some bytes may still be buffered from the > + * previous iteration of the outer loop. > + */ > +if (!buf_bytes) { > +buf_offset = 0; > +buf_bytes = read(fd_in, buf, > stat_out.st_blksize); > + > +if (!buf_bytes) { > +/* EOF */ > +break; > +} > +else if (buf_bytes < 0) { > +error = 1; > +break; > +} > +} > + > +copyfunc_ret = write(fd_out, buf + buf_offset, > buf_bytes); > +if (copyfunc_ret < 0) { > +error = 1; > +break; > +} > + > +buf_bytes -= copyfunc_ret; > +buf_offset += copyfunc_ret; > +} > +} > +} > +} > + > +Py_END_ALLOW_THREADS > + > +if (!(error && errno == EINTR && PyErr_CheckSignals() == 0)) > +eintr_retry = 0; To be honest, I can't do it but it'd be really a good idea if someone could analyze all the code paths where the code above could give 'error = 1' and ensure that: a. there is no case when we error=1 and leave errno unmodified/unset (i.e. can accidentally leave earlier EINTR there), b. we can correctly recover from any EINTR, i.e. if EINTR causes some function to be interrupted in the middle, our next iteration starts safely. > +} > + > +free(buf); if (buf != NULL) ? > + > +if (error) > +return PyErr_SetFromErrno(PyExc_OSError); > + > +return Py_BuildValue("i", offset_out); > +} -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCH] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

2017-03-04 Thread Michał Górny
> +++ > 5 files changed, 223 insertions(+), 5 deletions(-) > create mode 100644 pym/portage/tests/resolver/test_bdeps.py Just to be clear, this will break stuff for people who have more than one slot of LLVM enabled since different slots build-depend on collidin

Re: [gentoo-portage-dev] [PATCH] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

2017-03-04 Thread Michał Górny
W dniu 04.03.2017, sob o godzinie 02∶30 -0800, użytkownik Zac Medico napisał: > On 03/04/2017 01:59 AM, Michał Górny wrote: > > W dniu 04.03.2017, sob o godzinie 01∶34 -0800, użytkownik Zac Medico > > napisał: > > > It's useful to enable --with-bdeps by default so t

[gentoo-portage-dev] [PATCHES] Checksums pt. 3 -- one more fallback for SHA3

2017-03-12 Thread Michał Górny
Hi, Just a quick patchset fixing fallback doc and adding pysha3 support. Next time, I'll probably replace the whole horrible fallback logic there with something cleaner (and lazy). -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 2/2] portage.checksum: Support pysha3 fallback for SHA3

2017-03-12 Thread Michał Górny
pysha3 provides a stand-alone FIPS-compliant SHA3 implementation that can be used as a fallback for Python < 3.6. --- pym/portage/checksum.py | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index ac11d3f4b..3e61a

[gentoo-portage-dev] [PATCH 1/2] portage.checksum: Update fallback doc for SHA3/BLAKE2*

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index 042a0a745..ac11d3f4b 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -24,10 +24,10 @@ import tempfile # SHA512: has

[gentoo-portage-dev] [PATCH 04/14] portage.checksum: Support getting byte string checksums

2017-03-12 Thread Michał Górny
Add a checksum_str() method to Portage hashes and a matching function to make it possible to compute checksums of arbitrary bytestrings rather than just files. --- pym/portage/checksum.py | 29 + 1 file changed, 29 insertions(+) diff --git a/pym/portage/checksum.py b/p

[gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-12 Thread Michał Górny
backs (for ripemd160, Whirlpool and SHA3). 6. Support for Streebog (GOST R 34.11-2012) hash algorithm, provided either by pygcrypt or pure Python pygost module. [TODO: find a better pure Python provider] -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 02/14] portage.checksum: Remove python-fchksum support

2017-03-12 Thread Michał Górny
Remove the support for MD5 implementation from python-fchksum package. The package is rarely installed, supports Python 2.7 only and the code handles checksumming a whole file only. --- pym/portage/checksum.py | 11 +-- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/pym/port

[gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
Use get_valid_checksum_keys() function instead of accessing hashfunc_map directly throughout the Portage. --- pym/portage/_emirrordist/FetchTask.py | 2 +- pym/portage/dbapi/bintree.py | 4 ++-- pym/portage/eclass_cache.py | 2 +- pym/portage/manifest.py | 4 ++

[gentoo-portage-dev] [PATCH 06/14] portage.checksum: Fix BLAKE2* fallbacks from pycryptodome

2017-03-12 Thread Michał Górny
Fix BLAKE2* fallback functions to explicitly provide digest length as required by pycryptodome. --- pym/portage/checksum.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index 9ba251f29..82e8bec00 100644 --- a/pym/portag

[gentoo-portage-dev] [PATCH 03/14] portage.checksum: create explicit checksum_file() method

2017-03-12 Thread Michał Górny
Make the file checksum generation code use an explicit checksum_file() method rather than implicit __call__. This should be more readable, and make it cleanly possible to add more methods. --- pym/portage/checksum.py | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git

[gentoo-portage-dev] [PATCH 07/14] portage.checksum: Remove fallbacks for algorithms guaranteed since py2.7

2017-03-12 Thread Michał Górny
The MD5, SHA1 and SHA2 algorithms are guaranteed to be available in hashlib for Python 2.7 and newer, making the fallbacks to other implementations meaningless. Remove them to simplify the code. --- pym/portage/checksum.py | 18 +- 1 file changed, 5 insertions(+), 13 deletions(-)

[gentoo-portage-dev] [PATCH 08/14] portage.checksum: Remove exception handling for missing hashlib

2017-03-12 Thread Michał Górny
Remove the try-except block for potential ImportError of hashlib. The hashlib module should be available in all supported Python versions, and we do not really test or support the case when it is not available. --- pym/portage/checksum.py | 55 ++--- 1 f

[gentoo-portage-dev] [PATCH 05/14] tests: Add minimal safety checks for checksums

2017-03-12 Thread Michał Górny
--- pym/portage/tests/util/test_checksum.py | 88 + 1 file changed, 88 insertions(+) create mode 100644 pym/portage/tests/util/test_checksum.py diff --git a/pym/portage/tests/util/test_checksum.py b/pym/portage/tests/util/test_checksum.py new file mode 100644 ind

[gentoo-portage-dev] [PATCH 09/14] portage.checksum: Stop exposing global hash variables

2017-03-12 Thread Michał Górny
Stop exposing global variables such as 'md5hash'. Those are not used anywhere anymore, exposing them makes the code more complex and makes it easy to accidentally fail to set them properly (this happened already for SHA3). Instead, rely on them being inserted into hashfunc_map. --- pym/portage/che

[gentoo-portage-dev] [PATCH 11/14] portage.checksum: Support pygcrypt as optimized fallback

2017-03-12 Thread Michał Górny
pygcrypt uses libgcrypt which provides support for ripemd160, whirlpool, SHA3, plus some algorithms not provided by any other library. --- pym/portage/checksum.py | 38 ++ 1 file changed, 38 insertions(+) diff --git a/pym/portage/checksum.py b/pym/portage/check

[gentoo-portage-dev] [PATCH 12/14] Support STREEBOG{256,512} hash function (from pygcrypt), #597736

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 5 - pym/portage/const.py| 3 ++- pym/portage/tests/util/test_checksum.py | 18 ++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index 92b41b1

[gentoo-portage-dev] [PATCH 14/14] travis: Install additional (pure Python) crypto providers if necessary

2017-03-12 Thread Michał Górny
Install pysha3 to provide the SHA3 routines on Python < 3.6, and install pygost to provide a Streebog implementation. --- .travis.yml | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 7fe3c06aa..196e3520a 100644 --- a/.travis.yml +++ b/.tra

[gentoo-portage-dev] [PATCH 10/14] portage.checksum: Reorder to avoid loading redundant impls

2017-03-12 Thread Michał Górny
Reorder the checksum implementations to start with the most preferred implementation, and try fallbacks only if the most preferred implementation is not available. Most importantly, this means that Portage will no longer attempt to load all hash libraries in the system, especially when it can satis

[gentoo-portage-dev] [PATCH 13/14] portage.checksum: Support pygost as fallback Streebog provider

2017-03-12 Thread Michał Górny
Support the pure Python implementation of Streebog in pygost as a fallback algorithm. The code is horrible (it stores all the data in memory before hashing), so it is really intended as last fallback. --- pym/portage/checksum.py | 15 +++ 1 file changed, 15 insertions(+) diff --git a/

Re: [gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
W dniu 12.03.2017, nie o godzinie 13∶36 -0700, użytkownik Zac Medico napisał: > On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: > > Use get_valid_checksum_keys() function instead of accessing hashfunc_map > > directly throughout the Portage. > > --- > >

[gentoo-portage-dev] [PATCH] portage.checksum: Store supported hash types in a frozenset

2017-03-13 Thread Michał Górny
Copy the list of supported hash types (hashfunc_map dict) into a frozenset to support efficient access in the public API. --- pym/portage/checksum.py | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py in

Re: [gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-13 Thread Michał Górny
W dniu 12.03.2017, nie o godzinie 21∶31 -0700, użytkownik Zac Medico napisał: > On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: > > Hi, > > > > Here's a huge batch of patches for portage.checksum and relevant stuff. > > It's not a complete rewrite a

[gentoo-portage-dev] [PATCH] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-16 Thread Michał Górny
Use a model of fake FILESDIR path to ensure that invalid accesses to FILESDIR will result in failures rather than being silently allowed by Portage. This mostly involves accesses in the global scope and pkg_* phases, although the current model does not cover the latter completely (i.e. does not gua

Re: [gentoo-portage-dev] [PATCH] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-16 Thread Michał Górny
On czw, 2017-03-16 at 21:29 +0100, Ulrich Mueller wrote: > > > > > > On Thu, 16 Mar 2017, Michał Górny wrote: > > + mysettings["FILESDIR"] = os.path.join(settings["PORTAGE_BUILDDIR"], > > "files") > > I believe that this contr

[gentoo-portage-dev] [PATCH v2] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-18 Thread Michał Górny
Use a model of fake FILESDIR path to ensure that invalid accesses to FILESDIR will result in failures rather than being silently allowed by Portage. This mostly involves accesses in the global scope and pkg_* phases, although the current model does not cover the latter completely (i.e. does not gua

[gentoo-portage-dev] [PATCHES] Restrict DISTDIR to src_unpack and further

2017-03-18 Thread Michał Górny
hadow directory. The directory itself is not created until src_unpack, so mistaken accesses in earlier phases are caught early. The accesses in pkg_pre/post phases are not caught yet. As usual, this patch can be applied after reasonable set of blockers in ::gentoo is resolved. -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 4/4] _emerge.Ebuild*: delay creating DISTDIR shadow until src_unpack

2017-03-18 Thread Michał Górny
--- pym/_emerge/EbuildExecuter.py | 4 pym/_emerge/EbuildPhase.py| 4 +++- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/pym/_emerge/EbuildExecuter.py b/pym/_emerge/EbuildExecuter.py index 7a454c3e6..81257aa9e 100644 --- a/pym/_emerge/EbuildExecuter.py +++ b/pym/_emerge/E

[gentoo-portage-dev] [PATCH 3/4] portage.package.ebuild.doebuild: Override DISTDIR unconditionally, #612972

2017-03-18 Thread Michał Górny
Ensure that DISTDIR is always defined to the path to the shadow directory. This ensures that PMS rules for consistent value are followed, and that no global scope calls should be able to access the distfile directory. This also ensures that global-scope assignments (e.g. in PATCHES) do not work aro

  1   2   3   4   5   6   7   8   9   10   >