[gentoo-portage-dev] [PATCH 1/3] checksum: Add summary on top to help following the logic

2017-02-28 Thread Michał Górny
Add a summary of all supported digest algorithms on top of the file along with the supported implementations and their order of preference. This will help people follow the crazy logic below. --- pym/portage/checksum.py | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff

[gentoo-portage-dev] [PATCH 2/3] checksum: Remove redundant internal fallbacks

2017-02-28 Thread Michał Górny
Remove the internal digest fallbacks since they have no real use nowadays. The hashlib versions are preferred later in the logic anyway, and they are available since Python 2.5. --- pym/portage/checksum.py | 19 ++- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git

[gentoo-portage-dev] [PATCH 1/2] checksum: Fix overriding fallbacks on broken pycrypto

2017-02-28 Thread Michał Górny
The pycrypto override used the same variables as actual hash functions before determining whether its functions are useful. As a result, if pycrypto had a broken module and no hash function was generated, the possible previous implementation was replaced by None. --- pym/portage/checksum.py | 12

[gentoo-portage-dev] [PATCH 2/2] checksum: Add pycryptodome fallbacks for SHA3 and BLAKE2

2017-02-28 Thread Michał Górny
--- pym/portage/checksum.py | 22 ++ 1 file changed, 22 insertions(+) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index fc38417a7..042a0a745 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -116,6 +116,28 @@ try: except ImportError:

Re: [gentoo-portage-dev] [PATCHES] Fix md5 refs in vartree

2017-03-01 Thread Michał Górny
W dniu 28.02.2017, wto o godzinie 11∶03 -0800, użytkownik Zac Medico napisał: > On 02/28/2017 02:12 AM, Michał Górny wrote: > > Hi, > > > > Here's a prequel to my other patch set. It cleans up the use of MD5 > > in vartree. > > > > Currently, MD5 is loaded

Re: [gentoo-portage-dev] [PATCH 1/2] checksum: Fix overriding fallbacks on broken pycrypto

2017-03-01 Thread Michał Górny
W dniu 28.02.2017, wto o godzinie 23∶57 -0800, użytkownik Zac Medico napisał: > On 02/28/2017 11:34 PM, Michał Górny wrote: > > The pycrypto override used the same variables as actual hash functions > > before determining whether its functions are useful. As a result, if > >

Re: [gentoo-portage-dev] portage-2.3.2 stable request?

2016-11-04 Thread Michał Górny
rypt/gkeys since it's needed by emerge-webrsync now). Wouldn't it be better to have a really working version of gkeys before it's stabilized? Like one that could be used without having to create custom configuration files and/or run it as root? -- Best regards, Michał Górny <http://dev.gentoo.org/~mgorny/> pgpVR3redBmlG.pgp Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-06 Thread Michał Górny
On Sat, 5 Nov 2016 15:56:20 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 11/05/2016 03:22 PM, Michał Górny wrote: > > On Sat, 5 Nov 2016 15:11:10 -0700 > > Zac Medico <zmed...@gentoo.org> wrote: > > > >> On 11/05/2016 02:50 PM, Michał Górny w

Re: [gentoo-portage-dev] [PATCH v2] parse_metadata_use: prefer first language found (bug 599060)

2016-11-06 Thread Michał Górny
continue > + > # emulate the Element.itertext() method from > python-2.7 > inner_text = [] > stack = [] This really doesn't fix the bug at hand. We should always pref

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-05 Thread Michał Górny
'git', 'prune'], > + cwd=portage._unicode_encode(self.repo.location)) > + > git_cmd_opts += " --depth %d" % self.repo.sync_depth > git_cmd = "%s fetch %s%s" % (self.bin_command, > remo

Re: [gentoo-portage-dev] [PATCH] sync: call git prune before shallow fetch (bug 599008)

2016-11-05 Thread Michał Górny
On Sat, 5 Nov 2016 15:11:10 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 11/05/2016 02:50 PM, Michał Górny wrote: > > On Sat, 5 Nov 2016 13:43:15 -0700 > > Zac Medico <zmed...@gentoo.org> wrote: > > > >> This is necessary in order to avoid

Re: [gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:39:53 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 10/30/2016 12:23 PM, Michał Górny wrote: > > Run `git update-index --refresh` to force proper index recheck before > > running `git reset --merge` on a shallow pull. This fixes syncing o

[gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
Run `git update-index --refresh` to force proper index recheck before running `git reset --merge` on a shallow pull. This fixes syncing on some filesystem configurations including overlayfs on squashfs. --- pym/portage/sync/modules/git/git.py | 21 - 1 file changed, 16

Re: [gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 14:44:26 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 10/30/2016 02:34 PM, Michał Górny wrote: > > Increate the default git sync-depth from 1 to 10, to ensure that > > s/Increate/Increase/ Fixed. > > gentoo-mirror git clones retain at lea

[gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
Increate the default git sync-depth from 1 to 10, to ensure that gentoo-mirror git clones retain at least a single original signed commit. --- pym/portage/sync/modules/git/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pym/portage/sync/modules/git/__init__.py

Re: [gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:22:51 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On Sun, Oct 30, 2016 at 1:19 PM, Zac Medico <zmed...@gentoo.org> wrote: > > On 10/30/2016 12:08 PM, Michał Górny wrote: > >> Increate the default git sync-depth from 1 to 10, to ensure that

[gentoo-portage-dev] [PATCH] [sync] Increase the default git sync-depth to 10

2016-10-30 Thread Michał Górny
Increate the default git sync-depth from 1 to 10, to ensure that gentoo-mirror git clones retain at least a single original signed commit. --- man/portage.5| 10 -- pym/portage/sync/modules/git/__init__.py | 2 +- 2 files changed, 9 insertions(+), 3

Re: [gentoo-portage-dev] [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-30 Thread Michał Górny
On Sun, 30 Oct 2016 13:53:05 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 10/30/2016 01:44 PM, Michał Górny wrote: > > On Sun, 30 Oct 2016 13:39:53 -0700 > > Zac Medico <zmed...@gentoo.org> wrote: > > > >> On 10/30/2016 12:23 PM, Michał Górny wrot

Re: [gentoo-portage-dev] Re: [PATCH] [sync] Run `git update-index --refresh` when doing shallow pulls

2016-10-31 Thread Michał Górny
On Mon, 31 Oct 2016 07:16:24 + (UTC) Martin Vaeth <mar...@mvath.de> wrote: > Michał Górny <mgo...@gentoo.org> wrote: > > + if quiet: # -q needs to go first > > + update_index_cmd.append('-q') > > The options -q

Re: [gentoo-portage-dev] Leadership election

2016-10-11 Thread Michał Górny
On Tue, 11 Oct 2016 01:59:38 -0700 Zac Medico <zmed...@gentoo.org> wrote: > On 10/11/2016 12:06 AM, Alexander Berntsen wrote: > > Right, voting. I vote for... Brian? Yeah. Brian! > > > > I vote for Brian too. Brian! :) ++Brian. -- Best regards, Michał Górny <

Re: [gentoo-portage-dev] [PATCH] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-24 Thread Michał Górny
On czw, 2017-03-16 at 20:56 +0100, Michał Górny wrote: > Use a model of fake FILESDIR path to ensure that invalid accesses to > FILESDIR will result in failures rather than being silently allowed by > Portage. This mostly involves accesses in the global scope and pkg_* > phas

[gentoo-portage-dev] [RFC] eslurp helper

2017-03-26 Thread Michał Górny
listed packages in PKGDIR (respecting subdirectories, updating index). If neither of the options were specified, it would guess the directory type by the filename suffix. What do you think? -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

[gentoo-portage-dev] [PATCH] phase-functions: Fix cleaning fake FILESDIR, reported by Arfrever

2017-03-26 Thread Michał Górny
--- bin/phase-functions.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh index 50f0fdb9b..dfd8733c8 100644 --- a/bin/phase-functions.sh +++ b/bin/phase-functions.sh @@ -283,6 +283,7 @@ __dyn_clean() { rm -rf

Re: [gentoo-portage-dev] [PATCH] phase-functions: Fix cleaning fake FILESDIR, reported by Arfrever

2017-03-26 Thread Michał Górny
On nie, 2017-03-26 at 00:38 -0700, Zac Medico wrote: > On Sun, Mar 26, 2017 at 12:30 AM, Michał Górny <mgo...@gentoo.org> wrote: > > --- > > bin/phase-functions.sh | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/bin/phase-functions.sh b/bin

[gentoo-portage-dev] [PATCH v2] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-18 Thread Michał Górny
Use a model of fake FILESDIR path to ensure that invalid accesses to FILESDIR will result in failures rather than being silently allowed by Portage. This mostly involves accesses in the global scope and pkg_* phases, although the current model does not cover the latter completely (i.e. does not

[gentoo-portage-dev] [PATCH 4/4] _emerge.Ebuild*: delay creating DISTDIR shadow until src_unpack

2017-03-18 Thread Michał Górny
--- pym/_emerge/EbuildExecuter.py | 4 pym/_emerge/EbuildPhase.py| 4 +++- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/pym/_emerge/EbuildExecuter.py b/pym/_emerge/EbuildExecuter.py index 7a454c3e6..81257aa9e 100644 --- a/pym/_emerge/EbuildExecuter.py +++

[gentoo-portage-dev] [PATCH 3/4] portage.package.ebuild.doebuild: Override DISTDIR unconditionally, #612972

2017-03-18 Thread Michał Górny
Ensure that DISTDIR is always defined to the path to the shadow directory. This ensures that PMS rules for consistent value are followed, and that no global scope calls should be able to access the distfile directory. This also ensures that global-scope assignments (e.g. in PATCHES) do not work

[gentoo-portage-dev] [PATCH 1/4] portage.package.ebuild: Move _prepare_fake_distdir to .prepare_build_dirs

2017-03-18 Thread Michał Górny
--- pym/_emerge/EbuildExecuter.py| 2 +- pym/portage/package/ebuild/doebuild.py | 32 +--- pym/portage/package/ebuild/prepare_build_dirs.py | 31 +++ 3 files changed, 33 insertions(+), 32 deletions(-) diff --git

[gentoo-portage-dev] [PATCH 2/4] portage.package.ebuild.fetch: Use PORTAGE_ACTUAL_DISTDIR for fetching

2017-03-18 Thread Michał Górny
--- pym/portage/package/ebuild/fetch.py | 45 - 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/pym/portage/package/ebuild/fetch.py b/pym/portage/package/ebuild/fetch.py index 0431e11ea..90b6f2e54 100644 ---

[gentoo-portage-dev] [PATCHES] Restrict DISTDIR to src_unpack and further

2017-03-18 Thread Michał Górny
directory. The directory itself is not created until src_unpack, so mistaken accesses in earlier phases are caught early. The accesses in pkg_pre/post phases are not caught yet. As usual, this patch can be applied after reasonable set of blockers in ::gentoo is resolved. -- Best regards, Michał Górny

Re: [gentoo-portage-dev] [PATCH 3/4] portage.package.ebuild.doebuild: Override DISTDIR unconditionally, #612972

2017-03-18 Thread Michał Górny
On sob, 2017-03-18 at 13:45 -0700, Zac Medico wrote: > On Sat, Mar 18, 2017 at 1:12 PM, Zac Medico <zmed...@gentoo.org> wrote: > > On Sat, Mar 18, 2017 at 12:57 PM, Zac Medico <zmed...@gentoo.org> wrote: > > > On Sat, Mar 18, 2017 at 12:04 PM, Michał Górny <mgo...@

[gentoo-portage-dev] [PATCH v3] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-21 Thread Michał Górny
Use a model of fake FILESDIR path to ensure that invalid accesses to FILESDIR will result in failures rather than being silently allowed by Portage. This mostly involves accesses in the global scope and pkg_* phases, although the current model does not cover the latter completely (i.e. does not

Re: [gentoo-portage-dev] [PATCH] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-16 Thread Michał Górny
On czw, 2017-03-16 at 21:29 +0100, Ulrich Mueller wrote: > > > > > > On Thu, 16 Mar 2017, Michał Górny wrote: > > + mysettings["FILESDIR"] = os.path.join(settings["PORTAGE_BUILDDIR"], > > "files") > > I believe that this

[gentoo-portage-dev] [PATCH] portage.package.ebuild: Use a fake FILESDIR to catch invalid accesses

2017-03-16 Thread Michał Górny
Use a model of fake FILESDIR path to ensure that invalid accesses to FILESDIR will result in failures rather than being silently allowed by Portage. This mostly involves accesses in the global scope and pkg_* phases, although the current model does not cover the latter completely (i.e. does not

[gentoo-portage-dev] [PATCH] repoman: Add a check for relative dosym candidates

2017-04-15 Thread Michał Górny
Add a check for dosym with target path matching absolute paths controlled by the package manager, e.g. /bin, /etc... Example output: ebuild.absdosym 5 app-editors/nano/nano-2.5.3.ebuild: dosym '/bin/nano'... could use relative path on line: 81 --- repoman/man/repoman.1

[gentoo-portage-dev] [PATCH] dosym: protect Prefix hack from double EPREFIX

2017-04-19 Thread Michał Górny
Add an additional conditional to the dosym Prefix hack to ensure that the symlink is not using double Prefix when the ebuild uses ${EPREFIX} explicitly. This ensures that Portage on Prefix systems is both compatible with the ebuilds relying on the hack, and using dosym in the PMS-defined manner.

[gentoo-portage-dev] [PATCH v3] dosym: protect Prefix hack from double EPREFIX

2017-04-20 Thread Michał Górny
Add an additional conditional to the dosym Prefix hack to ensure that the symlink is not using double Prefix when the ebuild uses ${EPREFIX} explicitly. This ensures that Portage on Prefix systems is both compatible with the ebuilds relying on the hack, and using dosym in the PMS-defined manner.

Re: [gentoo-portage-dev] [PATCH] repoman: Add a check for relative dosym candidates

2017-04-15 Thread Michał Górny
On sob, 2017-04-15 at 10:11 -0700, Zac Medico wrote: > On Sat, Apr 15, 2017 at 4:04 AM, Michał Górny <mgo...@gentoo.org> wrote: > > diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py > > b/repoman/pym/repoman/modules/scan/ebuild/checks.py > > index

Re: [gentoo-portage-dev] [PATCH] dosym: protect Prefix hack from double EPREFIX

2017-04-19 Thread Michał Górny
On śro, 2017-04-19 at 18:06 +0200, Ulrich Mueller wrote: > > > > > > On Wed, 19 Apr 2017, Michał Górny wrote: > > Add an additional conditional to the dosym Prefix hack to ensure that > > the symlink is not using double Prefix when the ebuild uses ${EPREFIX

Re: [gentoo-portage-dev] [PATCH v2] movefile: support in-kernel file copying on Linux (bug 607868)

2017-03-03 Thread Michał Górny
else { > +while (1) { > +/* Some bytes may still be buffered from the > + * previous iteration of the outer loop. > + */ > +if (!buf_bytes) { > +

Re: [gentoo-portage-dev] [PATCH] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

2017-03-04 Thread Michał Górny
+++ > 5 files changed, 223 insertions(+), 5 deletions(-) > create mode 100644 pym/portage/tests/resolver/test_bdeps.py Just to be clear, this will break stuff for people who have more than one slot of LLVM enabled since different slots build-depend on colliding pac

Re: [gentoo-portage-dev] [PATCH] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

2017-03-04 Thread Michał Górny
W dniu 04.03.2017, sob o godzinie 02∶30 -0800, użytkownik Zac Medico napisał: > On 03/04/2017 01:59 AM, Michał Górny wrote: > > W dniu 04.03.2017, sob o godzinie 01∶34 -0800, użytkownik Zac Medico > > napisał: > > > It's useful to enable --with-bdeps by defaul

[gentoo-portage-dev] [PATCH] portage.checksum: Store supported hash types in a frozenset

2017-03-13 Thread Michał Górny
Copy the list of supported hash types (hashfunc_map dict) into a frozenset to support efficient access in the public API. --- pym/portage/checksum.py | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py

[gentoo-portage-dev] [PATCH 2/2] portage.checksum: Support pysha3 fallback for SHA3

2017-03-12 Thread Michał Górny
pysha3 provides a stand-alone FIPS-compliant SHA3 implementation that can be used as a fallback for Python < 3.6. --- pym/portage/checksum.py | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index

[gentoo-portage-dev] [PATCH 1/2] portage.checksum: Update fallback doc for SHA3/BLAKE2*

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index 042a0a745..ac11d3f4b 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -24,10 +24,10 @@ import tempfile # SHA512:

[gentoo-portage-dev] [PATCH 10/14] portage.checksum: Reorder to avoid loading redundant impls

2017-03-12 Thread Michał Górny
Reorder the checksum implementations to start with the most preferred implementation, and try fallbacks only if the most preferred implementation is not available. Most importantly, this means that Portage will no longer attempt to load all hash libraries in the system, especially when it can

[gentoo-portage-dev] [PATCH 11/14] portage.checksum: Support pygcrypt as optimized fallback

2017-03-12 Thread Michał Górny
pygcrypt uses libgcrypt which provides support for ripemd160, whirlpool, SHA3, plus some algorithms not provided by any other library. --- pym/portage/checksum.py | 38 ++ 1 file changed, 38 insertions(+) diff --git a/pym/portage/checksum.py

[gentoo-portage-dev] [PATCH 12/14] Support STREEBOG{256,512} hash function (from pygcrypt), #597736

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 5 - pym/portage/const.py| 3 ++- pym/portage/tests/util/test_checksum.py | 18 ++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index

[gentoo-portage-dev] [PATCH 14/14] travis: Install additional (pure Python) crypto providers if necessary

2017-03-12 Thread Michał Górny
Install pysha3 to provide the SHA3 routines on Python < 3.6, and install pygost to provide a Streebog implementation. --- .travis.yml | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 7fe3c06aa..196e3520a 100644 --- a/.travis.yml +++

[gentoo-portage-dev] [PATCH 13/14] portage.checksum: Support pygost as fallback Streebog provider

2017-03-12 Thread Michał Górny
Support the pure Python implementation of Streebog in pygost as a fallback algorithm. The code is horrible (it stores all the data in memory before hashing), so it is really intended as last fallback. --- pym/portage/checksum.py | 15 +++ 1 file changed, 15 insertions(+) diff --git

[gentoo-portage-dev] [PATCH 05/14] tests: Add minimal safety checks for checksums

2017-03-12 Thread Michał Górny
--- pym/portage/tests/util/test_checksum.py | 88 + 1 file changed, 88 insertions(+) create mode 100644 pym/portage/tests/util/test_checksum.py diff --git a/pym/portage/tests/util/test_checksum.py b/pym/portage/tests/util/test_checksum.py new file mode 100644

[gentoo-portage-dev] [PATCH 03/14] portage.checksum: create explicit checksum_file() method

2017-03-12 Thread Michał Górny
Make the file checksum generation code use an explicit checksum_file() method rather than implicit __call__. This should be more readable, and make it cleanly possible to add more methods. --- pym/portage/checksum.py | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff

[gentoo-portage-dev] [PATCH 08/14] portage.checksum: Remove exception handling for missing hashlib

2017-03-12 Thread Michał Górny
Remove the try-except block for potential ImportError of hashlib. The hashlib module should be available in all supported Python versions, and we do not really test or support the case when it is not available. --- pym/portage/checksum.py | 55 ++--- 1

Re: [gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
W dniu 12.03.2017, nie o godzinie 13∶36 -0700, użytkownik Zac Medico napisał: > On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny <mgo...@gentoo.org> wrote: > > Use get_valid_checksum_keys() function instead of accessing hashfunc_map > > directly throughout the Portage. >

[gentoo-portage-dev] [PATCH 04/14] portage.checksum: Support getting byte string checksums

2017-03-12 Thread Michał Górny
Add a checksum_str() method to Portage hashes and a matching function to make it possible to compute checksums of arbitrary bytestrings rather than just files. --- pym/portage/checksum.py | 29 + 1 file changed, 29 insertions(+) diff --git a/pym/portage/checksum.py

[gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-12 Thread Michał Górny
, Whirlpool and SHA3). 6. Support for Streebog (GOST R 34.11-2012) hash algorithm, provided either by pygcrypt or pure Python pygost module. [TODO: find a better pure Python provider] -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH 02/14] portage.checksum: Remove python-fchksum support

2017-03-12 Thread Michał Górny
Remove the support for MD5 implementation from python-fchksum package. The package is rarely installed, supports Python 2.7 only and the code handles checksumming a whole file only. --- pym/portage/checksum.py | 11 +-- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git

[gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
Use get_valid_checksum_keys() function instead of accessing hashfunc_map directly throughout the Portage. --- pym/portage/_emirrordist/FetchTask.py | 2 +- pym/portage/dbapi/bintree.py | 4 ++-- pym/portage/eclass_cache.py | 2 +- pym/portage/manifest.py | 4

Re: [gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-13 Thread Michał Górny
W dniu 12.03.2017, nie o godzinie 21∶31 -0700, użytkownik Zac Medico napisał: > On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny <mgo...@gentoo.org> wrote: > > Hi, > > > > Here's a huge batch of patches for portage.checksum and relevant stuff. > > It's not a comp

Re: [gentoo-portage-dev] [PATCH] make.globals: Enable FEATURES=multilib-strict by default

2017-07-31 Thread Michał Górny
Dnia 31 lipca 2017 10:17:56 CEST, Zac Medico <zmed...@gentoo.org> napisał(a): >On Wed, Jul 26, 2017 at 1:44 PM, Michał Górny <mgo...@gentoo.org> >wrote: >> On śro, 2017-07-26 at 11:29 -0700, Zac Medico wrote: >>> You might think that, but I just check

Re: [gentoo-portage-dev] [PATCH v2] Support different compressors for binary packages

2017-07-29 Thread Michał Górny
we have already: PORTAGE_COMPRESS PORTAGE_COMPRESS_FLAGS ^^ and you've added: BINPKG_COMPRESSION ^^^ BINPKG_COMPRESSION_ARGS Wouldn't it be better to at least try having consistent variable naming? -- Best regards, Michał Górny signature.

Re: [gentoo-portage-dev] [PATCH] make.globals: Enable FEATURES=multilib-strict by default

2017-08-02 Thread Michał Górny
On śro, 2017-08-02 at 00:30 -0700, Zac Medico wrote: > On Wed, Jul 26, 2017 at 12:20 AM, Michał Górny <mgo...@gentoo.org> wrote: > > Enable the multilib-strict feature necessary for Portage to detect > > ebuilds not respecting libdir for libraries. Since those issues were &g

Re: [gentoo-portage-dev] [PATCH] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-03 Thread Michał Górny
On czw, 2017-08-03 at 11:18 -0400, Alec Warner wrote: > On Thu, Aug 3, 2017 at 10:18 AM, Michał Górny <mgo...@gentoo.org> wrote: > > > Support two new options: --bug (-b) and --closes (-c) to add a plain > > 'Bug' reference and a 'Closes' footer for a GitHub pull reque

[gentoo-portage-dev] [PATCH] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-03 Thread Michał Górny
Support two new options: --bug (-b) and --closes (-c) to add a plain 'Bug' reference and a 'Closes' footer for a GitHub pull request. Both options can be specified multiple times, resulting in multiple footer tags being written. The --bug option accepts either a Gentoo Bugzilla bug number or an

[gentoo-portage-dev] [PATCH] repoman commit: Merge code generating common part of the footer (DCO)

2017-08-03 Thread Michał Górny
The DCO Signed-off-by footer looks the same on the git branch, and on the branch for other VCS-es. Therefore, move the code generating it above the split branches. This also prepares the code for further footer elements being added. --- repoman/pym/repoman/actions.py | 13 +++-- 1 file

[gentoo-portage-dev] [PATCH] elog mod_echo: Print log path if PORT_LOGDIR is used

2017-08-16 Thread Michał Górny
Include the path to the log file if PORT_LOGDIR is being used (and therefore the log is going to be preserved past the build). This is useful when elog messages contain QA warnings or other errors that require reporting a bug. In such case, having a path to the log is handy. After this commit,

[gentoo-portage-dev] [PATCH 3/2] Update caches after reporting missing cache updates

2017-08-16 Thread Michał Górny
Call the appropriate updater to update caches after reporting a missing cache update, in order to stop repeating the same issue on subsequent packages that did not install any relevant files. --- bin/postinst-qa-check.d/50gnome2-utils | 8 bin/postinst-qa-check.d/50xdg-utils| 16

[gentoo-portage-dev] [PATCH 1/2] Support post-postinst QA checks

2017-08-15 Thread Michał Górny
Extend the QA check mechanics in Portage to support post-postinst QA checks. They are like post-install QA checks, except they are run after pkg_postinst(), and so they can be used to verify that necessary postinst actions were performed (e.g. regenerating caches). --- bin/misc-functions.sh

[gentoo-portage-dev] [PATCH 2/2] Add post-postinst checks for a few missed cache updates

2017-08-15 Thread Michał Górny
Add postinst-qa-check.d checks for missed desktop, mime-info and GTK+ icon cache updates. In all of the cases the checks simply look for any installed files that are newer than the cache. This check has some limitations: it assumes that mtime is not preserved when copying files to D, it can't

Re: [gentoo-portage-dev] [PATCH] elog mod_echo: Print log path if PORT_LOGDIR is used

2017-08-17 Thread Michał Górny
W dniu śro, 16.08.2017 o godzinie 09∶12 -0700, użytkownik Brian Dolbec napisał: > On Wed, 16 Aug 2017 12:10:52 +0200 > Michał Górny <mgo...@gentoo.org> wrote: > > > Include the path to the log file if PORT_LOGDIR is being used > > (and therefore the log is going to

Re: [gentoo-portage-dev] Leader election

2017-07-09 Thread Michał Górny
der > > > > > > > > > > > > > I've picked Sunday, July 2 at 4:00 PM UTC > > > > > > > We've decided (the members in attendance), to do the lead election via > > > email. > > > So, nominations are open from now to July 5, 2017. > > > Voting will be closed July 10, 2017, results posted here again. > > > - -- > > > Brian Dolbec > > > > > > I nominate Zac > > Thank you, I accept. > > I also nominate Brian Dolbec and Alexander Berntsen. My vote goes for Zac as well. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCH] doman: QA-complain on compressed files

2017-07-09 Thread Michał Górny
On nie, 2017-07-02 at 11:57 -0700, Brian Dolbec wrote: > On Sun, 2 Jul 2017 19:57:57 +0200 > Michał Górny <mgo...@gentoo.org> wrote: > > > Passing compressed files to doman is not permitted by the PMS. > > However, Portage permitting it results in many develope

[gentoo-portage-dev] [PATCH] make.globals: Enable FEATURES=multilib-strict by default

2017-07-26 Thread Michał Górny
Enable the multilib-strict feature necessary for Portage to detect ebuilds not respecting libdir for libraries. Since those issues were reliably fixed over the years and we're nearing the removal of 'lib' symlink, I think we can safely enable it by default for our users without risking many

Re: [gentoo-portage-dev] [PATCH] make.globals: Enable FEATURES=multilib-strict by default

2017-07-26 Thread Michał Górny
On śro, 2017-07-26 at 09:58 -0700, Zac Medico wrote: > On Wed, Jul 26, 2017 at 12:20 AM, Michał Górny <mgo...@gentoo.org> wrote: > > Enable the multilib-strict feature necessary for Portage to detect > > ebuilds not respecting libdir for libraries. Since those issues were &g

Re: [gentoo-portage-dev] [PATCH] make.globals: Enable FEATURES=multilib-strict by default

2017-07-26 Thread Michał Górny
On śro, 2017-07-26 at 11:29 -0700, Zac Medico wrote: > On Wed, Jul 26, 2017 at 10:11 AM, Michał Górny <mgo...@gentoo.org> wrote: > > On śro, 2017-07-26 at 09:58 -0700, Zac Medico wrote: > > > On Wed, Jul 26, 2017 at 12:20 AM, Michał Górny <mgo...@gentoo.org> wrot

Re: [gentoo-portage-dev] [PATCH] Support different (de)compressors for binary packages

2017-07-05 Thread Michał Górny
t to something a standard install doesn't include. > > Assume all binpkgs are set to be compressed with zstd, the patch makes > sure an app-arch/zstd binpkg will still be compressed with bzip2. And app-arch/bzip2 binpackage? In other words, that's the wrong approach. If I set binpackages to use X, I want them to use X, not 'use X if someone thought it correct to use X'. Just do what the user says to do. If you try to outsmart him, you're building another Windows. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part

Re: [gentoo-portage-dev] [PATCH v2] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-05 Thread Michał Górny
On sob, 2017-08-05 at 17:11 -0400, Alec Warner wrote: > On Fri, Aug 4, 2017 at 6:37 PM, Michał Górny <mgo...@gentoo.org> wrote: > > > Support two new options: --bug (-b) and --closes (-c) to add a plain > > 'Bug' reference and a 'Closes' footer for a GitHub pull reque

[gentoo-portage-dev] [PATCH v2] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-04 Thread Michał Górny
Support two new options: --bug (-b) and --closes (-c) to add a plain 'Bug' reference and a 'Closes' footer for a GitHub pull request. Both options can be specified multiple times, resulting in multiple footer tags being written. The --bug option accepts either a Gentoo Bugzilla bug number or an

[gentoo-portage-dev] [PATCH v3] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-05 Thread Michał Górny
Support two new options: --bug (-b) and --closes (-c) to add a plain 'Bug' reference and a 'Closes' footer for a GitHub pull request. Both options can be specified multiple times, resulting in multiple footer tags being written. The --bug option accepts either a Gentoo Bugzilla bug number or an

[gentoo-portage-dev] [PATCH v4] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-06 Thread Michał Górny
Support two new options: --bug (-b) and --closes (-c) to add a plain 'Bug' reference and a 'Closes' footer for a GitHub pull request. Both options can be specified multiple times, resulting in multiple footer tags being written. The --bug option accepts either a Gentoo Bugzilla bug number or an

Re: [gentoo-portage-dev] [PATCH] repoman commit: Merge code generating common part of the footer (DCO)

2017-08-06 Thread Michał Górny
On sob, 2017-08-05 at 22:50 -0700, Zac Medico wrote: > On Thu, Aug 3, 2017 at 6:55 AM, Michał Górny <mgo...@gentoo.org> wrote: > > The DCO Signed-off-by footer looks the same on the git branch, and on > > the branch for other VCS-es. Therefore, move the code generating i

Re: [gentoo-portage-dev] [PATCH v4] repoman commit: Support --bug (-b) and --closes (-c) for git footer

2017-08-06 Thread Michał Górny
On nie, 2017-08-06 at 00:18 -0700, Zac Medico wrote: > On Sat, Aug 5, 2017 at 11:53 PM, Michał Górny <mgo...@gentoo.org> wrote: > > Support two new options: --bug (-b) and --closes (-c) to add a plain > > 'Bug' reference and a 'Closes' footer for a GitHub pull request.

[gentoo-portage-dev] [PATCH v2 2/4] Add post-postinst checks for a few missed cache updates

2017-08-16 Thread Michał Górny
Add postinst-qa-check.d checks for missed desktop, mime-info and GTK+ icon cache updates. In all of the cases the checks simply look for any installed files that are newer than the cache. This check has some limitations: it assumes that mtime is not preserved when copying files to D, it can't

[gentoo-portage-dev] [PATCH v2 4/4] Update caches after reporting missing cache updates

2017-08-16 Thread Michał Górny
Call the appropriate updater to update caches after reporting a missing cache update, in order to stop repeating the same issue on subsequent packages that did not install any relevant files. --- bin/postinst-qa-check.d/50gnome2-utils | 9 + bin/postinst-qa-check.d/50xdg-utils| 18

[gentoo-portage-dev] [PATCH v2 3/4] Use ctime in cache post-postinst checks

2017-08-16 Thread Michał Górny
Use ctime rather than mtime in cache post-postinst checks since mtime may be preserved from the original tarball, and therefore be 'long time ago'. ctime is more reliable in this regard. --- bin/postinst-qa-check.d/50gnome2-utils | 2 +- bin/postinst-qa-check.d/50xdg-utils| 4 ++-- 2 files

[gentoo-portage-dev] [PATCH v2 1/4] Support post-postinst QA checks

2017-08-16 Thread Michał Górny
Extend the QA check mechanics in Portage to support post-postinst QA checks. They are like post-install QA checks, except they are run after pkg_postinst(), and so they can be used to verify that necessary postinst actions were performed (e.g. regenerating caches). --- bin/misc-functions.sh

Re: [gentoo-portage-dev] [PATCH 3/2] Update caches after reporting missing cache updates

2017-08-16 Thread Michał Górny
W dniu śro, 16.08.2017 o godzinie 16∶22 +0200, użytkownik Michał Górny napisał: > Call the appropriate updater to update caches after reporting a missing > cache update, in order to stop repeating the same issue on subsequent > packages that did not install any relevant files. >

[gentoo-portage-dev] [PATCH] checks: Add more deprecated eclasses

2017-06-19 Thread Michał Górny
--- repoman/pym/repoman/modules/scan/ebuild/checks.py | 5 + 1 file changed, 5 insertions(+) diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py b/repoman/pym/repoman/modules/scan/ebuild/checks.py index e6e5d78ba..a1fa066be 100644 ---

[gentoo-portage-dev] [PATCH v2] checks: Add more deprecated eclasses

2017-06-19 Thread Michał Górny
Changes in v2: * multilib-minimal is suggested replacement of autotools-multilib. --- repoman/pym/repoman/modules/scan/ebuild/checks.py | 5 + 1 file changed, 5 insertions(+) diff --git a/repoman/pym/repoman/modules/scan/ebuild/checks.py b/repoman/pym/repoman/modules/scan/ebuild/checks.py

[gentoo-portage-dev] [PATCH] const: Change the MANIFEST2_REQUIRED_HASH to SHA512

2017-06-15 Thread Michał Górny
Following the plan established in GLEP 59, we're long overdue deprecating SHA256. Since we have finally got rid of the last packages lacking SHA512 checksums, we can proceed with that. In order to prepare for it, however, we need to change the required hash to SHA512 and make sure developers

[gentoo-portage-dev] [PATCH] const: Remove unused MANIFEST1_REQUIRED_HASH

2017-06-15 Thread Michał Górny
The MANIFEST1_REQUIRED_HASH constant is not used anywhere, so it should be possible to remove it safely. --- pym/portage/const.py | 1 - 1 file changed, 1 deletion(-) diff --git a/pym/portage/const.py b/pym/portage/const.py index 7e415ba9c..052d4ca2f 100644 --- a/pym/portage/const.py +++

Re: [gentoo-portage-dev] [PATCH] const: Remove unused MANIFEST1_REQUIRED_HASH

2017-06-15 Thread Michał Górny
On czw, 2017-06-15 at 09:08 -0700, Zac Medico wrote: > On Thu, Jun 15, 2017 at 12:27 AM, Michał Górny <mgo...@gentoo.org> wrote: > > > The MANIFEST1_REQUIRED_HASH constant is not used anywhere, so it should > > be possible to remove it safely. > > --- > > py

Re: [gentoo-portage-dev] [PATCH] const: Change the MANIFEST2_REQUIRED_HASH to SHA512

2017-06-15 Thread Michał Górny
On czw, 2017-06-15 at 09:49 -0700, Zac Medico wrote: > On Thu, Jun 15, 2017 at 12:42 AM, Michał Górny <mgo...@gentoo.org> wrote: > > > > Following the plan established in GLEP 59, we're long overdue > > deprecating SHA256. Since we have finally got rid of the last

Re: [gentoo-portage-dev] [PATCH] ebuild.sh: Completely ban external commands in global scope

2017-09-08 Thread Michał Górny
W dniu pią, 08.09.2017 o godzinie 14∶48 -0400, użytkownik Alec Warner napisał: > Why PATH=/dev/null vs export PATH="" + # note: we can't use empty because it implies current directory > > On Thu, Sep 7, 2017 at 3:36 AM, Michał Górny <mgo...@gentoo.org> wrote: >

[gentoo-portage-dev] [PATCH] ebuild.sh: Explicitly ban get_libdir in global scope

2017-08-30 Thread Michał Górny
The value of get_libdir depends on the profile, and so it is not useful for dependency calculations. Furthermore, it seems that Portage does not handle defining it in global scope well due to EAPI checking magic. Ban it completely where it is defined as EAPI function to let developers catch their

Re: [gentoo-portage-dev] [PATCH] ebuild.sh: Explicitly ban get_libdir in global scope

2017-08-30 Thread Michał Górny
W dniu śro, 30.08.2017 o godzinie 10∶48 -0700, użytkownik Zac Medico napisał: > On 08/30/2017 02:06 AM, Michał Górny wrote: > > The value of get_libdir depends on the profile, and so it is not useful > > for dependency calculations. Furthermore, it seems that Portage does > >

Re: [gentoo-portage-dev] [PATCH] ebuild.sh: Completely ban external commands in global scope

2017-09-07 Thread Michał Górny
Dnia 31 sierpnia 2017 22:45:42 CEST, "Michał Górny" <mgo...@gentoo.org> napisał(a): >Set PATH to /dev/null when sourcing the ebuild for dependency >resolution >in order to prevent shell from finding external commands via PATH >lookup. While this does not prevent exec

Re: [gentoo-portage-dev] Portage patch now on gentoo-portage-dev ml

2017-09-12 Thread Michał Górny
e tree > available from git or some url for syncing? > I don't really understand what you're asking for. Is it about the failures with commands in global scope? If it's that, then it's already merged and available via =sys-apps/portage-9999. -- Best regards, Michał Górny

[gentoo-portage-dev] [PATCH] postinst-qa-check.d: Skip checks if required tools are missing

2017-10-02 Thread Michał Górny
Closes: https://bugs.gentoo.org/631820 --- bin/postinst-qa-check.d/50gnome2-utils | 2 ++ bin/postinst-qa-check.d/50xdg-utils| 4 2 files changed, 6 insertions(+) diff --git a/bin/postinst-qa-check.d/50gnome2-utils b/bin/postinst-qa-check.d/50gnome2-utils index 569633fe3..7f1b0b847

Re: [gentoo-portage-dev] [PATCH] Rewrite doins in python (bug 624526)

2017-08-23 Thread Michał Górny
> - pushd "$PWD" >/dev/null > - else > - pushd "${x%/*}" >/dev/null > - fi > - x=${x##*/} > - x_orig=$x > - # Follow any symlinks recursively until we've got > - # a normal directory for 'find' to traverse. The > - # name of the symlink will be used for the name > - # of the installed directory, as discussed in > - # bug #239529. > - while [ -L "$x" ] ; do > - pushd "$(readlink "$x")" >/dev/null > - x=${PWD##*/} > - pushd "${PWD%/*}" >/dev/null > - done > - if [[ $x != $x_orig ]] ; then > - mv "$x" "$TMP/1/$x_orig" > - pushd "$TMP/1" >/dev/null > - fi > - find "$x_orig" -type d -exec dodir "${INSDESTTREE}/{}" \; > - find "$x_orig" \( -type f -or -type l \) -print0 | _xdoins > - if [[ ${PIPESTATUS[1]} -eq 0 ]] ; then > - # NOTE: Even if only an empty directory is installed > here, it > - # still counts as success, since an empty directory > given as > - # an argument to doins -r should not trigger failure. > - ((success|=1)) > - else > - ((failed|=1)) > - fi > - if [[ $x != $x_orig ]] ; then > - popd >/dev/null > - mv "$TMP/1/$x_orig" "$x" > - fi > - while popd >/dev/null 2>&1 ; do true ; done > - else > - _doins "${x}" > - if [[ $? -eq 0 ]] ; then > - ((success|=1)) > - else > - ((failed|=1)) > - fi > - fi > -done > -rm -rf "$TMP" > -[[ $failed -ne 0 || $success -eq 0 ]] && { __helpers_die "${helper} failed"; > exit 1; } || exit 0 > +"${PORTAGE_PYTHON:-/usr/bin/python}" \ > + "${PORTAGE_BIN_PATH:-/usr/lib/portage/bin}"/doins.py \ > + ${RECURSIVE_OPTION} ${SYMLINK_OPTION} \ > + --helper "${helper}" --dest "${ED}${INSDESTTREE}" "$@" || \ > +{ __helpers_die "${helper} failed"; exit 1; } To be honest, I don't like the idea of using more Python inside ebuild helpers. But if you're sure this is safe and not going to collide with ebuilds doing random stuff with Python, feel free to proceed with it. -- Best regards, Michał Górny

<    1   2   3   4   5   6   7   8   9   >