[gentoo-portage-dev] [PATCH 2/2] portage.checksum: Support pysha3 fallback for SHA3

2017-03-12 Thread Michał Górny
pysha3 provides a stand-alone FIPS-compliant SHA3 implementation that can be used as a fallback for Python < 3.6. --- pym/portage/checksum.py | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index

[gentoo-portage-dev] [PATCH 1/2] portage.checksum: Update fallback doc for SHA3/BLAKE2*

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index 042a0a745..ac11d3f4b 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -24,10 +24,10 @@ import tempfile # SHA512:

[gentoo-portage-dev] [PATCH 10/14] portage.checksum: Reorder to avoid loading redundant impls

2017-03-12 Thread Michał Górny
Reorder the checksum implementations to start with the most preferred implementation, and try fallbacks only if the most preferred implementation is not available. Most importantly, this means that Portage will no longer attempt to load all hash libraries in the system, especially when it can

[gentoo-portage-dev] [PATCH 11/14] portage.checksum: Support pygcrypt as optimized fallback

2017-03-12 Thread Michał Górny
pygcrypt uses libgcrypt which provides support for ripemd160, whirlpool, SHA3, plus some algorithms not provided by any other library. --- pym/portage/checksum.py | 38 ++ 1 file changed, 38 insertions(+) diff --git a/pym/portage/checksum.py

[gentoo-portage-dev] [PATCH 12/14] Support STREEBOG{256,512} hash function (from pygcrypt), #597736

2017-03-12 Thread Michał Górny
--- pym/portage/checksum.py | 5 - pym/portage/const.py| 3 ++- pym/portage/tests/util/test_checksum.py | 18 ++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index

[gentoo-portage-dev] [PATCH 14/14] travis: Install additional (pure Python) crypto providers if necessary

2017-03-12 Thread Michał Górny
Install pysha3 to provide the SHA3 routines on Python < 3.6, and install pygost to provide a Streebog implementation. --- .travis.yml | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 7fe3c06aa..196e3520a 100644 --- a/.travis.yml +++

[gentoo-portage-dev] [PATCH 13/14] portage.checksum: Support pygost as fallback Streebog provider

2017-03-12 Thread Michał Górny
Support the pure Python implementation of Streebog in pygost as a fallback algorithm. The code is horrible (it stores all the data in memory before hashing), so it is really intended as last fallback. --- pym/portage/checksum.py | 15 +++ 1 file changed, 15 insertions(+) diff --git

Re: [gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Zac Medico
On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: > Use get_valid_checksum_keys() function instead of accessing hashfunc_map > directly throughout the Portage. > --- > pym/portage/_emirrordist/FetchTask.py | 2 +- > pym/portage/dbapi/bintree.py | 4 ++-- >

[gentoo-portage-dev] [PATCH 05/14] tests: Add minimal safety checks for checksums

2017-03-12 Thread Michał Górny
--- pym/portage/tests/util/test_checksum.py | 88 + 1 file changed, 88 insertions(+) create mode 100644 pym/portage/tests/util/test_checksum.py diff --git a/pym/portage/tests/util/test_checksum.py b/pym/portage/tests/util/test_checksum.py new file mode 100644

[gentoo-portage-dev] [PATCH 03/14] portage.checksum: create explicit checksum_file() method

2017-03-12 Thread Michał Górny
Make the file checksum generation code use an explicit checksum_file() method rather than implicit __call__. This should be more readable, and make it cleanly possible to add more methods. --- pym/portage/checksum.py | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff

[gentoo-portage-dev] [PATCH 08/14] portage.checksum: Remove exception handling for missing hashlib

2017-03-12 Thread Michał Górny
Remove the try-except block for potential ImportError of hashlib. The hashlib module should be available in all supported Python versions, and we do not really test or support the case when it is not available. --- pym/portage/checksum.py | 55 ++--- 1

Re: [gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
W dniu 12.03.2017, nie o godzinie 13∶36 -0700, użytkownik Zac Medico napisał: > On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: > > Use get_valid_checksum_keys() function instead of accessing hashfunc_map > > directly throughout the Portage. > > --- > >

Re: [gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Zac Medico
On Sun, Mar 12, 2017 at 2:13 PM, Michał Górny wrote: > W dniu 12.03.2017, nie o godzinie 13∶36 -0700, użytkownik Zac Medico > napisał: >> On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: >> > Use get_valid_checksum_keys() function instead of accessing

[gentoo-portage-dev] [PATCH 04/14] portage.checksum: Support getting byte string checksums

2017-03-12 Thread Michał Górny
Add a checksum_str() method to Portage hashes and a matching function to make it possible to compute checksums of arbitrary bytestrings rather than just files. --- pym/portage/checksum.py | 29 + 1 file changed, 29 insertions(+) diff --git a/pym/portage/checksum.py

[gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-12 Thread Michał Górny
Hi, Here's a huge batch of patches for portage.checksum and relevant stuff. It's not a complete rewrite as I have originally planned, and it's still not the code I'd like to see but functionally it has been improved a lot. Notable changes: 1. Other modules no longer use hashfunc_map, now they

[gentoo-portage-dev] [PATCH 02/14] portage.checksum: Remove python-fchksum support

2017-03-12 Thread Michał Górny
Remove the support for MD5 implementation from python-fchksum package. The package is rarely installed, supports Python 2.7 only and the code handles checksumming a whole file only. --- pym/portage/checksum.py | 11 +-- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git

[gentoo-portage-dev] [PATCH 01/14] Use public API: hashfunc_map -> get_valid_checksum_keys()

2017-03-12 Thread Michał Górny
Use get_valid_checksum_keys() function instead of accessing hashfunc_map directly throughout the Portage. --- pym/portage/_emirrordist/FetchTask.py | 2 +- pym/portage/dbapi/bintree.py | 4 ++-- pym/portage/eclass_cache.py | 2 +- pym/portage/manifest.py | 4

Re: [gentoo-portage-dev] [PATCHES] portage.checksum hacking, pt. 4

2017-03-12 Thread Zac Medico
On Sun, Mar 12, 2017 at 11:59 AM, Michał Górny wrote: > Hi, > > Here's a huge batch of patches for portage.checksum and relevant stuff. > It's not a complete rewrite as I have originally planned, and it's still > not the code I'd like to see but functionally it has been