Re: [gentoo-dev] Manifest2 hashes, take n+1-th: one hash to decide them all

[Robin H. Johnson]

On Tue, Oct 24, 2017 at 11:33:39PM +0200, Allan Wegan wrote:
> >> That is currently the case with portage, but not an inevitable
> >> consequence of having 3 hash functions in the Manifest. Portage could
> >> be made to check only one or two of them (even by default), giving
> >> the tie-breaking ability to those who need it, and speeding up things
> >> for those who don't.
> > No, it can't. The specification (GLEP 59) requires it to check all
> > hashes.
> 
> Of course it can: The code of the specification just has to be changed
> before changing the code of portage. The question is not whether it is
> possible to make portage skip hash verification - but whether it is a
> good idea to make it do that...
> 
> I would not mind as long as the default is to always check all the
> hashes and the option to disable it is properly named (like
> "--disable-hash-verification" or something similar) and documented.
At that point, and this is a serious proposal:
The package manager shall decide which hashes to check, but is required
to check at least one hash. The choice may be 'fastest', 'most secure',
or any local factor.

For local values of 'most secure' or 'fastest'.

I wrote GLEP59, and specified at the time that all hashes should be
checked, based on prior experience with hash implementation problems.

Skipping them entirely is a bad idea, but only checking one of them is a
reasonable suggestion.

I retract my prior suggestion that there should be 3 hashes, as I
realize a key statements in GLEP59 that were NEVER implemented:
>> - Removal of depreciated checksums shall happen after no less than 18
>>   months or one major Portage version cycle, whichever is greater.
>> ...
>> After the majority of Portage installations include SHA512 support:
>> - Remove SHA256.

This GLEP to update the GLEP59 specification should state the following:
- The package manager or verification tool is required to verify at
  least one hash, preferably the strongest supported hash.
- Multiple hashes may be present for transitional periods.
- SHA3 or BLAKE2B shall be added to the official Manifest2 hashes.

For implementation:
- Generation of WHIRLPOOL and SHA256 shall be disabled in the next
  Portage minor release (as soon as possible)
- Generation of the next choice of hash, SHA3 or BLAKE2B shall be
  enabled in an upcoming minor Portage release (soon)
- 18 months after the next GLEP is approved, SHA512 shall be dropped
  (put the date into the Portage code so it happens automatically this
  time, unlike SHA256 that should have been removed in 2010!).

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Asst. Treasurer
E-Mail   : robb...@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136


signature.asc
Description: Digital signature

Re: [gentoo-dev] [RFC] GLEP 65 v2: Post-install QA checks (now with post-merge checks)

[Michael Orlitzky]

On 10/17/2017 02:12 PM, Michał Górny wrote:
> 
> Abstract
> 
> 
> ...
> The QA checks can inspect the installation image or live system respectively,

Respective to what?


> output and store both user- and machine-oriented QA warning logs, manipulate
> the files and abort the install, as necessary.
> 

An oxford comma would make that easier to read: "files, and abort the
install as necessary."


> Motivation
> ==
> 
> ...
> 
> Over time, many different QA checks have been added to Portage. That includes
> checks corresponding to generic Gentoo rules (like filesystem hierarchy,
> security requirements), checks enforcing Gentoo team policies and correct
> eclass uses.

Another weird comma issue, maybe try: "policies, and checks enforcing
correct eclass usage."



> There are two kinds of QA checks defined in this specification:
> 
> 1. Post-install QA checks (``install-qa-check.d``),
> 
> 2. Post-merge (postinst) QA checks (``postinst-qa-check.d``).
> 

It's a bit sadistic to have something called a "post-install check" but
then have "postinst check" refer to the other thing.


> 
> In case of severe QA issues, the checks are allowed to alter the contents of
> the installation image in order to sanitize them, or call the ``die`` function
> to abort the build.

I'm not sure that having the image silently modified is a good idea. It
seems like everyone would benefit more if the QA check crashed, and let
the maintainer fix his ebuild. Is this already possible with the Portage
checks, or is it new in the repository-based checks?


> 
> The post-merge QA checks are executed after the ``pkg_postinst`` ebuild phase
> finishes. They can use the same commands as are permitted in ``pkg_postinst``,
> and access the installed system location ``${ROOT}`` and the temporary
> directory ``${T}``.
> 
> The checks are allowed to alter the contents of the filesystem to the same
> degree as ``pkg_postinst`` phase is. They must not call ``die``.

Ditto re: filesystem modification.



> 
> Aside to the standard PMS function
> 

Aside from



> eqatag
> ~~
> Synopsis
>   ``eqatag [-v]  [=...] [...]``
> 
> Tag the package with specific QA issues. The *tag* parameter is
> a well-defined name identifying specific QA issue. The tag can be additionally
> associated with some data in key-value form and/or one or more *files*.
> The file paths are relative to installation image (``${D}``), and need to
> start with a leading slash.

If this is used in a post-merge check, isn't it conceivable that you
would want to tag a path outside of $D? For example, if your src_install
creates ${D}/bin/foo and then your pkg_postinst phase gives ownership of
/bin/foo to the "foo" user, I might want to tag /bin/foo (and not
${D}/bin/foo) with a bad-owner tag.


> 
> The two types of QA checks were created to account for different kinds
> of common mistakes in ebuilds.
> 
> ...
> 
> Post-postinst QA checks can be used to verify the state of system after

I think we're calling these "Post-merge" checks.



> 
> QA check script format
> --
> 

This is the second appearance of a "QA check script format" section.


> 
> Function specification
> --


Also appears twice.

Re: [gentoo-dev] Manifest2 hashes, take n+1-th: 3 hashes for the tie-breaker case

[Allan Wegan]

>> That is currently the case with portage, but not an inevitable
>> consequence of having 3 hash functions in the Manifest. Portage could
>> be made to check only one or two of them (even by default), giving
>> the tie-breaking ability to those who need it, and speeding up things
>> for those who don't.
> No, it can't. The specification (GLEP 59) requires it to check all
> hashes.

Of course it can: The code of the specification just has to be changed
before changing the code of portage. The question is not whether it is
possible to make portage skip hash verification - but whether it is a
good idea to make it do that...

I would not mind as long as the default is to always check all the
hashes and the option to disable it is properly named (like
"--disable-hash-verification" or something similar) and documented.



-- 
Allan Wegan

Jabber: allanwe...@ffnord.net
 OTR-Fingerprint: E4DCAA40 4859428E B3912896 F2498604 8CAA126F
Jabber: allanwe...@jabber.ccc.de
 OTR-Fingerprint: A1AAA1B9 C067F988 4A424D33 98343469 29164587
ICQ: 209459114
 OTR-Fingerprint: 71DE5B5E 67D6D758 A93BF1CE 7DA06625 205AC6EC



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Manifest2 hashes, take n+1-th: 3 hashes for the tie-breaker case

[Michał Górny]

W dniu wto, 24.10.2017 o godzinie 13∶56 +0200, użytkownik Chí-Thanh
Christopher Nguyễn napisał:
> Michał Górny schrieb:
> > Oh, and most notably, the speed loss will be mostly visible to users.
> > An attacker would have to compute the additional hashes only
> > if the fastest hash already matched, i.e. rarely. Users will have to
> > compute them all the time.
> 
> That is currently the case with portage, but not an inevitable consequence of
> having 3 hash functions in the Manifest. Portage could be made to check only
> one or two of them (even by default), giving the tie-breaking ability to
> those who need it, and speeding up things for those who don't.

No, it can't. The specification (GLEP 59) requires it to check all hashes.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Manifest2 hashes, take n+1-th: 3 hashes for the tie-breaker case

[Rich Freeman]

On Tue, Oct 24, 2017 at 4:21 AM, Paweł Hajdan, Jr.
 wrote:
> On 24/10/2017 06:11, Michał Górny wrote:
>> W dniu wto, 24.10.2017 o godzinie 06∶04 +0200, użytkownik Michał Górny
>> napisał:
>>> Three hashes don't give any noticeable advantage. If we want a diverse
>>> construct, we take SHA3. SHA3 is slower than SHA2 + BLAKE2 combined, so
>>> even with 3 threaded computation it's going to be slower.
>>
>> Oh, and most notably, the speed loss will be mostly visible to users.
>> An attacker would have to compute the additional hashes only
>> if the fastest hash already matched, i.e. rarely. Users will have to
>> compute them all the time.
>
> I'm surprised to see bikeshedding about this, where the performance
> argument was shown to be speculative.
>
> Consider clarifying what's the goal of this thread.
>
> It seemed like a relatively obvious cleanup / modernizing the set of
> hash functions, and I'd still be supportive of that.
>

++

IMO nothing really new has come up for the most part.  People disagree
on a few points, as is inevitable.  The purpose of mailing lists isn't
to keep reiterating the same points until there is unanimous
agreement.  Best to just move on.

-- 
Rich

Re: [gentoo-dev] Manifest2 hashes, take n+1-th: 3 hashes for the tie-breaker case

[Chí-Thanh Christopher Nguyễn]

Michał Górny schrieb:
> Oh, and most notably, the speed loss will be mostly visible to users.
> An attacker would have to compute the additional hashes only
> if the fastest hash already matched, i.e. rarely. Users will have to
> compute them all the time.

That is currently the case with portage, but not an inevitable consequence of
having 3 hash functions in the Manifest. Portage could be made to check only
one or two of them (even by default), giving the tie-breaking ability to
those who need it, and speeding up things for those who don't.


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-dev] Manifest2 hashes, take n+1-th: 3 hashes for the tie-breaker case

[Paweł Hajdan , Jr .]

On 24/10/2017 06:11, Michał Górny wrote:
> W dniu wto, 24.10.2017 o godzinie 06∶04 +0200, użytkownik Michał Górny
> napisał:
>> Three hashes don't give any noticeable advantage. If we want a diverse
>> construct, we take SHA3. SHA3 is slower than SHA2 + BLAKE2 combined, so
>> even with 3 threaded computation it's going to be slower.
> 
> Oh, and most notably, the speed loss will be mostly visible to users.
> An attacker would have to compute the additional hashes only
> if the fastest hash already matched, i.e. rarely. Users will have to
> compute them all the time.

I'm surprised to see bikeshedding about this, where the performance
argument was shown to be speculative.

Consider clarifying what's the goal of this thread.

It seemed like a relatively obvious cleanup / modernizing the set of
hash functions, and I'd still be supportive of that.

Paweł



signature.asc
Description: OpenPGP digital signature