[gentoo-dev] cmake + ninja vs autotools

[William L. Thomson Jr.]

It maybe worth considering switching the default generator in the
cmake-utils.eclass from the default of emake to ninja.

- : ${CMAKE_MAKEFILE_GENERATOR:=emake}
+ : ${CMAKE_MAKEFILE_GENERATOR:=ninja}

For those with cmake ebuilds you can test this out now via 

CMAKE_MAKEFILE_GENERATOR="ninja"
inherit cmake-utils

Working with both cmake and meson. It seems the real performance of
meson comes from ninja. I am a bit more a fan of cmake than meson for
cpack, generation of deb, rpm, and binary tarball, in addition to
sources. That can be done with meson but not as elegantly at this time.

ninja is noticeably faster than make. I haven't seen any cases yet where
cmake autotools works, and ninja does not. They seem pretty equal, so
should be safe. Of course could use testing first.

Again something to consider. Myself I am avoiding autotools anywhere I
can, mostly because of the slow configure and also slow make. Anytime I
use cmake, I am using ninja generator.

-- 
William L. Thomson Jr.


pgpp2umjxeef0.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[Michał Górny]

W dniu śro, 15.11.2017 o godzinie 17∶28 +0100, użytkownik Michał Górny
napisał:
> Hi, everyone.
> 
> The Council has approved the manifest-hashes switch on 2017-11-12
> meeting [1]. The transition will occur to the initial plan, with small
> changes. The updated plan is included at the end of this mail.

Missing reference spotted by Nils Freydank (thanks!):

[1]:https://projects.gentoo.org/council/meeting-logs/20171112-summary.txt

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[William L. Thomson Jr.]

On Wed, 15 Nov 2017 17:10:11 -0500
"William L. Thomson Jr."  wrote:
>
> Like FreeBSD Foundation, a real organization paying for development
> via grants and other things...
> https://www.freebsdfoundation.org/
> https://www.freebsdfoundation.org/what-we-do/grants/
> 
> https://www.freebsdfoundation.org/wp-content/uploads/2015/12/2017-Q2-Profit-Loss.pdf

For the record, FreeBSD was not like that long ago when Gentoo/we had
booths near them at Linux World Expo. Most of that has happened since.

" A budget of $80,000 was allocated for 2008 to fund multiple
development projects." 
https://www.freebsd.org/news/2008/index.html

https://www.freebsdfoundation.org/wp-content/uploads/2015/12/Budget2011.pdf

-- 
William L. Thomson Jr.


pgpx4myn7oQvo.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[William L. Thomson Jr.]

On Wed, 15 Nov 2017 16:15:18 -0500
Rich Freeman  wrote:

> On Wed, Nov 15, 2017 at 3:21 PM, William L. Thomson Jr.
>  wrote:
> >
> > The council has no power
> > over Trustees, and Trustees do have legal power over all of
> > Gentoo.  
> 
> Sure, just keep in mind that legally Gentoo is basically nothing but a
> name and a logo.

That is what others have made it to be. It was never intended to be
such by the person who created the foundation in the first place.
Paying for those legal filings and attorney fees out of pocket! The
only time any proper IRS filings was done for Gentoo
I have showed in -project what it was supposed to be.
https://archives.gentoo.org/gentoo-project/message/3ac5418dd061fc53f4b8d55a99773f4c

Like FreeBSD Foundation, a real organization paying for development
via grants and other things...
https://www.freebsdfoundation.org/
https://www.freebsdfoundation.org/what-we-do/grants/

https://www.freebsdfoundation.org/wp-content/uploads/2015/12/2017-Q2-Profit-Loss.pdf

Presently working on raising $1.25 Million USD for next year.
Look at the donors on their home page. Intel, Netapp, Versign, Netflix
even Microsoft...

Gentoo is missing out big time! But it is the communities choice. If
Gentoo is just a name and logo, that is pretty sad
Limited vision

-- 
William L. Thomson Jr.


pgp0ONUuZb1Gt.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[Rich Freeman]

On Wed, Nov 15, 2017 at 3:21 PM, William L. Thomson Jr.
 wrote:
>
> The council has no power
> over Trustees, and Trustees do have legal power over all of Gentoo.

Sure, just keep in mind that legally Gentoo is basically nothing but a
name and a logo.

The Trustees could ask the community to stop using the name and logo,
but in practice that would be about all they could do, in a
hypothetical world where the Trustees and the Community were in some
kind of conflict (which is unlikely in practice).

The actual distro is largely community-run.

Gentoo isn't like Redhat.  It isn't a legal entity that happens to
have a community.  It is more of a community that happens to have a
legal entity.  Since the Foundation doesn't actually have any
employees it is actually fairly limited in its ability to make things
happen in the real world.  With its funding it is actually fairly
limited in making things happen legally as well.  Its main function at
this time (IMO) is to protect the name so that somebody else who has
more money doesn't tell us to stop using it, and to pay for moderate
expenses.

> Which is some what strange as Trustees can be responsible for the actions of 
> others. Despite having no say in such matters till it becomes a legal issue.

As long as the Trustees aren't violating any laws or blatantly
ignoring their violation I doubt they'd have much personal liability
to outsiders.  The Foundation has more liability, but really the main
risk there is losing whatever donations are sitting around unspent and
maybe having somebody tell us we can't call ourselves Gentoo.  The
practical independence of the community from the Foundation works both
ways.  Having to change our name would be inconvenient, but it is
still an unlikely outcome as we do try to be reasonably careful.

-- 
Rich

[gentoo-dev] Manifest2 hashes: validation of single hash per MANIFESTx_REQUIRED_HASH

[Robin H. Johnson]

Replying to your original question here, to repeat the answer I emphasised
before, along with significantly more detail in the history of Portage hashes
(pulled from my notes back to GLEP57 and some minor updates).

On Wed, Nov 08, 2017 at 12:57:49PM -0600, R0b0t1 wrote:
> These posts are concerning because it looks like someone became stir
> crazy and invented a problem to solve. The changes proposed to date
> have remained poorly justified, and no one has addressed the concern
> that multiple hashes *is* actually more secure.
> 
> If it was deemed necessary at one point, what justification was used?
> I.e. https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence.
On Wed, Nov 15, 2017 at 11:47:41AM -0600, R0b0t1 wrote:
> Does the existence of a decision mean I would need to contact the trustees
> if I feel the changes have not been adequately justified?

In GLEP59, I referenced a paper by Joux [J04], in which it was shown that a
concatenation of multiple hashes is NOT much more secure against collisions
than the strongest of the individual hashes.

That was cited as original logic in GLEP59 for the removal of SHA256 (that
removal was never implemented). WHIRLPOOL & SHA512 were kept out of an
abundance of caution at the time, mostly to implementation bugs in hashes (as I
have referenced in the related threads since).

Your logic regarding removing something you think I don't understand is wrong
(Chesterton's Fence): 

If you dig in the history of Portage, you will see that it's always been valid,
to have just a SINGLE hash for each file in a Manifest.  Required hashes has
NEVER contained more than one hash.

If multiple hashes are present, then Portage will validate all of them, but a
potential attacker can still modify the Manifest and have only a single hash
listed.  Exactly which hash MUST be present has changed over time. 

Manifest1 is very old, and was stored in $CAT/$PN/files/digest-$P
Manifest2 is the current $CAT/$PN/Manifest (and soon in more locations per 
MetaManifest).

1999/xx/xx: Portage starts with Manifest1 format, MD5-only (CVS)
2004/08/25: Portage gets SHA1 support in Manifest1, but is problematic, SHA1 
generation manual only.
https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-src/portage/pym/portage_checksum.py?revision=1.1&view=markup
https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-src/portage/pym/portage.py?r1=1.485&r2=1.486
2005/12/19: Portage Manifest1 supports MD5,SHA1,SHA256,RMD160, but still 
requires only a single hash present. Generates MD5+SHA256+RMD160.
https://gitweb.gentoo.org/proj/portage.git/commit/?id=cd3e3775966a9f58aebb91f58cbdb5903faad3de
2006/03/24: Manifest2 introduced.
https://gitweb.gentoo.org/proj/portage.git/commit/?id=f993747ca501e8a70d6f6174711149a172cfc3c2
2007/01/20: MANIFEST2_REQUIRED_HASH introduced, SHA1, it must be present & pass
https://gitweb.gentoo.org/proj/portage.git/commit/?id=e768571187d1655fbb558c23d61fa2983e48e411
2007/12/18: MANIFEST1_REQUIRED_HASH introduced, MD5, it must be present & pass
https://gitweb.gentoo.org/proj/portage.git/commit/?id=d9b10deaa03ce174d5ccc3b59c477549ad87e884
2008/02/28: Manifest1 support dropped.
https://gitweb.gentoo.org/proj/portage.git/commit/?id=66940e1f2f0549ee8f01dad59016e168105e193d
2011/10/02: GLEP59 implemented, MANIFEST2_REQUIRED_HASH changes to SHA256
https://gitweb.gentoo.org/proj/portage.git/commit/?id=c8cd3a985cc529299411d7343a11004b7d1330ef
2017/06/15: MANIFEST2_REQUIRED_HASH changes to SHA512
https://gitweb.gentoo.org/proj/portage.git/commit/?id=e6abcc0b7cbdca481862a5c7cca946c01c471ffb

[J04] Joux, Antoie. (2004). "Multicollisions in Iterated Hash Functions - 
Application to Cascaded Constructions;" 
Proceedings of CRYPTO 2004, Franklin, M. (Ed); Lecture Notes in Computer 
Science 3152, pp. 306-316. 
Available online from: 
http://web.cecs.pdx.edu/~teshrim/spring06/papers/general-attacks/multi-joux.pdf

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Asst. Treasurer
E-Mail   : robb...@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136


signature.asc
Description: Digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[William L. Thomson Jr.]

On Wed, 15 Nov 2017 14:21:59 -0500
NP-Hardass  wrote:
>
> 
> No, if you think there is an issue with the Council decision, you
> should speak with the Council.  Moreover... The Council is
> responsible for technical decisions within Gentoo.  Unless it
> violates the Social Contract, I cannot see how the Trustees should be
> involved here.  They have empowered the Council to make technical
> decisions as they see fit.

There is nothing empowering the Council from the Trustees. Legally the
Trustees could have final say. The Council is not a legal body nor
formal in any legal filing. They have no say when it comes down to it
from a legal perspective.

However everything is structured such that the Council does have final
say over all technical matters. That is something the community
did, and adheres to. There is nothing, to my recollection, in Foundation
By Laws or other that would connect the two. The council has no power
over Trustees, and Trustees do have legal power over all of Gentoo.

The Trustees just abstain from technical matters, as that is the
structure the community has accepted. I am not aware of anything else
with such structure.

Nothing legally gives Council final say technically. That is just how
things are, and likely will always remain as such. Which is some what
strange as Trustees can be responsible for the actions of others.
Despite having no say in such matters till it becomes a legal issue.

IMHO I rather see a structure where Council is more like CTO. Still top
for technical, but falls under Trustees for oversight, final say, etc.
A normal structure like exists in most any business globally. They
should work together more as one vs two bodies.

-- 
William L. Thomson Jr.


pgplVlLDTr5OY.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[William L. Thomson Jr.]

On Wed, 15 Nov 2017 11:47:41 -0600
R0b0t1  wrote:
.> 
> Does the existence of a decision mean I would need to contact the
> trustees if I feel the changes have not been adequately justified?

They have no power here. Consider the foundation as second head of a 2
headed snake. Used to be 3 when infra was more powerful. Trustees maybe
could if there was a different structure. But the foundations role is
pretty limited and intentionally crippled.

Unless you can provide a compelling legal case that would fall under
the legal protection duties of the trustees. But in general Trustees
cannot dictate to council, its the other way around. Council dictates
to Trustees. I doubt it will ever change. I tried long ago.
 
Council has final say on all technical matters unless it involves
legalities.

-- 
William L. Thomson Jr.


pgprHXOabiLOc.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[Nils Freydank]

Am Mittwoch, 15. November 2017, 17:28:44 CET schrieb Michał Górny:
> Hi, everyone.
> 
> The Council has approved the manifest-hashes switch on 2017-11-12
> meeting [1]. The transition will occur to the initial plan, with small
> changes. The updated plan is included at the end of this mail.
> [...]
Just for general convenience:
It appears you forgot the actual link [1], and I assume it should be:
https://wiki.gentoo.org/wiki/Project:Council/Meeting_logs
resp. taken from this site
https://projects.gentoo.org/council/meeting-logs/20171112.txt

Signatures for the textfile logs are linked on the wiki page aswell.
-- 
GPG fingerprint: '766B 8122 1342 6912 3401 492A 8B54 D7A3 FF3C DB17'
Holgersson

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[NP-Hardass]

On 11/15/2017 12:47 PM, R0b0t1 wrote:
> On Wednesday, November 15, 2017, Michał Górny  > wrote:
>> Hi, everyone.
>>
>> The Council has approved the manifest-hashes switch on 2017-11-12
>> meeting [1]. The transition will occur to the initial plan, with small
>> changes. The updated plan is included at the end of this mail.
>>
>> According to this plan, BLAKE2B will be enabled on 2017-11-21. This
>> means that starting at this time, all new and updated DIST entries will
>> use BLAKE2B+SHA512. Old DIST entries will still use the current hash set
>> until updated.
>>
>> The developers are required to upgrade to a package manager supporting
>> this hash. That is:
>>
>> a. Portage 2.3.5 when using py3.6+,
>>
>> b. Portage 2.3.13 + pyblake2 installed manually,
>>
>> c. Portage 2.3.13-r1 that includes the pyblake2 dep.
>>
>> Modern (and old) Portage will refuse to update Manifests if it does not
>> support the necessary hashes. However, Portage versions between 2.3.5
>> and 2.3.13 inclusively will create Manifests missing BLAKE2B hash rather
>> than failing when no hash provider is present. Those Manifests will be
>> rejected by the git hook.
>>
>> Users will not be affected noticeably as the SHA512 hash continues being
>> used for compatibility.
>>
>>
>> That said, I'd like to request developers not to start proactively
>> converting all old Manifest entries to the new set immediately,
>> and instead give some time for things to settle down.
>>
>>
>>
>> The updated plan
>> 
>>
>> Already done:
>>
>> - revbumped Portage with pyblake2 dep and started stabilizing it,
>>
>> - added git update hook to reject invalid Manifest entries.
>>
>> 2017-11-21 (T+7d):
>>
>> - manifest-hashes = BLAKE2B SHA512
>>
>> 2018-02-14 (T+3m):
>>
>> - manifest-required-hashes = BLAKE2B
>>
>> 2018-05-14 (T+6m):
>>
>> - last rite fetch-restricted packages that do not use BLAKE2B.
>>
>> The final removal of SHA512 will be decided by the Council separately.
>>
> 
> Does the existence of a decision mean I would need to contact the
> trustees if I feel the changes have not been adequately justified?
> 
> Respectfully,
>     R0b0t1

No, if you think there is an issue with the Council decision, you should
speak with the Council.  Moreover... The Council is responsible for
technical decisions within Gentoo.  Unless it violates the Social
Contract, I cannot see how the Trustees should be involved here.  They
have empowered the Council to make technical decisions as they see fit.

-- 
NP-Hardass



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[R0b0t1]

On Wednesday, November 15, 2017, Michał Górny  wrote:
> Hi, everyone.
>
> The Council has approved the manifest-hashes switch on 2017-11-12
> meeting [1]. The transition will occur to the initial plan, with small
> changes. The updated plan is included at the end of this mail.
>
> According to this plan, BLAKE2B will be enabled on 2017-11-21. This
> means that starting at this time, all new and updated DIST entries will
> use BLAKE2B+SHA512. Old DIST entries will still use the current hash set
> until updated.
>
> The developers are required to upgrade to a package manager supporting
> this hash. That is:
>
> a. Portage 2.3.5 when using py3.6+,
>
> b. Portage 2.3.13 + pyblake2 installed manually,
>
> c. Portage 2.3.13-r1 that includes the pyblake2 dep.
>
> Modern (and old) Portage will refuse to update Manifests if it does not
> support the necessary hashes. However, Portage versions between 2.3.5
> and 2.3.13 inclusively will create Manifests missing BLAKE2B hash rather
> than failing when no hash provider is present. Those Manifests will be
> rejected by the git hook.
>
> Users will not be affected noticeably as the SHA512 hash continues being
> used for compatibility.
>
>
> That said, I'd like to request developers not to start proactively
> converting all old Manifest entries to the new set immediately,
> and instead give some time for things to settle down.
>
>
>
> The updated plan
> 
>
> Already done:
>
> - revbumped Portage with pyblake2 dep and started stabilizing it,
>
> - added git update hook to reject invalid Manifest entries.
>
> 2017-11-21 (T+7d):
>
> - manifest-hashes = BLAKE2B SHA512
>
> 2018-02-14 (T+3m):
>
> - manifest-required-hashes = BLAKE2B
>
> 2018-05-14 (T+6m):
>
> - last rite fetch-restricted packages that do not use BLAKE2B.
>
> The final removal of SHA512 will be decided by the Council separately.
>

Does the existence of a decision mean I would need to contact the trustees
if I feel the changes have not been adequately justified?

Respectfully,
R0b0t1

[gentoo-dev] manifest-hashes changing to 'BLAKE2B SHA512' on 2017-11-21

[Michał Górny]

Hi, everyone.

The Council has approved the manifest-hashes switch on 2017-11-12
meeting [1]. The transition will occur to the initial plan, with small
changes. The updated plan is included at the end of this mail.

According to this plan, BLAKE2B will be enabled on 2017-11-21. This
means that starting at this time, all new and updated DIST entries will
use BLAKE2B+SHA512. Old DIST entries will still use the current hash set
until updated.

The developers are required to upgrade to a package manager supporting
this hash. That is:

a. Portage 2.3.5 when using py3.6+,

b. Portage 2.3.13 + pyblake2 installed manually,

c. Portage 2.3.13-r1 that includes the pyblake2 dep.

Modern (and old) Portage will refuse to update Manifests if it does not
support the necessary hashes. However, Portage versions between 2.3.5
and 2.3.13 inclusively will create Manifests missing BLAKE2B hash rather
than failing when no hash provider is present. Those Manifests will be
rejected by the git hook.

Users will not be affected noticeably as the SHA512 hash continues being
used for compatibility.


That said, I'd like to request developers not to start proactively
converting all old Manifest entries to the new set immediately,
and instead give some time for things to settle down.



The updated plan


Already done:

- revbumped Portage with pyblake2 dep and started stabilizing it,

- added git update hook to reject invalid Manifest entries.

2017-11-21 (T+7d):

- manifest-hashes = BLAKE2B SHA512

2018-02-14 (T+3m):

- manifest-required-hashes = BLAKE2B

2018-05-14 (T+6m):

- last rite fetch-restricted packages that do not use BLAKE2B.

The final removal of SHA512 will be decided by the Council separately.


-- 
Best regards,
Michał Górny