Re: [gentoo-dev] Manifest2 hashes, take n+1-th: one hash to decide them all
On 25/10/2017 14:32, Hanno Böck wrote:
> Good security includes reducing complexity. Tough (as evident by this
> thread) it's a thought many people find hard to accept.
>
> This thread is going into a completely different direction and I find
> that worriesome. We have two non-problems ("what if secure hash X gets
> broken?" and "what if it's too slow? I haven't benchmarked, but what if
> it's too slow??") and people proposing increasingly complex solutions.
>
> If you do what you propose my worries aren't that any hash gets broken
> or that it's too slow. It's that some bug will chime in where in some
> situation no hash gets checked whatsoever.
+1
I consider the multiple hashes we have a part of providing smooth
migration path (keeping around hashes supported by older portage
versions). Other than that, yeah, watch out for complexity.
Paweł
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] GLEP 65 v2: Post-install QA checks (now with post-merge checks)
Here's v2.0.1 with suggestions from mjo applied.
ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0065.rst
HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0065.html
---
GLEP: 65
Title: Post-install QA checks
Author: Michał Górny
Type: Standards Track
Status: Draft
Version: 2
Created: 2014-10-26
Last-Modified: 2017-10-17
Post-History: 2014-10-30, 2017-10-17
Content-Type: text/x-rst
---
Abstract
This GLEP provides two kinds of QA check: checks run on the installation image
once ``src_install`` returns, and checks run on the live system once
``pkg_postinst`` returns. The checks can be provided by the Package Manager,
repositories, packages (installed system-wide) and the system administrator.
The QA checks can inspect the installation image or live system, output
and store both user- and machine-oriented QA warning logs, manipulate files
and abort the install.
Motivation
==
The current Portage versions have two main QA check mechanisms: repoman
and post-install QA checks. While repoman is usually considered more
important, it has severe limitations. In particular, it is run on repository
without executing the ebuild, therefore it is incapable of inspecting
the installed files. This is where post-install QA checks become useful.
Over time, many different QA checks have been added to Portage. That includes
checks corresponding to generic Gentoo rules (like filesystem hierarchy,
security requirements), checks enforcing Gentoo team policies, and checks
enforcing correct eclass usage. Some of the checks depend on external tools
being present.
Keeping those checks directly in Portage sources has two major disadvantages:
1. The checks can not be properly updated without Portage upgrade.
In particular, a change in a QA check becomes fully effective when
the relevant Portage version becomes stable and the user upgrades.
There is no easy way to keep QA checks in sync with eclasses.
2. Gentoo-specific checks are enforced for all repositories and derived
distributions. Modifying the QA check list requires forking Portage.
Specification
=
QA check types
--
There are two kinds of QA checks defined in this specification:
1. Post-install QA checks (``install-qa-check.d``),
2. Post-merge QA checks (``postinst-qa-check.d``).
The post-install QA checks are are executed after the ``src_install`` ebuild
phase finishes but before the binary package is built or the ``pkg_preinst``
phase is executed. They can use the same commands as are permitted
in ``src_install``, and access the installation image ``${D}``
and the temporary directory ``${T}``.
In case of severe QA issues, the checks are allowed to alter the contents of
the installation image in order to sanitize them, or call the ``die`` function
to abort the build.
The post-merge QA checks are executed after the ``pkg_postinst`` ebuild phase
finishes. They can use the same commands as are permitted in ``pkg_postinst``,
and access the installed system location ``${ROOT}`` and the temporary
directory ``${T}``.
The checks are allowed to alter the contents of the filesystem to the same
degree as ``pkg_postinst`` phase is. They must not call ``die``.
QA check file format & locations
QA checks are stored as bash scripts. The checks are identified and ordered
by file name. If files with same names are present in multiple locations,
the file in location with the highest priority is used.
The specification defines four sources of QA checks, listed in the order
of increasing priority:
1. internal checks included in the Package Manager,
2. repository-specific QA checks,
3. package-installed QA checks,
4. sysadmin-defined QA checks.
The internal checks are stored in Package Manager-specific location and should
be installed along with the Package Manager. It is recommended that only
generic checks are included in the Package Manager and not checks specific to
Gentoo policies, packages or eclasses included in Gentoo.
Repository-specific QA checks are included in ``metadata/install-qa-check.d``
and ``metadata/postinst-qa-check.d`` directories of a repository.
For an ebuild in question, the repository containing it and its masters are
traversed for QA checks, with priority decreasing with each inheritance level.
The package-installed QA checks are located in ``/usr/lib/install-qa-check.d``
and ``/usr/lib/postinst-qa-check.d``, and are intended to be installed
by packages. The sysadmin-defined QA checks are located
in ``/usr/local/lib/install-qa-check.d``
and ``/usr/local/lib/postinst-qa-check.d``.
QA check script format
--
QA checks are sourced as bash scripts by the Package Manager. QA scripts are
run in an isolated subshell, and therefore can safely alter the environment
and change the working directory. QA scripts must always end with a command
terminating with a successful exit code.
Aside from the standard PMS functions, two additional commands are provided:
Re: [gentoo-dev] Manifest2 hashes, take n+1-th: one hash to decide them all
Hi,
On Wed, 25 Oct 2017 02:40:58 +
"Robin H. Johnson" wrote:
> At that point, and this is a serious proposal:
> The package manager shall decide which hashes to check, but is
> required to check at least one hash. The choice may be 'fastest',
> 'most secure', or any local factor.
Sorry to contribute again to the bikeshedding, but I'd really like to
get one thought across here:
Good security includes reducing complexity. Tough (as evident by this
thread) it's a thought many people find hard to accept.
I don't think this is most important in this discussion, but I feel
it's something people should keep in mind also for other decisions to
be made.
This thread is going into a completely different direction and I find
that worriesome. We have two non-problems ("what if secure hash X gets
broken?" and "what if it's too slow? I haven't benchmarked, but what if
it's too slow??") and people proposing increasingly complex solutions.
If you do what you propose my worries aren't that any hash gets broken
or that it's too slow. It's that some bug will chime in where in some
situation no hash gets checked whatsoever.
Having more than one hash is already unneeded complexity. Nobody does
that. TLS signatures use one hash. GPG signatures uses one hash. Signal
uses one hash. I'm not aware of any credible cryptographic product that
feels the need to have multiple hashes concatenated. (The only real
example I'm aware of is old TLS versions who chose to concat two
insecure hashes - MD5+sha1 - which obviously wasn't the smartest idea
either, but one can credibly say they didn't know better back then.)
Having a situation where one can either check one hash or multiple and
add configurability around that is another step of adding unneeded
complexity.
Also one more comment about the issue with potentially buggy Hash
implementations: I feel this is a software testing problem rather than
anything that should influence our package manager format or be tested
at runtime. Add a self-test of hash functions with a large batch of
test vectors that you can easily run.
--
Hanno Böck
https://hboeck.de/
mail/jabber: ha...@hboeck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42
pgpNTzKQZ2nN3.pgp
Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] GLEP 65 v2: Post-install QA checks (now with post-merge checks)
On 10/25/2017 03:18 AM, Michał Górny wrote: >>> ... >>> The QA checks can inspect the installation image or live system >>> respectively, >> >> Respective to what? > > To the type of check, as explained later? If you want to help, then > please be specific instead of asking questions and expecting me to > figure out how should I change it. > >>> output and store both user- and machine-oriented QA warning logs, manipulate >>> the files and abort the install, as necessary. >>> >> >> An oxford comma would make that easier to read: "files, and abort the >> install as necessary." > > Wouldn't that change the meaning? The point is it can do all those > things as necessary, not just the last. > The grammar is ambiguous but personally I would parse it the way you intend. To address both comments at once, I would change that whole sentence to, The QA checks can inspect the installation image or live system, output and store both user- and machine-oriented QA warning logs, manipulate files, and abort the install. The "can" makes the "as necessary" redundant. Usually "respectively" is used to match up two series of words in a one-to-one fashion, like "spoons and knives are used to eat soup and steak, respectively." Seeing it in the original abstract confused me a little because I couldn't tell what things I was supposed to be matching up in my head. Since the sentence reads fine without it, it can probably be omitted. (I know only one type of check can inspect the live image, but only one type of check can abort the install, too. It's OK to be a little vague in the abstract.) >>> In case of severe QA issues, the checks are allowed to alter the contents of >>> the installation image in order to sanitize them, or call the ``die`` >>> function >>> to abort the build. >> >> I'm not sure that having the image silently modified is a good idea. It >> seems like everyone would benefit more if the QA check crashed, and let >> the maintainer fix his ebuild. Is this already possible with the Portage >> checks, or is it new in the repository-based checks? > > I'm pretty sure this was used somewhere in Portage internally. Anyway, > if you want to change it, then convince people it's fine to add a new > check that is going to cause random packages to suddenly explode for > stable users when the problem can be fixed trivially. I wouldn't suggest that. Instead, we could do something like is happening on https://bugs.gentoo.org/588944 where we fix the packages before introducing the new check (it would have to be a very serious issue to "die" in the first place). If we want to commit the check as soon as possible -- before all of the ebuilds are fixed -- then we could add a whitelist at the top of the QA check script. Bugs would be filed against the packages, and once fixed, we would remove that $CAT/$P from the whitelist. >>> >>> Function specification >>> -- >> >> >> Also appears twice. > > How is that a problem? It appears twice because it strictly corresponds > to the same section in specification. > It's not, so long as it's intentional. Just checking.
Re: [gentoo-dev] [RFC] GLEP 65 v2: Post-install QA checks (now with post-merge checks)
W dniu wto, 24.10.2017 o godzinie 17∶57 -0400, użytkownik Michael
Orlitzky napisał:
> On 10/17/2017 02:12 PM, Michał Górny wrote:
> >
> > Abstract
> >
> >
> > ...
> > The QA checks can inspect the installation image or live system
> > respectively,
>
> Respective to what?
To the type of check, as explained later? If you want to help, then
please be specific instead of asking questions and expecting me to
figure out how should I change it.
> > output and store both user- and machine-oriented QA warning logs, manipulate
> > the files and abort the install, as necessary.
> >
>
> An oxford comma would make that easier to read: "files, and abort the
> install as necessary."
Wouldn't that change the meaning? The point is it can do all those
things as necessary, not just the last.
>
>
> > Motivation
> > ==
> >
> > ...
> >
> > Over time, many different QA checks have been added to Portage. That
> > includes
> > checks corresponding to generic Gentoo rules (like filesystem hierarchy,
> > security requirements), checks enforcing Gentoo team policies and correct
> > eclass uses.
>
> Another weird comma issue, maybe try: "policies, and checks enforcing
> correct eclass usage."
Done.
> > There are two kinds of QA checks defined in this specification:
> >
> > 1. Post-install QA checks (``install-qa-check.d``),
> >
> > 2. Post-merge (postinst) QA checks (``postinst-qa-check.d``).
> >
>
> It's a bit sadistic to have something called a "post-install check" but
> then have "postinst check" refer to the other thing.
Switched to using only post-merge.
> > In case of severe QA issues, the checks are allowed to alter the contents of
> > the installation image in order to sanitize them, or call the ``die``
> > function
> > to abort the build.
>
> I'm not sure that having the image silently modified is a good idea. It
> seems like everyone would benefit more if the QA check crashed, and let
> the maintainer fix his ebuild. Is this already possible with the Portage
> checks, or is it new in the repository-based checks?
I'm pretty sure this was used somewhere in Portage internally. Anyway,
if you want to change it, then convince people it's fine to add a new
check that is going to cause random packages to suddenly explode for
stable users when the problem can be fixed trivially.
> > The post-merge QA checks are executed after the ``pkg_postinst`` ebuild
> > phase
> > finishes. They can use the same commands as are permitted in
> > ``pkg_postinst``,
> > and access the installed system location ``${ROOT}`` and the temporary
> > directory ``${T}``.
> >
> > The checks are allowed to alter the contents of the filesystem to the same
> > degree as ``pkg_postinst`` phase is. They must not call ``die``.
>
> Ditto re: filesystem modification.
This is technically needed for the checks to work. If you don't update
outdated caches, then every single next postinst check will trigger.
>
> > Aside to the standard PMS function
> >
>
> Aside from
Fixed.
> > eqatag
> > ~~
> > Synopsis
> > ``eqatag [-v] [=...] [...]``
> >
> > Tag the package with specific QA issues. The *tag* parameter is
> > a well-defined name identifying specific QA issue. The tag can be
> > additionally
> > associated with some data in key-value form and/or one or more *files*.
> > The file paths are relative to installation image (``${D}``), and need to
> > start with a leading slash.
>
> If this is used in a post-merge check, isn't it conceivable that you
> would want to tag a path outside of $D? For example, if your src_install
> creates ${D}/bin/foo and then your pkg_postinst phase gives ownership of
> /bin/foo to the "foo" user, I might want to tag /bin/foo (and not
> ${D}/bin/foo) with a bad-owner tag.
It's a wording issue. The point is, path is the same as you would pass
to do* etc. Fixed.
> > QA check script format
> > --
> >
>
> This is the second appearance of a "QA check script format" section.
>
>
> >
> > Function specification
> > --
>
>
> Also appears twice.
How is that a problem? It appears twice because it strictly corresponds
to the same section in specification.
--
Best regards,
Michał Górny
