Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v3]

[Michał Górny]

W dniu wto, 21.11.2017 o godzinie 19∶20 +0100, użytkownik Ulrich Mueller
napisał:
> > > > > > On Tue, 21 Nov 2017, Michał Górny wrote:
> > All paths specified in the Manifest file must consist of characters
> > corresponding to valid UTF-8 code points excluding the NULL character
> > (``U+``), the backwards slash (``\``) and characters classified
> > as whitespace in the current version of the Unicode standard
> > [#UNICODE]_. It is an error to use Manifest files in directories
> > containing files whose names contain the disallowed characters.
> > The forward slash (``/``) must be used as path separator.
> 
> In addition to whitespace, you should also exclude C0 controls (U+
> to U+001F), DEL (U+007F), and C1 controls (U+0080 to U+009F).
> 
> Rationale, these control characters can leave the user's terminal
> in an unusable state when a package manager tries to output such a
> filename in a message. As you reserve the backslash for a future
> escape mechanism, this shouldn't be a too severe restriction.
> 

Works for me. I'll update the spec later. Can you think of any other
sequences that should be explicitly forbidden?

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v3]

[Ulrich Mueller]

> On Tue, 21 Nov 2017, Michał Górny wrote:

> All paths specified in the Manifest file must consist of characters
> corresponding to valid UTF-8 code points excluding the NULL character
> (``U+``), the backwards slash (``\``) and characters classified
> as whitespace in the current version of the Unicode standard
> [#UNICODE]_. It is an error to use Manifest files in directories
> containing files whose names contain the disallowed characters.
> The forward slash (``/``) must be used as path separator.

In addition to whitespace, you should also exclude C0 controls (U+
to U+001F), DEL (U+007F), and C1 controls (U+0080 to U+009F).

Rationale, these control characters can leave the user's terminal
in an unusable state when a package manager tries to output such a
filename in a message. As you reserve the backslash for a future
escape mechanism, this shouldn't be a too severe restriction.

Ulrich


pgpzrgA93CWHZ.pgp
Description: PGP signature

Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v3]

[Michał Górny]

W dniu czw, 16.11.2017 o godzinie 11∶19 +0100, użytkownik Michał Górny
napisał:
> Hi, everyone.
> 
> Here's the updated version of GLEP 74 taking into consideration
> the points made during the Council pre-review.
> 
> ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst
> HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html
> 
> Changes:
> 

5ba0654 glep-0074: Specify slash as path separator, disallow backwards
slash
d3b65ba glep-0074: Mention that newline needs to be restricted too in
rationale
54cc3ef glep-0074: Apply suggestions from Ulrich Müller


---
GLEP: 74
Title: Full-tree verification using Manifest files
Author: Michał Górny ,
Robin Hugh Johnson ,
Ulrich Müller 
Type: Standards Track
Status: Draft
Version: 1
Created: 2017-10-21
Last-Modified: 2017-11-16
Post-History: 2017-10-26, 2017-11-16
Content-Type: text/x-rst
Requires: 59, 61
Replaces: 44, 58, 60
---

Abstract


This GLEP extends the Manifest file format to cover full-tree file
integrity and authenticity checks. The format aims to be future-proof,
efficient and provide means of backwards compatibility.


Motivation
==

The Manifest files as defined by GLEP 44 [#GLEP44]_ provide the current
means of verifying the integrity of distfiles and package files
in Gentoo. Combined with OpenPGP signatures, they provide means to
ensure the authenticity of the covered files. However, as noted
in GLEP 57 [#GLEP57]_ they lack the ability to provide full-tree
authenticity verification as they do not cover any files outside
the package directory. In particular, they provide multiple ways
for a third party to inject malicious code into the ebuild environment.

Historically, the topic of providing authenticity coverage for the whole
repository has been mentioned multiple times. The most noteworthy effort
are GLEPs 58 [#GLEP58]_ and 60 [#GLEP60]_ by Robin H. Johnson from 2008.
They were accepted by the Council in 2010 but have never been
implemented. When potential implementation work started in 2017, a new
discussion about the specification arose. It prompted the creation
of a competing GLEP that would provide a redesigned alternative to
the old GLEPs.

This specification is designed with the following goals in mind:

1. It should provide means to ensure the authenticity of the complete
   repository, including preventing the injection of additional files.

2. The format should be universal enough to work both for the Gentoo
   repository and third-party repositories of different characteristics.

3. The Manifest files should be verifiable stand-alone, that is without
   knowing any details about the underlying repository format.


Specification
=

Manifest file format


This specification reuses and extends the Manifest file format defined
in GLEP 44 [#GLEP44]_. For the purpose of it, the *file type* field is
repurposed as a generic *tag* that could also indicate additional
(non-checksum) metadata. Appropriately, those tags can be followed by
other space-separated values.

Unless specified otherwise, the paths used in the Manifest files
are relative to the directory containing the Manifest file. The paths
must not reference the parent directory (``..``).

The Manifest files use UTF-8 encoding.


Manifest file locations and nesting
---

The ``Manifest`` file located in the root directory of the repository
is called top-level Manifest, and it is used to perform the full-tree
verification. In order to verify the authenticity, it must be signed
using OpenPGP, using the armored cleartext format.

The top-level Manifest may reference sub-Manifests contained
in subdirectories of the repository. The sub-Manifests are traditionally
named ``Manifest``; however, the implementation must support arbitrary
names, including the possibility of multiple (split) Manifests
for a single directory. The sub-Manifest can only cover the files inside
the directory tree where it resides.

The sub-Manifest can also be signed using OpenPGP armored cleartext
format. However, the signature verification can be omitted since it
already is covered by the signed top-level Manifest.


Directory tree coverage
---

The specification provides three ways of skipping Manifest verification
of specific files and directories (recursively):

1. explicit ``IGNORE`` entries in Manifest files,

2. injected ignore paths via package manager configuration,

3. using names starting with a dot (``.``) which are always skipped.

All files that are not ignored must be covered by at least one
of the Manifests.

A single file may be matched by multiple identical or equivalent
Manifest entries, if and only if the entries have the same semantics,
specify the same size and the checksums common to both entries match.
It is an error for a single file to be matched by multiple entries
of different semantics, file size or checksum values. It