Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v4]
> On Wed, 22 Nov 2017, Michał Górny wrote: > Path and filename encoding > -- > The path fields in the Manifest file must consist of characters > corresponding to valid UTF-8 code points excluding the NULL character > (``U+``), the backwards slash (``\``) and characters classified > as whitespace in the current version of the Unicode standard > [#UNICODE]_. As I said before, all C0 and C1 control characters and DEL should be excluded as well, i.e. 0x00 to 0x1f, 0x7f, and 0x80 to 0x9f. Allowing such characters in what is basically a text file is only asking for trouble. > Any of the excluded characters that are present in path must be encoded > using one of the following escape sequences: > - characters in the ``U+`` to ``U+007F`` range can be encoded > as ``\xHH`` where ``HH`` specifies the zero-padded, hexadecimal > character code, > - characters in the ``U+`` to ``U+`` range can be encoded > as ``\u`` where ```` specifies the zero-padded, hexadecimal > character code, > - characters in the UCS-4 range can be encoded as ``\U`` > where ```` specifies the zero-padded, hexadecimal character > code. > It is invalid for backwards slash to be used in any other context, > and a backwards slash present in filename must be encoded. Backwards > slash used as path component separator should be replaced by forward > slash instead. This entire section about the escape mechanism should be clearly labelled as being purely optional, as it is not relevant for Gentoo (and would break backwards compatibility with existing package manager implementations). Maybe add a reference to GLEP 31 too? > The encoding can be used for other characters as well. In particular, > escaping control characters is recommended to ensure that the file > works correctly in text editors. See above, this should not be "recommended", but literal control chars should be strictly forbidden. Ulrich pgpgwAnxngceA.pgp Description: PGP signature
[gentoo-dev] Package up for grab: dev-util/rebar
Hello everyone, I'm stepping down as the maintainer of dev-util/rebar, since I haven't been using it for quite a while. I already pointed this out on this very list a couple of months ago (when djc dropped erlang), but since noone else showed interest, I've dropped it to m-n now. rebar itself is EOL, but it can't be lastrited yet due to a number of revdeps. There's a successor being developed at https://github.com/erlang/rebar3, but it's not 100% compatible, so it should probably be added to the tree as a new ebuild or a new slot. Apart from a request to do this, there are no open bugs. @aidecoe: CC'ing you as the maintainer of rebar.eclass. Cheers, Luis Ressel
[gentoo-dev] Re: [RFC, PATCH] user.eclass: gracefully return when unprivileged
On 21-11-2017 20:19:21 +0900, Benda Xu wrote: > Francesco Riosa writes: > > > maybe ewarn() is more appropriate than einfo()? > > Just in case it's executed outside the scope of prefix > > I can't remember any use case when portage (or, paludis, etc.) is > executed as a normal user but not a from Prefix. > > This message will only affect Prefix users, who won't be surprised that > they cannot create new groups or users. Therefore I think einfo is more > appropriate. > > > Furthermore, we do have Prefix that runs as a root, mostly usable on NAS > or smartphones, when we do ultimately like portage to manage groups and > users. I think we could definitely live with this until someone requests otherwise. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: Digital signature
[gentoo-dev] Re: Prefix bootstrap script maintainability (Was: No more stable keywords for Games)
On 20-11-2017 11:53:32 -0600, R0b0t1 wrote: > Hello friends! > > On Monday, November 20, 2017, Sergei Trofimovich <[1]sly...@gentoo.org> wrote: > > On Sun, 19 Nov 2017 22:47:35 -0600 > > R0b0t1 <[2]r03...@gmail.com> wrote: > > > >> Understanding an existing codebase should not be a technical > >> challenge. I had to resort to reimplementing all of the steps myself, > >> in part to understand if they were done properly in the first place. > > > > Looks like you are an expert in this area now and are perfectly capable > > of submitting the patches. I can review them at least for crossdev. > > > > In my goal to understand bootstrap-rap I am still in the process of creating > something crossdev-like that can be used to generate a toolchain. > > A recurring problem I have had is that this set of related tasks - generating > cross compilers and packages, generating an initramfs, or generating a > prefixed > pseudoinstallation - all start by reimplementing some subset of portage. > > For prefix/RAP it makes sense, for the others possibly not. You may also want to understand that cross-compiling (or compilers) in itself is a very difficult topic to get right. Mixing that with Prefix FAICT never got out of the lab-setting in which it was attempted. > >> Unfortunately these are things that the original authors should > >> produce. Experience has shown me that documentation written by > >> ancillary contributors that do not have deep experience with the code > >> base is poor. Like Benda said, documentation can always be improved. In the case for bootstrap-prefix, it used to be documented in terms of steps and why one had to do them that way. Somewhere at the start of 2006, when there was like 150 packages, and one arch (ppc-macos), said script didn't exist. As it stands today, the key decisions and workarounds are actually documented, but as RAP actually shows, if you only focus on a specific use-case, you can get rid of a lot of (what appears to be) nonsense. It's the context in which you look at the projects you refer to. > Yes, that is what I am doing with my own code as I have the time to > write it. I mostly still have no idea what is going on in the already > written code. Perhaps open up the dicussion on the related project's mailing lists. At least I haven't come across any request to explain certain bits/decisions yet. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: Digital signature
Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v4]
W dniu czw, 16.11.2017 o godzinie 11∶19 +0100, użytkownik Michał Górny napisał: > Hi, everyone. > > Here's the updated version of GLEP 74 taking into consideration > the points made during the Council pre-review. > > ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst > HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html > > Changes: > b3964b6 glep-0074: Recommend escaping control characters, suggested by ulm 11f19f9 glep-0074: Provide encoding for disallowed characters da2aace glep-0074: Clarify ignoring directories --- GLEP: 74 Title: Full-tree verification using Manifest files Author: Michał Górny , Robin Hugh Johnson , Ulrich Müller Type: Standards Track Status: Draft Version: 1 Created: 2017-10-21 Last-Modified: 2017-11-16 Post-History: 2017-10-26, 2017-11-16 Content-Type: text/x-rst Requires: 59, 61 Replaces: 44, 58, 60 --- Abstract This GLEP extends the Manifest file format to cover full-tree file integrity and authenticity checks. The format aims to be future-proof, efficient and provide means of backwards compatibility. Motivation == The Manifest files as defined by GLEP 44 [#GLEP44]_ provide the current means of verifying the integrity of distfiles and package files in Gentoo. Combined with OpenPGP signatures, they provide means to ensure the authenticity of the covered files. However, as noted in GLEP 57 [#GLEP57]_ they lack the ability to provide full-tree authenticity verification as they do not cover any files outside the package directory. In particular, they provide multiple ways for a third party to inject malicious code into the ebuild environment. Historically, the topic of providing authenticity coverage for the whole repository has been mentioned multiple times. The most noteworthy effort are GLEPs 58 [#GLEP58]_ and 60 [#GLEP60]_ by Robin H. Johnson from 2008. They were accepted by the Council in 2010 but have never been implemented. When potential implementation work started in 2017, a new discussion about the specification arose. It prompted the creation of a competing GLEP that would provide a redesigned alternative to the old GLEPs. This specification is designed with the following goals in mind: 1. It should provide means to ensure the authenticity of the complete repository, including preventing the injection of additional files. 2. The format should be universal enough to work both for the Gentoo repository and third-party repositories of different characteristics. 3. The Manifest files should be verifiable stand-alone, that is without knowing any details about the underlying repository format. Specification = Manifest file format This specification reuses and extends the Manifest file format defined in GLEP 44 [#GLEP44]_. For the purpose of it, the *file type* field is repurposed as a generic *tag* that could also indicate additional (non-checksum) metadata. Appropriately, those tags can be followed by other space-separated values. Unless specified otherwise, the paths used in the Manifest files are relative to the directory containing the Manifest file. The paths must not reference the parent directory (``..``). Forward slash (``/``) is used as path component separator. The Manifest files use UTF-8 encoding. Manifest file locations and nesting --- The ``Manifest`` file located in the root directory of the repository is called top-level Manifest, and it is used to perform the full-tree verification. In order to verify the authenticity, it must be signed using OpenPGP, using the armored cleartext format. The top-level Manifest may reference sub-Manifests contained in subdirectories of the repository. The sub-Manifests are traditionally named ``Manifest``; however, the implementation must support arbitrary names, including the possibility of multiple (split) Manifests for a single directory. The sub-Manifest can only cover the files inside the directory tree where it resides. The sub-Manifest can also be signed using OpenPGP armored cleartext format. However, the signature verification can be omitted since it already is covered by the signed top-level Manifest. Directory tree coverage --- The specification provides three ways of skipping Manifest verification of specific files and directories (recursively): 1. explicit ``IGNORE`` entries in Manifest files, 2. injected ignore paths via package manager configuration, 3. using names starting with a dot (``.``) which are always skipped. All files that are not ignored must be covered by at least one of the Manifests. A single file may be matched by multiple identical or equivalent Manifest entries, if and only if the entries have the same semantics, specify the same size and the checksums common to both entries match. It is an error for a single file to be matched by multiple entries of different semantics, file size or checksum values. It is an error to specify anot
Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v2]
On Wed, Nov 22, 2017 at 2:02 AM, Michał Górny wrote: > W dniu wto, 21.11.2017 o godzinie 20∶59 -0600, użytkownik R0b0t1 > napisał: >> On Mon, Nov 20, 2017 at 12:42 PM, Michał Górny wrote: >> > W dniu czw, 16.11.2017 o godzinie 11∶19 +0100, użytkownik Michał Górny >> > napisał: >> > > Hi, everyone. >> > > >> > > Here's the updated version of GLEP 74 taking into consideration >> > > the points made during the Council pre-review. >> > > >> > > ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst >> > > HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html >> > > >> > >> > New changes: >> > >> > 9d819c9 glep-0074: Disallow filenames containing whitespace >> >> This seems like a bad idea. I apologize if this is covered in more >> detail somewhere, but the only justification I can see is that the >> current grammar does not permit quoting or some other method of >> specifying whitespace as part of a field value. >> >> Is there any way to assure that this won't break things in a >> non-obvious way? I'm having a hard time imagining how it would be an >> inflexible requirement to use a space in a filename, but it could come >> up if it was necessary to use Portage on a non-Gentoo distribution. > > Having a whitespace there *will* break the parser. Until a better parser > is provided, we need to reject it to prevent tools from accidentally > generating broken files. It's better to tell straight away 'sorry, you > can't use Manifest here' than cause completely unexpected behavior > in the parser. > > Using whitespace in filenames is going to break Portage in horrible > ways. Half of shell script in it is based on whitespace-separated lists. > PMS doesn't provide any means to replace some of them. It's not going to > happen. > Yes, I was talking about providing a better parser. I understand it is as it is now because whitespace is a delimiter. If it's not possible to know where all code that has this as a requirement is, that's fairly bad. http://langsec.org/occupy/ >> It seems very arbitrary. I think the better solution is to use a better >> parser. >> > > The parser is already there for 15 years or more. We can't just replace > it without breaking all old Portage versions. > It sounds like portage is already broken. Cheers, R0b0t1
[gentoo-dev] Last rites: sci-mathematics/qtoctave
# Andreas Sturmlechner (21 Nov 2017) # Totally broken, depends on dead Qt4. Use sci-mathematics/octave[gui]. # Masked for removal in 30 days. Bugs #598930, #615284, #614582, #631524 sci-mathematics/qtoctave
[gentoo-dev] Re: cmake + ninja vs autotools
Georg Rudoy <0xd34df...@gmail.com> wrote: > 1. Doing a full clean build [..] > the speed of make or ninja is hugely offset by the compilation speed, > and their overhead is negligible. It depends on the definition of negligible. For huge projects like boost or chromium it is several minutes: That's why these projects switched to ninja. Admittedly, for eix the difference is probably only a few seconds on most systems.
Re: [gentoo-dev] Packages up for grabs
On 21/11/17 21:01, Manuel Rüger wrote: > Packages up for grabs: > > * app-crypt/yubikey-manager-qt > * sys-apps/etckeeper > * sys-auth/yubico-piv-tool > * dev-libs/libsodium > * app-editors/retext > * sys-apps/rkflashtool > * dev-embedded/esptool > * app-shells/thefuck > * app-crypt/paperkey > * dev-util/bumpversion > * dev-python/python-afl > * dev-python/pyotherside > * dev-util/docker-distribution-pruner > I can proxy etckeeper and esptool as I use both those ... will check for bugs .. MJE signature.asc Description: OpenPGP digital signature
[gentoo-dev] Last rites: dev-perl/Mail-SPF-Query
# Kent Fredric (22 Nov 2017) # Long deprecated upstream in favour of dev-perl/Mail-SPF # Masked for removal in 30 days. Bug #638432 dev-perl/Mail-SPF-Query pgp6PtgkgQgsP.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v2]
W dniu wto, 21.11.2017 o godzinie 20∶59 -0600, użytkownik R0b0t1 napisał: > On Mon, Nov 20, 2017 at 12:42 PM, Michał Górny wrote: > > W dniu czw, 16.11.2017 o godzinie 11∶19 +0100, użytkownik Michał Górny > > napisał: > > > Hi, everyone. > > > > > > Here's the updated version of GLEP 74 taking into consideration > > > the points made during the Council pre-review. > > > > > > ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst > > > HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html > > > > > > > New changes: > > > > 9d819c9 glep-0074: Disallow filenames containing whitespace > > This seems like a bad idea. I apologize if this is covered in more > detail somewhere, but the only justification I can see is that the > current grammar does not permit quoting or some other method of > specifying whitespace as part of a field value. > > Is there any way to assure that this won't break things in a > non-obvious way? I'm having a hard time imagining how it would be an > inflexible requirement to use a space in a filename, but it could come > up if it was necessary to use Portage on a non-Gentoo distribution. Having a whitespace there *will* break the parser. Until a better parser is provided, we need to reject it to prevent tools from accidentally generating broken files. It's better to tell straight away 'sorry, you can't use Manifest here' than cause completely unexpected behavior in the parser. Using whitespace in filenames is going to break Portage in horrible ways. Half of shell script in it is based on whitespace-separated lists. PMS doesn't provide any means to replace some of them. It's not going to happen. > It seems very arbitrary. I think the better solution is to use a better > parser. > The parser is already there for 15 years or more. We can't just replace it without breaking all old Portage versions. -- Best regards, Michał Górny
