Re: [gentoo-dev] GLEP81 and /home
On Sun, Jan 19, 2020 at 2:27 PM Michael Orlitzky wrote: > > On 1/19/20 2:02 PM, Rich Freeman wrote: > > > >> If you're sharing /home, you also have to be sharing user accounts, > >> unless you want everyone to be assigned a random set of files. > > > > I imagine that most people setting up something like this would only > > be sharing high-value UIDs (>1000 in our case). There is no need for > > postfix on your Gentoo box and postfix on your Debian box to have the > > same UID. You wouldn't be sshing from postfix on the one to postfix > > on the other and expecting to have the same home directory contents. > > > > You can't do that. If you're going to mount files from one system onto > another system, using only an integer <--> username mapping as your > access control mechanism, then you'd better be damn sure that those > integers and usernames match on all systems. Otherwise I might wind up > sharing /home/mjo to rich0 because the "mjo" and "rich0" groups both > have gid 1000 locally. Obviously the UIDs associated with the shared /home need to be identical. Simplest solution is to sync anything > 1000 in /etc/passwd, and then not allow UIDs below 1000 in /home. A cron job could easily handle both, and of course regular users can't go creating stuff with the wrong UID anyway. > We've talked this to death. Barring any new evidence, /home still seems > like the best place for these, and I don't want to put them in the wrong > spot (forcing users to migrate) just to appease a QA warning from before > GLEP81 was a thing. Well, great, then by all means ask QA for a policy exception. Not my place to yell at you if you don't, but don't be surprised if somebody else does... -- Rich
Re: [gentoo-dev] New QA Policy Guide
On Sun, Jan 19, 2020 at 2:32 PM Kent Fredric wrote: > > Having a discussion at a bar, and you making a commit as a result is > one thing, but if I discovered a bug, and then only told you about it > at the bar, that would be possibly bad, because there's no guarantee > that the bug is communicated sufficient to ensure it gets addressed, > and you may go home at the end of the night and entirely forget the bug > existed, and people could continue to suffer it, and potentially > neglect to report it as well. ( End users are substantially less likely > to file bugs, IME ) > > When I mention bugs to people on IRC, I often follow up with "Would you > like me to file a bug?". > > Often, the answer is "yes". > > The crux of the matter being bugs that exist, and are communicated > outside the core bug tracker, weaken the assurance that it will be seen > and fixed, which amounts to a negative thing. Oh, I absolutely agree with this. My point is that right now we have no policy that requires bugs to be filed. And hence stuff that happens on github really is no different than your case of stuff happening in a bar. I can't speak for the QA repo, but don't we have a bot that notices open pull requests for the main repo mirror on github which are missing bug references to post notices to this effect? When this started happening I think a lot of the concerns were reduced. I mean, like was already mentioned, if there were a gitlab repo or whatever being hosted a lot of this might become moot. We're just not there yet. I'm not sure if the Foundation has considered approaching gitlab.com about hosting. Granted, that isn't their FOSS product, but I suspect the repos could be exported and imported into the FOSS product as a contingency. I have it on good authority from somebody who works there that their proprietary hosted product is identical to the FOSS one aside from the proprietary modules, so as long as you avoid the latter it should be the same thing. If they're willing to donate or offer cheaper hosting it might give us the benefits of the FOSS repository while avoiding the hassles of hosting Ruby or whatever it is written in. -- Rich
Re: [gentoo-dev] GLEP81 and /home
On Sun, Jan 19, 2020 at 1:37 PM Michael Orlitzky wrote: > > On 1/19/20 12:42 PM, Rich Freeman wrote: > > > > Typically you wouldn't share service accounts across multiple hosts. > > I'd think that something like amavisd is going to go on a mail server. > > You're not going to be logging into that account to do typical > > desktop-oriented functions. > > > > If you had three mail servers, you probably would want to share > > /home/mjo across all of them, but you probably wouldn't want to share > > your amavisd config across them. That is why the config goes in /etc. > > I don't see how stuff it launches would be any different. > > The stuff it launches is different because the stuff it launches is > different. SpamAssassin, for example, can be run by normal users in a > traditional UNIX mail setup. So its configuration goes in $HOME, because > that's how it works. When amavis runs spamassassin, the SA configuration > comes from $HOME, because that's how it works. Sure, I completely understand that and have no issues with it. Ditto with having some apache module running sendmail, which has some plugin which gpg signs emails, which requires a ~/.gnupg for the apache user. > If you're sharing /home, you also have to be sharing user accounts, > unless you want everyone to be assigned a random set of files. I imagine that most people setting up something like this would only be sharing high-value UIDs (>1000 in our case). There is no need for postfix on your Gentoo box and postfix on your Debian box to have the same UID. You wouldn't be sshing from postfix on the one to postfix on the other and expecting to have the same home directory contents. > And if > you're sharing user accounts, you have to start each instance of amavis > as a different user, because its configuration is per-user. That's just > the way it works. Since it is a local account, not in /home, then it would be a separate user even if the UID is the same (or otherwise). You'd set up amavis on each mail server. They might be running different distros. They would be using local users. Don't get me wrong, it would be cleaner if POSIX users had a scope the way that an OS like Windows does it, but it isn't a big deal if you use high-numbered UIDs for shared users, and low-numbered UIDs for local users. > Everything is fine here, this all works and has worked for 20 years. Sure, it works fine if you have a single host, or do nothing to share your home directories, which I imagine is what 95% of Gentoo users do. I doubt most Gentoo users even encrypt /home, even though this has been standard for most of those 20 years on just about every major distro out there. If a user wants to put this stuff in /home we should certainly support that, and it would work fine if the user sets up the account properly before installing the package. They might get a QA warning, but that is the user's concern. -- Rich
Re: [gentoo-dev] New QA Policy Guide
On Sun, Jan 19, 2020 at 1:45 PM Kent Fredric wrote: > > On Sun, 19 Jan 2020 07:08:30 -0500 > Rich Freeman wrote: > > > The official sources aren't in github. A bugzilla component is > > available, so if github goes away there is no problem and we aren't > > relying on it. > > If github goes away after bugs and PR's are filed on github, then that > historical context is lost, and may include the loss of open bugs and > open PRs, which all may still be relevant. Nothing of importance should be stored on github. If you and I have a conversation at a bar, and as a result you decide to make a commit without any useful comments, and then we both retire from the project, just as much information is lost. We don't require anybody to open a bug before making a commit today, so why would we be concerned when non-required outside documentation is stored in github? That is more information than we already require, so if it goes away nothing required by policy is lost. If we made it a policy that all commits required some kind of peer review in bugzilla, then of course we should do the same here. Right now we do not require that background for just about anything the distro does be recorded anywhere. If github's existence bothers you, then just pretend it doesn't exist - stick it in your hosts file or block it at your router. In theory it shouldn't change your Gentoo experience at all. :) -- Rich
Re: [gentoo-dev] GLEP81 and /home
On Sun, Jan 19, 2020 at 10:49 AM Michael Orlitzky wrote:
>
> On 1/19/20 6:29 AM, Rich Freeman wrote:
> >
> > Daemons are local users. There is no guarantee that /home is a local
> > filesystem. Heck, there is no guarantee that /home is even mounted
> > when portage is running. Portage shouldn't be touching /home at all.
> > With stuff like automounted or encrypted home directories and
> > systemd-homed and so on it seems like it is even more important to
> > avoid sticking stuff in /home (and this is hardly something started by
> > systemd - stuff in /home that is non-static has been a thing for some
> > time - certainly it was happening in the 90s on some IRIX workstations
> > I used).
>
> If you're sharing /home, you're also sharing users. At that point, the
> daemon user is no longer local.
Typically you wouldn't share service accounts across multiple hosts.
I'd think that something like amavisd is going to go on a mail server.
You're not going to be logging into that account to do typical
desktop-oriented functions.
If you had three mail servers, you probably would want to share
/home/mjo across all of them, but you probably wouldn't want to share
your amavisd config across them. That is why the config goes in /etc.
I don't see how stuff it launches would be any different.
This is why /root is typically outside of /home as well.
> I like your /var/lib/amavis/{home,work} suggestion second-best, but the
> most appropriate place for the home directory of an account that will be
> used interactively by a human (even if it's also used to start a daemon)
> is under /home. For example I do want to back up that home directory,
> but I don't want to back up the working directory.
Honestly, since you're only using it for what amounts to configuration
it almost makes sense to put it in /etc, and back it up for that
reason.
You don't really want to be using it interactively as a human per-se
any more than you interactively log in as root or any other service
account. There are rare occassions where I'll launch a shell as
apache or postfix or whatever, but that doesn't mean that you want it
to have a home in /home.
> > Portage should provide a safe mechanism to fix permissions. Or we
> > should just avoid nesting user home directories inside directories
> > that will be written to by that user.
> >
> > If this is the same hard-linking concern as with tmpfiles.d then
> > somebody really just needs to fix POSIX. :) But as a workaround just
> > avoiding nesting seems like the simpler solution...
>
> Essentially yes, but hard links aren't the only problem. It's unsafe to
> do anything as root in a user-controlled directory. POSIX can't fix
> that, and that means that portage will never be able to fix permissions
> (or do anything else) within a user-controlled directory safely.
I mean, you're still doing stuff as root. You're just not running chown.
POSIX certainly could fix it, though whether it could do it in a
manner that doesn't break everything in existence is another matter.
For example, if POSIX just got rid of hard links many of the issues
would just go away.
Obviously if the problem had a simple solution it would have been
implemented by now.
BTW, thanks to the recent QA post I can at least point you at
documentation for your issue. Per the article if you want to change
it the procedure is to ask QA for an exception or change in policy,
and if you don't like the answer go to Council...
https://projects.gentoo.org/qa/policy-guide/filesystem.html#installation-paths
--
Rich
Re: [gentoo-dev] New QA Policy Guide
On Sun, Jan 19, 2020 at 6:46 AM Ulrich Mueller wrote: > > > On Sun, 19 Jan 2020, Michał Górny wrote: > > > The sources are stored in proj/policy-guide.git [3]. If you wish to > > submit your own changes, you can either use the 'Policy Guide' bugzilla > > component [4] and/or GitHub mirror [5]. > > Please, no github for official policies. We should have a permanent > paper trail for this kind of things, which isn't guaranteed if the > discussion would happen entirely on github. > > Besides, by the Social Contract we cannot rely on a non-free service > for anything official. The official sources aren't in github. A bugzilla component is available, so if github goes away there is no problem and we aren't relying on it. It looks like there is the optional ability to do work on github, just as people can optionally talk about anything, anywhere. If I have a chat with another package maintainer at a bar, and they modify their ebuild and push that to the Gentoo repo on infra, and no bug is ever opened, that is 100% within our current policy. I don't see how having that discussion on github instead of at the bar changes things. They're just offering an alternative place to get things done. Anybody who wants to could just file a bug instead. If we want to have an additional Gentoo policy that nobody is allowed to discuss a Gentoo policy outside of the lists and bugzilla that would of course create issues with stuff like github, and probably non-logged IRC channels and private messages as well. However, that is not our current policy. Plenty of council decisions happen with much of the actual discussion not being recorded anywhere. I'm not sure you could reasonably operate in any other way, as people do need the ability to talk things out without having to posture. I feel like this discussion has already happened in the past though... -- Rich
Re: [gentoo-dev] GLEP81 and /home
On Sat, Jan 18, 2020 at 9:50 PM Michael Orlitzky wrote: > > On 1/18/20 7:21 PM, Rich Freeman wrote: > > On Sat, Jan 18, 2020 at 6:38 PM Michael Orlitzky wrote: > >> > >> But now users have to follow one more step (create /home/amavis) when > >> setting up amavisd-new. Is the QA check really assuring a quality user > >> experience here? > >> > > > > Lots of daemons need a home directory for their users, and usually > > they manage to get by in /var/lib. It really seems like a bad > > practice to start having packages creating stuff in /home. Certainly > > I don't want random daemons sticking stuff in /home, which I manage > > > > * The daemon DOES NOT need a home directory for its user. > * I DO NOT want to install anything to anyone's home directory. My concern isn't with installing stuff to "anyone's home directory." My concern is with creating ANYTHING in /home. Daemons are local users. There is no guarantee that /home is a local filesystem. Heck, there is no guarantee that /home is even mounted when portage is running. Portage shouldn't be touching /home at all. With stuff like automounted or encrypted home directories and systemd-homed and so on it seems like it is even more important to avoid sticking stuff in /home (and this is hardly something started by systemd - stuff in /home that is non-static has been a thing for some time - certainly it was happening in the 90s on some IRIX workstations I used). > * With respect to user.eclass, I'm proposing that /home be treated > EXACTLY THE SAME as it always has been. I'm not aware that it was ever considered good practice to stick stuff in /home. I suspect it just wasn't on QA's radar in the past. > > All I want to do is *set* a user's home directory to /home/foo. You don't just want to "set" it. You want to create the directory, which is modifying the filesystem (otherwise, why have a .keepdir?). And setting a home directory to something that doesn't exist doesn't seem like an improvement unless it is /dev/null. I get though that the daemon itself doesn't use the home directory, and that you just want it for configuring other stuff it might launch. > Spamassassin itself is a better example than amavis. We already set the > spamd user's home directory to /home/spamd with user.eclass. We don't > install anything there, and this works fine. If a human logs into that > account and generates some configuration in ~/.spamassassin, then it's > within the spirit of the FHS to have that data stored in /home. Looks like we might have another package to deal with, and perhaps others as well, depending on what comes out of this thread. > Now, with GLEP 81, we get a QA warning for that, because the eclass > installs a keepdir file there. I would like things to remain exactly as > they are, with no QA warning. Otherwise, we have to tell users to move > /home/spamd to /var/lib/spamd-home or something stupid like that. I can > think of no good reason to do so. Well, you won't get QA warnings from the tinderbox if the default home isn't under /home. Users who already have the home there might get warnings at install time. They can just ignore them. You could output an einfo to explain the situation to the user. If they're fine with /home then they can ignore the warning, and if they want to move it they can do so. I'll also note that GLEP 81 itself is silent on WHERE home directories should be placed. If some sort of definitive answer comes out of this thread the GLEP should probably be updated to reflect this, unless there is a better place to put it. Seems like the fact that this practice was undocumented in the past is part of how we got to where we're at. > > It seems like the straightforward solution is to stick everything in > > /var/lib/amavis, and fix things so that everything has the right > > permissions regardless of install order. > > Please stop telling people to do this. Calling chown on the live > filesystem is rarely safe, and I already fixed one root exploit (bug > 630836) in the amavisd-new ebuild from the last guy who tried to adjust > wrong permissions after the fact. That bug appears to be restricted. Perhaps it is embargoed? If not, then it probably shouldn't be restricted, especially if already fixed and GLSA'ed (and if it wasn't GLSA'ed then it isn't fixed, not that this has anything to do with you most likely). Portage should provide a safe mechanism to fix permissions. Or we should just avoid nesting user home directories inside directories that will be written to by that user. If this is the same hard-linking concern as with tmpfiles.d then somebody really just needs to fix P
Re: [gentoo-dev] GLEP81 and /home
On Sat, Jan 18, 2020 at 6:38 PM Michael Orlitzky wrote: > > But now users have to follow one more step (create /home/amavis) when > setting up amavisd-new. Is the QA check really assuring a quality user > experience here? > Lots of daemons need a home directory for their users, and usually they manage to get by in /var/lib. It really seems like a bad practice to start having packages creating stuff in /home. Certainly I don't want random daemons sticking stuff in /home, which I manage differently from the OS-owned directories. I'll just end up having to manually create stuff where it belongs in /var/lib and then symlink everything back from /home, and now I have distro cruft in /home and non-distro cruft in /var/lib, and neither is ideal. It seems like the straightforward solution is to stick everything in /var/lib/amavis, and fix things so that everything has the right permissions regardless of install order. If /var/lib/amavis is getting installed root-owned then it should be chowned when amavis is installed, especially for the first time. That seems sane. Another option is to have /var/lib/amavis/home and /var/lib/amavis/work. -- Rich
Re: [gentoo-dev] Vanilla sources
On Sat, Jan 4, 2020 at 3:13 PM Christopher Head wrote: > > > Of course this would be a bad argument if V-S were lagging behind upstream > significantly, and it’s a much better argument for packages that come with > expectations of security team support than those that don’t, but it is > something to consider. > This was my main concern when it was mentioned that it wasn't security-supported. If it is always up-to-date that definitely helps mitigate things. Though, there should definitely be some kind of warning on the package that it isn't security supported. Even if it is up to date it won't get GLSAs and GLSA-checker won't work. Though, that really only makes a difference insofar as the GLSAs are also timely. In any case, if the just-announced distribution kernel project takes off and remains active I could easily see that becoming the most commonly used kernel option. I'm not knocking minimal kernels but I suspect a LOT of users are going to be well-served by a modular kernel that just works 99% of the time. -- Rich
Re: [gentoo-dev] Vanilla sources
On Sat, Jan 4, 2020 at 6:42 AM Roy Bamford wrote: > > On 2020.01.04 11:01, Rich Freeman wrote: > > > > Is there some reason that we should keep vanilla sources despite not > > getting security handling? > > > > Gentoo had this discussion before. The outcome was that > vanilla-sources is just as Linus intended. > If Gentoo did anything to it, it wouldn't be vanilla any longer. Obviously. I wasn't suggesting that we keep vanilla sources but not make them vanilla. That doesn't mean that they couldn't be security-supported, or that we have to have them in the repo. > Yes, it should be kept. We should not force users to learn > git or tar. Uh, all it does is install kernel sources. They're useless unless you build a kernel using them. Apparently git and tar are too complicated for Gentoo users, but managing symlinks, using make, managing a bootloader, dealing with the kernel's configuration system, and so on are just fine? I completely get the point of the distribution kernel project that was just announced, as I already said. > I agree git or a tarball of vanilla-sources is faster and more > efficient but that's not a reason to drop it. > By the same argument we could drop linux-firmware too. > There are probably other packages that only install whatever > they fetch. Could they be dropped? So, a few issues with that argument: 1. Those other packages are security supported. 2. Those other packages are largely functional once installed, and to the degree that they require configuration that is generally one-time and after updates they remain functional. All that said, it seems like vanilla-sources is pretty up-to-date, so I'm not sure what we mean by it not being security supported. I just took that as a given. Does that mean that we're not releasing patches before upstream? If so, that seems like a pretty minor issue since upstream generally does security bumps pretty quickly. 4.4.208 isn't in our repo but was released today - I'm not sure how quickly these get bumped. If our repo could be days behind that is definitely another reason not to host this stuff, as users should be directed upstream if our packages aren't security supported. On a further aside, I just noticed how up-to-date gentoo-sources are. Kudos to whoever is doing that these days - for a while it was tending to slip a bit but it seems like we're basically current. -- Rich
Re: [gentoo-dev] Vanilla sources
On Fri, Jan 3, 2020 at 11:28 AM Aaron Bauman wrote: > On January 3, 2020 9:55:31 AM EST, Michael Orlitzky wrote: > >On 1/3/20 9:52 AM, Michael Orlitzky wrote: > >> > >> But here we are. Do we make OpenRC Linux-only and steal the fix from > >> systemd? Or pretend to support other operating systems, but leave > >them > >> insecure? > >> > > > >Or the gripping hand: rewrite opentmpfiles in C, so that it's only as > >insecure as checkpath. > > > >Every option sucks. I was only trying to point out that vanilla-sources > >gets no security support -- security@ has stated this, but it's on a > >private bug, so I won't quote it -- and the risk is more than academic. > > This should be known. Security does not support vanilla-sources. This is one > reason vanilla-sources are not stabilized. > Packages without security support should be masked. Really I don't see the point of even having this in the repo. I run vanilla sources personally but I just get them from upstream. Makes way more sense than worrying about whether the version in the repo is up to date for the longterm kernel I'm following. People running vanilla sources are probably using out-of-tree modules (like me) and as such are going to have particular requirements around how they're updated. So, Gentoo is adding fairly little value. All they do is download sources anyway, which is trivially done from git more efficiently (or tarballs that are probably easy to obtain just as efficiently). I can see more of the point in the new distribution kernel project which will be turnkey. I can see some of the value in gentoo-sources (particularly as the upstream for the distribution kernels) especially if they're tied to Gentoo-specific bugs. For more general bugs that apply to all distros I really don't see the point in trying to compete with the upstream stable branches (if they're taking forever to merge a patch, chances are there is a reason for it, and I'm skeptical that Gentoo users are special in some way). Is there some reason that we should keep vanilla sources despite not getting security handling? -- Rich
Re: [gentoo-dev] Vanilla sources
On Fri, Jan 3, 2020 at 9:41 AM Michael Orlitzky wrote:
>
> On 1/3/20 9:40 AM, Toralf Förster wrote:
> > On 1/3/20 3:37 PM, Michael Orlitzky wrote:
> >> The gentoo-sources aren't 100% safe either, but the exploitable scenario
> >> is less common thanks to fs.protected_{hardlinks,symlinks}=1.
> >
> > But this can be easily achieved w/o installing gentoo-sources, or?
> >
>
> Yes, if you know how to do it. And the hard part: if you know that you
> *should* do it.
>
If OpenRC contains a vulnerability wouldn't it make more sense to set
this as part of OpenRC, then to assume somebody is running a kernel
patch that does it, especially since OpenRC doesn't in any way ensure
that gentoo-sources is actually being used?
Of course, fixing the vulnerability seems like a better option. At
least on Linux based on your one bug description it sounds like
systemd has a Linux-specific fix already. Obviously it would be best
to secure this on all kernels but there is no reason not to at least
use that fix on Linux. You could also try to convince the entire
world not to use tmpfiles.d but since it is only a problem if you
aren't using systemd I suspect you won't get much traction there.
In any case this seems more like an OpenRC issue than a Gentoo issue.
--
Rich
Re: [gentoo-dev] Needs ideas: Upcoming circular dependency: expat <> CMake
On Fri, Dec 20, 2019 at 8:41 AM Gerion Entrup wrote: > > Am Donnerstag, 19. Dezember 2019, 19:43:37 CET schrieb Sebastian Pipping: > > On 19.12.19 18:37, Michał Górny wrote: > > > We have a better alternative that lets us limit the impact on the users. > > > Why not use it? > > > > Which one? The CMake bootstrap copy? The adding to stage3 one? > > If an error message can be shown, maybe this is enough as a hint: > "expat and cmake have circular dependencies. Emerge it the first time with: > USE=bundled-cmake emerge -1 cmake expat > and then just don't care anymore about this use flag." > Not sure about custom error messages but in any case when using that bundled-cmake USE flag it would probably make sense to do an ewarn that the bundling took place and is only intended for temporary use, and that the package should be re-merged without the flag once expat is working. That would also cover users who inadvertently set the flag. I think that bundled-cmake also might make more sense than system-cmake, even though the latter is more established. We have a lot of users who set USE=-* and that means they're going to end up with lots of stuff bundled that doesn't need to be. I'm no fan of USE=-* in general but it seems like we should try to avoid making not setting a USE flag the less secure option since it does happen a lot. I'm just commenting on the USE-based solution. The compat package solution obviously bypasses this particular problem entirely. -- Rich
Re: [gentoo-dev] [EAPI 8 RFC] Selective fetch/mirror (un-)restriction
On Mon, Dec 16, 2019 at 8:33 AM Ulrich Mueller wrote: > > > On Mon, 16 Dec 2019, Francesco Riosa wrote: > > > what about getting rid of RESTRICT="fetch" and manage everything > > inside SRC_URI? Would that be technically feasible? Ideally marking > > only the not re-distributable download and leaving untouched the > > others > > That would have the disadvantage that mirror and bindist restrictions > (which are strongly correlated) would be listed in different places. An obvious way to address that is to also move the mirror restriction into SRC_URI, so that they are both exclusively in this location as well. I believe those are the only two redistribution-oriented options for RESTRICT currently. -- Rich
Re: [gentoo-dev] [RFC] Revisiting GLEP 81 (acct-*) policies (reviews, cross-distro syncing)
On Tue, Dec 10, 2019 at 9:50 AM Michael Orlitzky wrote: > > For esoteric packages with a dedicated user, though, you're probably > right. The main benefit of the mailing list posts so far is that they > let me track down pull requests and suggest that people ignore the > example in the devmanual. > Do the list reviews really put people off that much? It seems like eclasses. Plenty of packages have one-off eclasses that nobody cares about except the specific project, in which case the list posts are just a formality and largely a NOOP. However, this list isn't really high-traffic. Ditto with last-rites and so on. I think having the opportunity for review is probably worth it even if often it is just a NOOP. If people are afraid to post something for review because of potential criticism then maybe we need to work more to make sure people understand that everybody makes mistakes and nobody knows everything, and this is why we have reviews in the first place. Nobody is going to have their commit access removed because they didn't notice something and were thoughtful enough to get more eyes on it before commiting it. IMO that is a sign of responsible commit access. -- Rich
Re: [gentoo-dev] [RFC] Revisiting GLEP 81 (acct-*) policies (reviews, cross-distro syncing)
On Tue, Dec 10, 2019 at 8:25 AM Thomas Deutschmann wrote: > > On 2019-12-10 13:44, Rich Freeman wrote: > > I'm not talking about container-host mapping. I'm talking about > > building the same container 100 times and having the container end up > > with the same UIDs inside each time. > > > > Build order in portage isn't really deterministic, especially over > > long periods of time, so you can't rely on stuff getting installed in > > the same order. > > Assume you are building a container for dev-db/mysql. I can only think > of one scenario where you would end up with different UIDs: That's when > dev-db/mysql (or a dependency) would suddenly create an own user and > will be merged before mysql's user was created. > > But this is very theoretically. Especially in a container world, you > will create one container per services so it's *very* unlikely that > something like that will ever happen. Not? There are cases where you might have more than one service in a container, and there is certainly the issue of dependencies. It certainly makes sense to limit a container to a single function, but internally that could involve multiple processes. > Aside benefits from reproducible builds in general (which Gentoo doesn't > provide), please share reasons why one would care about used UIDs/GIDs > in containers... Well, that is simple. In the mysql example /var/lib/mysql would be bind-mounted from outside the container, since it needs to be persistent anytime the container is updated. If you update the container and it ends up with a different UID for mysqld, then it wouldn't be able to read anything in /var/lib/mysql as it would still have the previous UID. You'd need to keep the two in sync somehow. In fact, that is the very example you go on to offer... > > Uh, the container processes shouldn't even see the host > > processes/files whether they have the same UIDs or not... > > Especially when you put mysql or any other service using data into a > container, service running in that container must be able to access this > data. And one common way to do that is allowing container to access data > stored on host, i.e. > > > $ docker run \ > > --name some-mysql \ > > -v /my/own/datadir:/var/lib/mysql \ > > -e MYSQL_ROOT_PASSWORD=my-secret-pw \ > > -d mysql:tag > > which will make /my/own/datadir from host available in container as > /var/lib/mysql. > This is obviously exactly how you'd do it if you were using docker, but you don't need to keep the UID in the container in sync with anything else in the host. If security is a concern you'd probably want to make sure that nothing non-root can reach the directory since its UID might be in use for something else. In any case, this is why consistent UIDs in scratch installs are useful. When you're building a new version of a container you don't want all its UIDs to change. And of course this isn't just limited to containers, but anything where you have persistent storage. It is just that the idea of creating new instances from scratch instead of updating them in-place has become more fashionable around the same time that containers have become more fashionable. You could do the same thing with a bare-metal host, though it would be a bit more painful (perhaps using A/B partition booting, or less painful if you're booting from a SAN or something like that). -- Rich
Re: [gentoo-dev] [RFC] Revisiting GLEP 81 (acct-*) policies (reviews, cross-distro syncing)
On Tue, Dec 10, 2019 at 7:26 AM Thomas Deutschmann wrote: > > On 2019-12-10 12:47, Rich Freeman wrote: > > Having UIDs chosen completely at random seems fairly non-optimal. > > Suppose you're building containers/etc and then bind-mounting in > > persistent storage (/var/lib/mysql and so on). Wouldn't it be nice if > > the default were that mysql would get the same UID on every build? I > > guess you could provide an initial /etc/passwd on every fresh build > > but it just seems like an extra step. > > In practice you will *never* assume proper container <> host user > mapping. *Never*. If you do that, you are doing it wrong: I'm not talking about container-host mapping. I'm talking about building the same container 100 times and having the container end up with the same UIDs inside each time. Build order in portage isn't really deterministic, especially over long periods of time, so you can't rely on stuff getting installed in the same order. > - Container sometimes switch base images. You won't notice that unless > you follow container provider very closely. But you are using container > because you are focused on containerized application, not the container > itself... I'm talking about Gentoo containers here that YOU are the one building. Not just doing "docker run foo." Obviously if you're using somebody else's images you're going to end up with whatever UIDs they use. Chances are they're from some distro that actually DOES manage their UIDs so they'll still be stable over time unless the base image changes as you say. > - Your host is maybe running some real services. You really don't want > that a container suddenly become able to access these services just > because container <> host mapping has match. Uh, the container processes shouldn't even see the host processes/files whether they have the same UIDs or not... -- Rich
Re: [gentoo-dev] [RFC] Revisiting GLEP 81 (acct-*) policies (reviews, cross-distro syncing)
On Tue, Dec 10, 2019 at 12:44 AM Joonas Niilola wrote: > > Honestly I'd say just put -1 on all acct- packages then let sys admins > modify them locally to whatever they need. I feel like this whole GLEP > just serves the minority while making many other contributors uneasy. > I think we're worrying far too much about people with decade-old installs. Just come up with a reasonable set of defaults and as long as it can adapt to whatever is already in /etc/passwd we're fine. Having UIDs chosen completely at random seems fairly non-optimal. Suppose you're building containers/etc and then bind-mounting in persistent storage (/var/lib/mysql and so on). Wouldn't it be nice if the default were that mysql would get the same UID on every build? I guess you could provide an initial /etc/passwd on every fresh build but it just seems like an extra step. This isn't about serving the minority so much as not letting the perfect be the enemy of the good. Yes, there are reasons why GLEP 81 won't be perfect. That doesn't mean that it isn't a good idea... -- Rich
Re: [gentoo-dev] unsanctioned python 2.7 crusade
On Fri, Dec 6, 2019 at 8:06 AM Thomas Deutschmann wrote: > > Sure, if packages don't work anymore or are blocking something, we will > start last-rite process. But for the sabnzbd example (I haven't looked > closely on any other package from that list) there isn't anything > blocking and it's a working piece of software. The only thing which > stands out is: It's a Py2-only package. > Well, that's simple enough. If the python maintainers intend to remove python2 then they need to remove anything that depends on it at the same time. Otherwise all those packages are going to break anyway and users just end up with a mess of error messages due to a broken depgraph. That said, as I've already commented I think it makes more sense to mask the reverse dependencies at the same time as masking python2 itself. And of course for something this big it wouldn't have hurt to announce the plans and what was going to get masked so that mistakes could get caught. Even though it is just a mask it is still a bit disruptive to have packages masked/unmasked because of incorrect identification of reverse/optional deps. Ultimately though it is up to the python2 maintainers to decide when they want to remove it. If others want to step up and replace them as python2 maintainers and they have a reasonable plan for keeping it working that would seem like the approach that would make the most people happy. We can't force people to maintain python2 if they don't want to. -- Rich
Re: [gentoo-dev] unsanctioned python 2.7 crusade
On Thu, Dec 5, 2019 at 5:23 PM David Seifert wrote: > > And that's exactly the straw-man argument I've been making. You can > always come up with an excuse to delay action on python 2, because > "someone, somewhere, will maintain it". Hey, if somebody actually does want to maintain it I don't see any reason it can't stick around forever. Of course maintain means maintain, not just ignore bugs/etc if it causes grief for other packages and so on, or allow security issues to fester. So far I'm getting the impression that everybody wants somebody else to maintain it, and that is when it becomes an issue. "WE ought to do this" - where "WE" usually means "not me." There is no nebulous "Gentoo" out there who will maintain ebuilds. If they are to stay in the repo then somebody has to actually tend them. If somebody wants to keep around 2.7 for a long time IMO the most straightforward thing to do is announce a desire to do it with a plan. I really doubt that anybody is likely to interfere, and if they do it can always be escalated to Council. But, again, it has to be done right and not cause issues for other packages (at least at the start that shouldn't be a huge problem). Personally I've always admired the Wikipedia policy: https://en.wikipedia.org/wiki/WP:BOLD If you want to see a change, go about it in a positive way. If such a change bothers you, assume good faith, but point out the issues. Don't over-react in either direction. This is how 99% of everything positive that has ever happened in Gentoo has come about. Most of our improvements are the result of "unsanctioned crusades." That doesn't mean that you should go around breaking systems left and right, but in this case we're just talking about a mask, or announcing an intent to do a project. But, such a thing will certainly involve work. IMO it is work for diminishing returns. If it is an itch that bothers you, though, you can always scratch it... -- Rich
Re: [gentoo-dev] unsanctioned python 2.7 crusade
On Thu, Dec 5, 2019 at 8:59 AM Jason A. Donenfeld wrote: > > It's quite another to mask random packages that have USE flags to > optionally support whatever python 2.7 library. If you're going to > last rites these, talk with the maintainer first, and only then, send > emails one at a time. Doing that en masse isn't appropriate. ++ - I have no idea if that happened. For anything USE-controlled it would make more sense to file a bug or mask the package-flag combo itself. > > On another topic, I'd prefer for python 2.7 not to be removed from > gentoo. Tons of code still uses it. > I'm sure a million people would share that preference. I'm not sure what the upstream/security status is of 2.7. Obviously to keep it around it would need to be reasonably secure, and somebody within Gentoo would have to want to maintain it. That's basically the criteria for keeping anything like this around. If somebody stepped up and said "I'm maintaining 2.7 and here is why it will remain secure..." I doubt they'd get a lot of resistance. -- Rich
Re: [gentoo-dev] unsanctioned python 2.7 crusade
On Thu, Dec 5, 2019 at 8:42 AM Jason A. Donenfeld wrote: > > Hi, > > Aaron has marked tons of important and useful Python 2.7 packages for removal: > > Can we not do this prematurely? I've revered this commit until such a > thing an be appropriately agreed upon. Might make sense to wait to mask them at the same time as masking python 2.7 itself? Maybe file a bug if not already done to make maintainers aware that this is coming? I assume the python team is the one deciding when python 2.7 has to go (after all, who else is going to maintain it?). The fact that this is about a month off did come up in another recent thread but I don't think it was intended as a formal announcement. -- Rich
Re: [gentoo-dev] Why adding python3_8 to Gentoo sucks?
On Fri, Nov 15, 2019 at 3:05 AM Michał Górny wrote: > > On Wed, 2019-11-13 at 22:16 +0100, Michał Górny wrote: > > I'd like to share my frustration at the state of Python in general, > > and Python packages in Gentoo. So I'd like to 'bootstrap' python3_8 -- > > that is, add it to the most common dependency, dev-python/setuptools. > > Simple thing, right? > > > > So I went with plan B instead: I'll do as much testing locally > as possible, and add py3.8 when I manage to get the tests on the package > in question working, independently of the testing of all deep test deps. > This will mean that some packages will have tests disabled temporarily > for end users. > Perhaps an overlay would be simpler just so that you can generally avoid worrying about QA until you're tidying up, but otherwise this seems like it could be done in-tree by just masking the use flag so that only those willing to test/contribute run into issues. You've described a number of issues and my sense is that many are just inherent to python itself (the complex dep graph/etc - unless we want a monolithic package). Some of course go to Gentoo practices, some of which cause pain outside of python. In particular it seems like many still don't understand when revbumping is necessary. I'd have to dig up the wording of the actual decision but as I recall when the Council made the decision they wanted to leave a bit of room for maintainer discretion, trusting that maintainers would use it properly. An alternative proposal was to just make a strict rule that would have erred on the side of QA, at the cost of extra rebuilds for users (but at least consistent ones that didn't break package managers). Obviously developers can't exercise proper discretion if they don't fully understand the impacts. If in doubt a revbump should always be safe, just at the cost of some rebuilds (which are probably cheap for python packages). -- Rich
Re: [gentoo-dev] RFC: Require full $P not just $PN on stable/keyword commit messages
On Fri, Nov 1, 2019 at 5:34 PM Michael 'veremitz' Everitt wrote: > > > git log --format=oneline glibc-2.29-r2.ebuild | grep stable > > > How well does git handle that when the ebuild is deleted from the tree? > git log --format=oneline -- glibc-2.29-r4.ebuild 23d1c015230d9ff44bcdd7b72e00ca3533815fa4 sys-libs/glibc: drop old f3872a506edc7da0d987bcf0a90d4709945328a7 sys-libs/glibc: restore strip quirk for 'libpthread.so.0' 650d70eb5d91265329e2f730bc1aed0fa5863db6 sys-libs/glibc: disable stripping for cross-glibc e14229b10b513a164f8379ff14cc8c644c071f27 sys-libs/glibc: drop prepallstrip, bug #587296 fb2fe75af62ad29a44aeba1b8e9e41ce5acb3992 sys-libs/glibc: Add 2.29 revision with compile-locales support I was too lazy to find something that had stable keywords, but I'm sure substituting the appropriate filename and grepping would work fine. The trick is to put the "--" in the command line. However, you could hardly be blamed for hitting your head against the wall trying to figure out how to do this. I had to google it as I've run into this myself. As many have said git is an amazing data model with a terrible UI. -- Rich
Re: [gentoo-dev] RFC: Require full $P not just $PN on stable/keyword commit messages
On Fri, Nov 1, 2019 at 4:36 PM Matt Turner wrote:
>
> On Fri, Nov 1, 2019 at 12:59 PM Michael 'veremitz' Everitt
> wrote:
> >
> >
> > Therefore, it would be much /more/ useful to have the package-version
> > tagged in the commit message, so that you could easily grep logs for when a
> > given version of a package was stabilised, and/or keyworded.
git log --format=oneline glibc-2.29-r2.ebuild | grep stable
9c04d06d06d51d9c76b3fe5ceb573213769f45ae sys-libs/glibc-2.29-r2: sparc
stable, bug 685818
b61ab167e82261ed2078c068ba0c2fc3a7b58aa3 sys-libs/glibc: stable
2.29-r2 for hppa, bug #685818
fad52f75c759ca326ce0f8c37e227827f01cd2f1 sys-libs/glibc: m68k stable
wrt bug #685818
0fe91535a7ba382f10084def5482e61359f201cb sys-libs/glibc: sh stable wrt
bug #685818
7b7ec9a6b3355d6111e1a449ca13e24cb6ef0295 sys-libs/glibc: s390 stable
wrt bug #685818
bcddad6780ead2b44528a4aa1d51107b4a225524 sys-libs/glibc-2.29-r2: alpha stable
2ca6a4b9d647f567d2300e7b90829993d7575b41 sys-libs/glibc: ia64 stable
wrt bug #685818
e56c3c1f1c0a256c228a59be94869751d7fd31d7 sys-libs/glibc: ppc64 stable
wrt bug #685818
52355459ec00b9ca9921bd5f788bad9b95346910 sys-libs/glibc: ppc stable
wrt bug #685818
745b07e84b5035576737d3e1a719121d02e53feb sys-libs/glibc: arm stable
wrt bug #685818
332fc91e3e72a6dd1b183ce4a19d08b45daa8e00 sys-libs/glibc: x86 stable
(bug #685818)
9e06c1242e104b66a532e7d5d919c1b3b1f8343d sys-libs/glibc: arm64 stable
(bug #685818)
b3ad265998a04a40820d078d25c06b7cb51173ef sys-libs/glibc: amd64 stable
wrt bug #685818
Seems to work fine for me.
>
> In the past people have argued that the version in the title is
> superfluous since you can get the same info from git (log|show) --stat
> but the same (misguided argument) can be used to justify something
> absurd like simply making the bug number the subject.
I don't think these are at all equivalent. The current state just
relies on finding version history in git, which is basically the main
purpose git serves. Your example involves joining two disparate
datasets, neither of which natively offer an SQL-compatible interface.
I think the rationale for not putting more mandatory content in the
commit summary was that its length is limited and the more boilerplate
we cram in there, the less room we have for meaningful description,
when the boilerplate is already easily searched using git (though
admittedly not from changelog extracts). Sure, for stable/keyword
changes there isn't much in the way of description to worry about, but
for other changes I suspect that this could be limiting.
>From GLEP 66: The summary line must not exceed 69 characters, and must
not be wrapped.
In the example I cited the longest summary is already 59 chars:
sys-libs/glibc: Fix handling of ${EPREFIX} when building cross-glibc
If you wanted to stick 7 more chars of PV info in there then we'd be
just 3 chars short of the limit, and this is just a randomly chosen
example. Packages with longer PV certainly exist, along with ones
with longer summaries.
Personally I don't really care that much one way or another as long as
repoman is updated to follow any new standard, but this seems like it
could be impactful to people doing more complex work in the tree. I
get that some people really seem to want to avoid using git, but this
is basically what it was made to do, and IMO seems like a step in the
wrong direction. I think the main purpose of putting PN in there at
all was so that commits that hit multiple packages would be more clear
in logs.
--
Rich
[gentoo-dev] Re: [RFC] News Item: Desktop profile switching USE default to elogind
On Sun, Oct 27, 2019 at 10:19 PM Andreas Sturmlechner wrote: > > Enter the elogind project [2], which is a standalone logind implementation > based on systemd code, currently maintained by a fellow Gentoo user. A few minor comments: 1. While it is somewhat implicit in the headers, you might want to mention in the text that this will not impact systemd profiles and that those will just use logind. It is a natural question people will end up asking anyway. 2. As evidenced in [1], many users probably have no idea what consolekit, logind, or elogind actually do (or policykit/dbus and a bunch of other modern desktop-oriented tooling that wasn't around back when we were all editing X11 mode lines). You might want to just toss in a sentence or two to explain that as background, since people are going to worry especially with the dreaded s-word in there. Or find some website that explains what it is reasonably well and link it with a note in the text... 1 - https://archives.gentoo.org/gentoo-user/message/8dae2579be22c206d0f4bded84154f2d -- Rich
Re: [gentoo-dev] New distfile mirror layout
On Mon, Oct 21, 2019 at 12:42 PM Richard Yao wrote:
>
> Also, another idea is to use a cheap hash function (e.g. fletcher) and just
> have the mirrors do the hashing behind the scenes. Then we would have the
> best of both worlds.
I think something that is getting missed in this discussion is that we
don't control all of our mirrors, and they're generally donated
resources. Somebody has some webserver, and they stick a Debian
mirror in one directory tree, and an Arch one in another, and they're
kind enough to give us one too.
That is why we're seeing odder situations like ntfs and so on being
mentioned. They're not necessarily even running Linux, let alone zfs
or some other optimized filesystem. And their webserver might be set
up to do browsable directory indexes which could perform terribly even
if the filesystem itself is fine with direct filename lookups. It
doesn't matter if you have hashed b-trees or whatever for filename
lookups if you're going to ask the filesystem to give you a list of
every file in a large directory - it is going to have to traverse
whatever data structure it uses entirely to do so.
If we want to start putting requirements on hosting a mirror, then
we'll end up with less mirrors, and with mirrors more is usually
better. Ideally a mirror should just be a black box to us - we don't
really care what they're running because we don't depend on any mirror
individually. Likewise if we negatively impact mirror hosts we'll end
up with less mirrors. Sure, maybe those hosts have odd
configurations, but we're still better off with them than without.
That said we do seem to have a lot of mirrors so it probably isn't the
end of the world if we lose a limited number.
And there is nothing to say that we can't have some infra mirror set
up more for interactive browsing that we don't have people fetch from
but which dispenses with all the hashing or which bins by the first
letter of the filename/etc. It seems like most of the use cases where
hashing is inconvenient are for more casual use.
To avoid another reply, people are talking about having utilities that
can fetch distfiles using the new scheme. I'd think that "ebuild
foo.ebuild fetch" is probably the simplest solution for this. Chances
are that you're dealing with SRC_URI strings that have variable
substitution in them anyway, so just letting ebuild do the fetching
means you're not substituting ${PV} and so on, let alone all the stuff
versionator and its ilk do. And of course you can always just fetch
from upstream anyway if you do have a clean URI.
--
Rich
Re: [gentoo-dev] Re: stable-bot is down. Temporary? Forever? Can we have a contacts page for it?
On Tue, Oct 8, 2019 at 7:57 AM Michael Palimaka wrote: > > On 10/8/19 7:21 AM, Andreas K. Huettel wrote: > > In any case, since many people *do* rely on it, maybe we should declare it > > official? [+] > > > > And, if that's OK with both of you, move it onto infra hardware? > > > > Happy to sponsor both for the next council meeting agenda. > > > > > > [+] At some point the one remaining whiner doesnt count anymore. > > > > In the past, infra has been understandably hesitant to take on new > services due to staffing issues. > > Additionally, I understand that the current infra design does not easily > allow granular access control, preventing non-infra members from easily > performing maintenance on individual services. > > Has this situation changed? I doubt infra want to take responsibility > for the bot, and I don't fancy the hassle of trying to find people to > poke things on my behalf. > IMO we should have a few tiers: 1. Absolutely core stuff that infra has to run (authentication, LDAP, maybe some services, etc). 2. Community-run stuff that is FOSS, with public config tracking (minus passwords/etc), and reasonably good docs. 3. Community-run stuff that is the wild west. IMO having a service catalog that includes all of this stuff is beneficial, with clear indications as to which tier each thing is in and who to contact with issues. Depending on #1-2 shouldn't really be a problem. #3 can be a playground for experimentation but shouldn't be something we really depend on for core workflow. To mitigate the risk of #2 we could have exercises to clone services following docs/etc. If anything #2 has the potential to be more reliable than #1 if it gets enough attention (though there is no reason our internal services couldn't also be made easy-to-replicate). I think the issue here is that we don't really have any standards for #2, but it is clear that this particular bot is intended to meet those requirements but doesn't quite do so today. I think this is a compromise that could help us focus our infra resources where they're needed most, with some separation of concerns. Ideally we should also make it possible via single-sign-on technologies to leverage infra's authentication services for stuff in tier 2, and maybe tier 3. Biggest risk is phishing if somebody spoofs a sign-on page. -- Rich
Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags
On Mon, Jul 22, 2019 at 8:35 AM Kent Fredric wrote: > > Though I suspect *literally* using USE flags for this as-is might be > the wrong approach, as that just causes user-side pollution :/ > Maybe in some other situations this might be true, but as I mentioned in my previous email, users who DO want to build their own manpages wouldn't want to use the pregenerated ones. Also, packages that have them need to know on the user side so that they can fetch them. So, there is a relationship between packages that need to have manpages pregenerated and the package manager. -- Rich
Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags
On Sat, Jul 20, 2019 at 4:22 PM Michał Górny wrote: > > > Yes, I get it. User experience is not important if it would mean > developers would actually do anything but the bare minimum to get > from one paycheck to another. The usual Gentoo attitude. > Not sure where I go to sign up for those paychecks. However, even employers have to accept that policies have a resource cost to them. Requiring people to do more than the bare minimum often just ensures that they won't even bother to do the bare minimum. I'm all for finding ways to standardize things so that everybody benefits at a very low cost. This doesn't seem that, and honestly requiring packages to bundle pre-built manpages seems a bit non-Gentooish to begin with. -- Rich
Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags
On Sat, Jul 20, 2019 at 2:28 PM Michał Górny wrote: > > Could you please read the proposed policy? It explicitly says you are > *not* supposed to force extra deps on users but build manpages for them. > This seems like a significant increase in maintainer effort compared to just leaving things as they are for very little benefit. Simple revbumps turn into needing to do a separate build just to build the manpages, then package those up, host them somewhere, then fetch and install that from the ebuild. It would be easier to just make the whole package a binary package since then all the logic happens outside the ebuild, and all the ebuild does is fetch/install a tarball, which it would have to do anyway just for the manpages. Most packages with stable build systems take almost no effort to revbump, and this would add a fair bit of complexity. I suspect that you'll find far more maintainers stop going to the trouble to strip out the dependencies needed for building manpages vs maintaining two build systems in parallel, with one having no place to host it. Then whenever a maintainer disappears the package goes to maintainer-needed, and anybody who wants to touch it has to figure out how to build the manpages likely without the benefit of any scripts the original maintainer had lying around. If we REALLY wanted to do something like this it seems like it would be better to build some tooling around it. Maybe an eclass combined with a special USE flag like "man-build". A daemon would check for packages that have this IUSE and would build the package using it, which will generate the manpages. It would then store those pages using a standardized naming convention. The ebuild would inherit the eclass which would check if man-build was set, and if not it would automatically fetch and install the manpages built by the build server from the mirrors. Then ebuilds that currently have IUSE=man would just inherit the eclass and change to the man-build flag. That flag would only be used by the build servers and not by end users, unless they wanted to build their own manpages from scratch, which would work fine, as the flag would not suppress building the rest of the package. Really though I don't see THAT much benefit from doing either. -- Rich
Re: [gentoo-dev] rfc: making sysvinit optional
On Thu, Jul 11, 2019 at 1:22 PM William Hubbs wrote: > > On Thu, Jul 11, 2019 at 12:46:02PM -0400, Rich Freeman wrote: > > On Thu, Jul 11, 2019 at 11:56 AM William Hubbs wrote: > > > > > > On Thu, Jul 11, 2019 at 09:42:02AM -0400, Rich Freeman wrote: > > > > If somebody just installs openrc their expectation is going to be that > > they get sysvinit or your substitute that actually works with openrc > > out of the box. They're not going to want to have neither installed > > simply because they have runit or systemd already installed. If > > somebody is migrating from systemd to openrc that is exactly the > > situation they would be in. > > And this would be handled by virtual/init and virtual/service-manager... > > If you are migrating, you would definitely want to be careful with > --depclean until you knew exactly what you wanted to remove or make sure > everything is in your world file that you don't want removed. > What value is virtual/init adding though? You don't have to be careful with migrating if you don't use it. You get no benefit from using the virtual instead of just depending directly on sysvinit, because that is the only package in the virtual that provides a reasonable init implementation for openrc to use. Yes, we can add that extra layer and then half the time it doesn't do anything and the other half the time it automatically does what the user doesn't want it to do, and users can work around it. What it won't ever do is make it easier to do what the user actually wants to do. If you do create a virtual/init then I'd just limit it to stuff that provides a generic sysvinit implementation that will easily work with any other service manager. I'd argue that is sysvinit, and maybe busybox[make-symlinks]. Maybe your openrc init implementation does, but I haven't looked at it. The rest simply don't provide an init that is designed to be used with an arbitrary service manager. Maybe to look at it another way: is this actually fixing a problem that anybody is concerned about? It just seems like it is giving portage freedom to shoot the user in the foot, for little real gain. -- Rich
Re: [gentoo-dev] rfc: making sysvinit optional
On Thu, Jul 11, 2019 at 11:56 AM William Hubbs wrote: > > On Thu, Jul 11, 2019 at 09:42:02AM -0400, Rich Freeman wrote: > > On Wed, Jul 10, 2019 at 11:02 PM William Hubbs wrote: > > > > > > > RDEPEND="sysv-utils? ( !sys-apps/sysvinit ) > > > > !sysv-utils? ( sys-apps/sysvinit )" > > > > > > I like this, but the second branch (!sysv-utils) is not really needed, > > > because if we put sysvinit as the first RDEPEND of virtual/init, we > > > don't need to worry about installing it through rdepend in openrc. > > > > Does openrc actually work with all the stuff you have in your proposed > > virtual/init? > > Remember that OpenRC wasn't originally an init process at all. it was > designed to work with any init process you want it to work with. That > hasn't changed, I've just added an init to it which you can use if you > want. > > > For example, you have systemd in there. I'm pretty sure you can't use > > systemd as PID1 and then use openrc as your service manager. I mean, > > you probably could come up with some way to do that, but certainly > > openrc doesn't work that way today, or systemd for that matter. > > There is nothing stopping you from that on the openrc side. It would > take a lot of custom systemd units to make it work, but that is an > exercise for the reader. > > > You have runit in there as well. Can you use runit as PID1 and openrc > > as your service manager? > > Sure. There's no reason you can't. I'm not talking about hypotheticals. Sure, somebody could completely dispose of the default set of systemd units and whatever runit uses as its equivalents, and create new ones that invoke openrc and have it take over, maybe, but none of this stuff actually exists right now. And I'm not sure how easy it would be to even get this working since systemd will still be trying to manage cgroups and whatever else that openrc will presumably also be trying to do. If somebody just installs openrc their expectation is going to be that they get sysvinit or your substitute that actually works with openrc out of the box. They're not going to want to have neither installed simply because they have runit or systemd already installed. If somebody is migrating from systemd to openrc that is exactly the situation they would be in. Neither systemd nor runit presume to be a drop-in replacement for sysvinit to be used with other service managers. Maybe you could mangle it into something like this, but at that point you might as well add python or bash to your virtual/init since you could just write your own script for init. My goal isn't to block user choice here - just to not just make it trivial for users to get their system into non-working conditions to support a configuration that nobody in their right mind is likely to actually care to use. I can't see the folks torn between Devuan and Gentoo saying that they're interested in using Gentoo with openrc but they've seen the light and it makes sense to use systemd as their PID1 with all its dbus dependencies just so that it can do nothing more than run a few bash scripts to launch openrc. I'd just stick with a direct conditional dependency on sysvinit. That is the only config that actually works with openrc reliably today. Now, if somebody comes up with a nice openrc wrapper for runit or systemd or whatever, then maybe add that wrapper to a virtual and revisit the issue, and then the wrapper can pull in the init implementation. Then users still get a working config no matter how portage resolves the virtual. -- Rich
Re: [gentoo-dev] rfc: making sysvinit optional
On Wed, Jul 10, 2019 at 11:02 PM William Hubbs wrote: > > > RDEPEND="sysv-utils? ( !sys-apps/sysvinit ) > > !sysv-utils? ( sys-apps/sysvinit )" > > I like this, but the second branch (!sysv-utils) is not really needed, > because if we put sysvinit as the first RDEPEND of virtual/init, we > don't need to worry about installing it through rdepend in openrc. Does openrc actually work with all the stuff you have in your proposed virtual/init? For example, you have systemd in there. I'm pretty sure you can't use systemd as PID1 and then use openrc as your service manager. I mean, you probably could come up with some way to do that, but certainly openrc doesn't work that way today, or systemd for that matter. You have runit in there as well. Can you use runit as PID1 and openrc as your service manager? If the only init implementations that openrc actually works with are sysvinit and its own init, then I'd just do it the systemd way. The init virtual only adds value insofar as these other packages actually provide an init that any other service manager could actually use. If openrc works with busybox init/etc I could see an argument for maybe having a virtual that can pull in either, though in that case it might make sense to use that in systemd as well. > We > can also add sys-apps/openrc as an rdepend of sys-apps/sysvinit > possibly. I'll take a look at that. I think it makes more sense to have a service manager pull in a compatible PID1 rather than the reverse. For example, systemd can pull in sysvinit for access to shutdown/telinit/etc but it makes no sense in that case to force openrc to get installed. You could even use sysvinit without any other service manager. -- Rich
Re: [gentoo-dev] rfc: making sysvinit optional
On Wed, Jul 10, 2019 at 8:03 PM William Hubbs wrote: > > On Wed, Jul 10, 2019 at 07:30:57PM -0400, Michael Orlitzky wrote: > > On 7/10/19 7:16 PM, William Hubbs wrote: > > > 3. add a sysvinit use flag to openrc, which will be off by default. When > > > it is on, openrc will block sysvinit since it will provide /sbin/init > > > and /sbin/shutdown. > > > > > > > This logic, or maybe the name of the flag, sounds backwards to me. I > > only get sysvinit when USE=sysvinit is NOT set? > > If you don't set sys-apps/openrc[sysvinit], you would have /sbin/init > and /sbin/shutdown as they are now, from sys-apps/sysvinit. > Systemd already has IUSE=+sysv-utils which has a similar function: [- ] sysv-utils sys-apps/systemd: Install sysvinit compatibility symlinks and manpages for init, telinit, halt, poweroff, reboot, runlevel, and shutdown RDEPEND="sysv-utils? ( !sys-apps/sysvinit ) !sysv-utils? ( sys-apps/sysvinit )" sysv-utils seems like a generic enough flag and I'd suggest that it would be appropriate to use for openrc as well if it can install its own implementation of these tools. (For those who aren't aware, systemd is compatible with the sysvinit versions of these tools, so you can run systemd with sysvinit installed.) -- Rich
Re: [gentoo-dev] Gentoo on Discord
On Tue, Apr 30, 2019 at 1:22 PM Kristian Fiskerstrand wrote: > > It matters if things are perceived as official Gentoo and causing a > negative reputation as in the article in this thread. One some level > that actually goes to trademark infringement that should be of interest > to the foundation, but the issue is broader than that. > While I don't speak for the Foundation, they already have a fairly decent policy addressing this: https://www.gentoo.org/inside-gentoo/foundation/name-logo-guidelines.html I believe this really only applies to use outside of Gentoo, and not internal use. Whether a service like a Discord site falls under internal use probably depends on the degree to which they are completely subject to Council/Trustees/etc, and the social contract and code of conduct as enacted by those bodies. For non-internal use the name/logo guidelines already have requirements around reputation and code of conduct. You can't just call yourself "Gentoo" and do whatever you want (not that I'm implying that this is what any particular site is doing - I haven't even seen the discord). -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Thu, Apr 25, 2019 at 6:29 PM Kristian Fiskerstrand wrote: > > On 4/26/19 12:26 AM, Rich Freeman wrote: > > I mean, I'd expect any Gentoo dev to be able to figure out how to use > > git as well, but git also has a terrible command line interface, > > Not really, it is quite intuitive once you understand the basics. > So, I love git, and certainly understand the basics, and created a python script to map/reduce our repo into a list of all unique files in the history with their hashes and commit info to compare to cvs during the migration. I certainly think the command line is terrible. How many different ways are branches identified in the various commands? git-am is a favorite whipping boy of many critics. It is just inconsistent from command to command, and some common things like backing out one commit require arcane syntax. Yes, it all makes sense if you understand the data model, but it is hardly a clean interface. I'm certainly not the first to say this, and it is not because of a lack of git knowledge. I suspect Linus himself would concur. It was a personal tool that scratched an itch that we're all stuck with because it works well enough that nobody will want to create a new interface. gpg is the same. Yes, the concepts are great once you understand them (though the smartcard standard is needlessly limited). The actual command line interface is just painful to use if you're doing more than just encrypting/signing something. If you want to use something other than your default key you pass --default-key, which seems odd, since you don't really want to change your default, and there isn't any way to pass a "non-default" key. I get having a default key option in a config file, since that is what it describes. And then there is all the interactivity, which makes sense to have as an option, but not without a command line override. I mean, the FTP interactive console also makes sense but there is a reason everybody uses curl/wget/etc, and not FTP+expect. > > > > Personally I think we ought to make it easier to just use the > > Nitrokeys we spent all this money on in a more secure manner than just > > leaving primary keys lying around on hard drives, > > The decisions involved are disjunct here, but leaving around on > hard-drives is generally a bad idea irrespective of any nitrokey, and > certainly something expected not to happen from a Gentoo Dev to begin > with (for primary key material at least) IMO this is quite naive. The desire to store it offline isn't even documented in the GLEP, nor is it enforceable. I get that some people like to care for their gpg keys the way other people like to wax their cars, but not everybody signed up as a Gentoo dev so that they have an excuse to participate in a WoT. And I think that the practical side of security is VERY relevant here. My argument is that having a single primary+signing key generated on a smartcard is more secure than having a separate primary+signing key stored on an internet-connected PC hard drive. And yet the first is forbidden by the GLEP and the second is allowed. The fact that you can't conceive of somebody using gpg in basically the most out-of-the-box way available doesn't mean that this isn't how most devs would actually do it. The integrity of this entire exercise is as secure as the dev who takes the least care to secure their keys, not the one who takes the greatest care. IMO it is in the interests of security to create a process that is both convenient and secure, and I think that keys generated on a smartcard achieve a better balance of this than other alternatives allowed under the current policy. -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Thu, Apr 25, 2019 at 5:54 PM James Le Cuirot wrote: > > if I understood it correctly, it only removes the primary private key > from the online copy and not the entire primary key. The --list-keys > option shows an [SC] primary with an [E] subkey and an [S] subkey and I > gathered from a conversation in #gentoo-dev that this is correct. Are > you suggesting the [SC] primary should not appear here at all? No, the public key should remain in your keychain. It is, after all, public, so there is no risk of compromise. You really want it to be published as widely as possible actually to reduce the risk of somebody using the wrong key. > > > Secondly, the reason for that is not (just) to have a backup > > but that the primary private key gives you virtually unlimited control. > > Are you contradicting yourself here? You explained why the private key > must be kept secure but you didn't say anything about the rest of the > primary key. The only keys you need to secure are the private keys. These keys are created in public/private pairs always. In the case of our GLEP we have three pairs: a primary, a signing, and an encryption. The signing and encryption pairs are referred to as subkeys, but this is just a convention - mathematically they work exactly the same. Ideally you want all your keys to be secure, but the concept of having tiered keys is that you can keep the primary in a safer place, since it can be used to invalidate and issue new subkeys, and thus you don't have to completely replace the trust chain if one key is compromised. -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Thu, Apr 25, 2019 at 4:55 PM Kristian Fiskerstrand wrote: > > Quite frankly I'd expect a Gentoo Developer to be able to manage the gpg > interface. > Being able to is not the same as caring enough to be bothered with it... I don't want to custom-tailor my Gentoo key. I just want to generate a key that will make the commit scripts happy. The key is completely disposable from a personal standpoint - when the GLEP was recently revised to make my old key no longer valid, I just generated a new one. I didn't even bother revoking the old one, since it had no function as soon as I changed the fingerprint in LDAP. I was generating PGP keys back when it used idea and I'm guessing md5. I've had gpg keys for decades. I used my personal one for Gentoo until the point where there were specific requirements for a Gentoo key, and rather than try to personally live with the Gentoo requirements it makes far more sense to just generate a Gentoo-specific key. Then we can change the GLEP as often as we like it it really doesn't bother me much. I can just discard my key and create a new one, though it would be nice if those creating the GLEPs would actually document the simplest way to do this for those who really can't be bothered to read the man page. I mean, I'd expect any Gentoo dev to be able to figure out how to use git as well, but git also has a terrible command line interface, so rather than put a bunch of requirements in a document and force everybody to dig through manpages to get it to generate signed commits/pushes/etc we just give a handy workflow. After all, our goal is to maintain the repo, not spend all day independently decipering how to sign pushes or figuring out that a commit sig and a push sig are two different things. Personally I think we ought to make it easier to just use the Nitrokeys we spent all this money on in a more secure manner than just leaving primary keys lying around on hard drives, which is where I suspect the vast majority will reside, completely negating the expense the Foundation and Nitrokey both went through to provide them for us. While I'm all for GLEPs themselves sticking to specs, having a workflow document to go along with it would go a long way to helping devs to comply, rather than spending all our effort writing increasingly clever scripts to yell at them when they aren't complying. -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Thu, Apr 25, 2019 at 4:34 PM James Le Cuirot wrote: > > On Thu, 25 Apr 2019 11:30:27 -0400 > Alec Warner wrote: > > > > Seeing as separating the primary and the signing key has been part of > > > OpenPGP best practices for a long, long time, I have got highly mixed > > > feelings about this statement. On the one hand, it is not reasonable to > > > expect someone with no or minimal prior knowledge of OpenPGP to master > > > it overnight. On the other, we are not just some random people from Teh > > > Intarwebz and we *have* been using OpenPGP signatures on commits for > > > quite a while now. > > > > > > > This is untrue though; we *are* random people from teh interwebs. > > > > I store my primary key on my desktop. > > I don't have copies of my primary key. > > My primary key is protected by a passphrase. > > Most of the time its cached in gpg-agent, so the passphrase is easily > > stealable by local attackers. > > I've been a dev for like > 10 years. > > > > I assume that every other dev does the same. Obviously some do not (and > > I've spoken to many who have better practices) but I assume > > people do the lazy / easy thing and I highly recommend this assumption. If > > you assume that people have your security practices, you should prepare to > > be disappointed. > > > > Many devs have *no idea* how GPG works. > > GPG is quite possibly the worst program I've even been forced to use in > > terms of doing any operation, particularly around setup (hmm maybe Imation > > Ironkeys were worse?) > > Many devs are just following the wiki instructions and get what they get. > > I can sort of echo this. I believe I'm close to the recommendations now > but it took me several evenings to actually wrap my head around all > this and even then, I still felt very nervous setting it up and I had > to rehearse it beforehand. As a professional software engineer for many > years, it really shouldn't be this hard. People talk about GPG best > practices but it was really difficult to find a reliable update-to-date > guide and it certainly doesn't feel like best practise when you have to > manually delete ~/.gnupg/private-keys-v1.d/KEYGRIP.key, where KEYGRIP > is returned by the obscure --with-keygrip option. I think a big problem is that gpg is sorely lacking in command line commands/options for key management. Almost anything having to do with key management involves a back-and-forth console interaction. This means that you can't just tell somebody to run "gpg --long --list --of --options" and have it just do the right thing. You also can't script anything unless you feed input or even worse use something like expect. Some of the guides I've seen require editing config files because presumably these options can't be set on the command line. I completely get what asymmetric crypto is. It is just a royal PITA to actually get gpg to do something very specific like have a separate signing key without pouring through manpages. Generating a key with the default options is easy, but after that you're on your own largely. Oh sure, once you know how to do it then it isn't a big deal. Until you have to do it again because you don't generate new gpg keys every other week... -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Thu, Apr 25, 2019 at 7:57 AM Marek Szuba wrote: > > On 2019-04-24 20:34, Rich Freeman wrote: > > > The only reason to have a separate primary key is to have an offline > > copy, > > Not quite. First and foremost, you don not want to have an offline copy > of the primary private key - you want to have the primary ENTIRELY > offline. Agree, I could have said that better. Though, to be clear the primary key needs to be on a PC anytime you use it, which includes time of generation and renewal. Typically this PC will be online. > > So, maintaining this requirement with a Nitrokey means that we in > > reality have the primary key online most of the time, > > Seeing as separating the primary and the signing key has been part of > OpenPGP best practices for a long, long time, I have got highly mixed > feelings about this statement. On the one hand, it is not reasonable to > expect someone with no or minimal prior knowledge of OpenPGP to master > it overnight. On the other, we are not just some random people from Teh > Intarwebz and we *have* been using OpenPGP signatures on commits for > quite a while now. IMO this has nothing to do with knowledge, and everything with risk tolerance and incentives. Generating a key on a smartcard is practically a one-liner and is convenient. It is also VERY secure. Now, if you're going to have a completely offline PC that never gets connected to the internet, and use that for key generation, and treat any media used to transfer keys as if you're working on a classified software project, then sure, that would be more secure. It is also a LOT less convenient. I'd argue that most devs who understand how to use GPG fairly well would not bother with this. I've never kept a primary key offline and I was using PGP back when you had to be located in the US to download it from the original official source. > > > when if it were the same as the signing key then both would be > > offline in the Nitrokey. > > Using a hardware security device is not the same as making the key > offline - especially given that the Gentoo NitroKey workflow as > currently posted on the Wiki suggests disabling forcesig, which could > effectively leave the signing private key unlocked for extended periods > of time. I'm all for revising this, but it isn't part of the GLEP. Maybe it should be. A smartcard is a practical compromise. It gives a great deal more security than online keys, while being convenient. Sure, it might not be the most secure approach possible, but it is far more secure than approaches most are likely to actually use, even if they know better options exist. > > > Also, by generating the key outside the Nitrokey it is exposed to a > > far higher risk of compromise. > > As Kristian has already mentioned, in principle one could keep the > primary private key on a separate token. Sure, though this is definitely more cumbersome, and not a one-liner in gpg for sure. And last time I checked we're only issuing one Nitrokey per dev, so it is unlikely many would do this. > > > In any case, I think it is far more likely that somebody generating > > keys using software has a rooted box than somebody is going to come > > up with a way to extract keys from a Nitrokey. > > You do not have to extract keys from a smartcard in order to be able to > use keys physically present on it. All you have to do observe the > smartcard user's PIN - either physically or using said rooted box - then > nick the smartcard off them, ehich given that we are talking about keys > that are supposed to be used on a regular basis might be very simple. Sure, but this requires physical theft, which is a HUGE escalation of effort. IMO the most likely attack is some script kiddie on the other side of the planet. I mean, somebody could steal my ID and get into my work and go cause a mess. However this is extremely unlikely in practice. If we were defending against the CIA or whatever I'd consider this a more serious concern, but this isn't realistic, and we would need far better standards to do that. > Hell, if said smartcard contains the primary key you might even return > it to them once you're done compromising them (e.g. by generating your > own set of subkeys) and chances are pretty good that as long as > everything keeps on working fine for them, it will take a quite a while > before anyone notices. I don't see how it differs whether the primary vs signing key is stolen, unless you regenerate new signing keys frequently. If you keep just re-extending expiry on your signing key then that stolen key will work forever. And if you do generate signing keys often then the window of compromise is the same either way. -- Rich
[gentoo-dev] [PATCH] glep-0063: Allow a single primary/signing key for smartcards
The OpenPGP smartcard standard, and the Nitrokey Pro smartcards that are being distributed to Gentoo developers, do not support having a separate primary/signing key for keys that are generated on the cards. As a result they can only be used in accordance with our current requirements if the keys are generated in software, which places them at a higher risk of compromise. The intent of the separate primary key is to reduce the risk of it being compromised by keeping it offline. However, if it were generated on a smartcard it would be exclusively be maintained offline, so it is counterproductive to require that it be generated online and then recommend that it be kept offline after this. Additionally this key needs to be brought back online anytime the key expiration is updated, which is at least annually. I believe this is creating a false sense of security, and that developers should be encouraged to generate new keys using their smartcards, so that these keys are never stored outside the protected hardware. By default gpg creates revocation certificates at this time as well. If a key is lost it can still be revoked, and of course the gpg fingerprint associated with the developer can be changed in any case and is the de-facto root of our trust system. These failure modes really are no different from an offline primary key that is separate from the signing keys. The revision adds an exception for keys generated on a smartcard. It does not prohibit those who wish to have separate keys from doing so, though doing this with card-generated keys requires having two smartcards. An obvious objection I can see is that nobody will be able to tell whether any particular key was generated on a smartcard or not. While this is true, it is also the case that there is no way to verify that a primary key is being stored offline, or used on a non-compromised PC, or that it even has a passphrase set. Unless we want to use some kind of hardware key that supports remote attestation we're going to have to trust developers to be security conscious, and IMO making it easy to generate keys on smartcards actually will facilitate security. This change actually makes the more secure mode of operation simpler than the less secure one (unless the primary key is kept online, in which case it provides no benefit). Full version online at: https://gist.github.com/rich0/7d11e2297d8b8a5586997baec2a99e30 Patch follows: diff --git a/glep-0063-v3.rst b/glep-0063-v3.rst index 5895873..86e5fd9 100644 --- a/glep-0063-v3.rst +++ b/glep-0063-v3.rst @@ -12,6 +12,12 @@ OpenPGP key management policies for the Gentoo Linux distribution. Changes === +v3 + The requirement to have a separate signing and primary key was removed + in the case of keys generated/stored on smartcards, to encourage the use + of these keys, and acknowledging that the main use case for a separate + primary key is largely fulfilled by having all the keys stay offline. + v2 The distinct minimal and recommended expirations have been replaced by a single requirement. The rules have been simplified to use @@ -69,7 +75,8 @@ not be used to commit. at least 256-bit. All subkey self-signatures must use this digest. 2. Signing subkey that is different from the primary key, and does not - have any other capabilities enabled. + have any other capabilities enabled. This requirement does not apply + if the primary key was generated on a smartcard. 3. Primary key and the signing subkey are both of type EITHER: -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Wed, Apr 24, 2019 at 11:57 AM Kristian Fiskerstrand wrote: > > On 4/24/19 4:19 PM, Rich Freeman wrote: > > If it is the case that Nitrokeys can't support a separate primary key, > > I'd suggest modifying the GLEP to remove that requirement when a > > smartcard is in use. Its main purpose is to keep a key component > > offline, and if the key is generated on the card that is already > > accomplished. Maybe somebody has a suggestion for how to make the two > > work together, otherwise I'll go ahead and suggest a GLEP revision for > > the next Council meeting. > > > The GLEP should not be changed on the requirement for distinct signing > subkey, this is one of the expected results of it to begin with. So, I intend to propose this. Council is welcome to vote it down. However, I'm really interested in what the concern is here. The only reason to have a separate primary key is to have an offline copy, but it is not a current requirement that it be kept offline, and I suspect that 99% of devs won't keep it offline, and of course there is no way to verify if this is being done anyway. So, maintaining this requirement with a Nitrokey means that we in reality have the primary key online most of the time, when if it were the same as the signing key then both would be offline in the Nitrokey. This requirement effectively makes the key less secure in practice. It doesn't matter if the signing key is safely stored in the Nitrokey if somebody can steal the primary key, since they can just create a new signing key at will. Also, by generating the key outside the Nitrokey it is exposed to a far higher risk of compromise. At best for the 1% of devs who would actually keep the primary key offline then this achieves the same level of security you have with the Nitrokey anyway, which always keeps its keys offline (effectively). The only exception to this would be an exploit in the Nitrokey, which seems like a pretty low risk. And of course it is only a risk when the Nitrokey is plugged in, and the intent would be for devs to only plug it in when working on commits. In any case, I think it is far more likely that somebody generating keys using software has a rooted box than somebody is going to come up with a way to extract keys from a Nitrokey. Now, I think it is a great best practice which we should support for anybody who wants to have their offline key-generation machine they keep in a vault or whatever. And by all means require the additional key when using keys not generated on a Nitrokey. As to how you can tell which are which, I'd simply point out that you can't tell if keys are being stored offline or not, so really your risk is unchanged in taking devs at their word. I think most companies issuing these sorts of tokens to users would generate their keys on the tokens for these very reasons. The reason they're using these hardware tokens is because they know that users in practice will not take extraordinary care to protect keys, and the token provides a similar level of security with very little inconvenience. Really the only thing we're missing with the Nitrokey is some form of attestation to ensure the keys are in fact generated on the device. -- Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Wed, Apr 24, 2019 at 10:25 AM Alon Bar-Lev wrote:
>
> On Wed, Apr 24, 2019 at 5:19 PM Rich Freeman wrote:
> > Well, in that case recommendations for how to generate a key that
> > complies in software would be helpful. There used to be a wiki
> > article on it, but it is marked with various warnings saying it isn't
> > recommended to follow it, and has seemingly vanished with a note that
> > it moved somewhere.
> In the meantime, I think that you may find the following commands useful...
>
> $ gpg --expert --full-generate-key
Well, sure, but the part that comes after would be the key bit.
> $ gpg --expert --edit-key ${MASTER_KEY_ID}
> gpg> addkey
This bit is already covered on the Nitrokey guide.
In any case, I'll stick with the key I have for the moment until the
Council rules on the GLEP. Neither key I have currently complies with
the current one so I'm generating a new one either way, unless the
GLEP changes.
--
Rich
Re: [gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
On Wed, Apr 24, 2019 at 10:01 AM Marek Szuba wrote: > > On 2019-04-24 13:41, Rich Freeman wrote: > > > What is the recommended way to create an on-card key? > > I haven't got my NitroKey yet but between the specifications (which say > NK2 can hold up to 3 private RSA keys) and my prior experience with > OpenPGP smartcards (which have always had at most one slot each assigned > to authentication, encryption and signing), chances are pretty high you > cannot have two separate signing keys in hardware. If so, your best bet > is probably to generate the primary key in software (preferably with > usage bits stripped down so that it can ONLY be used for signing keys), > generate hardware subkeys associated with it, then stash the private > primary key away somewhere. > Well, in that case recommendations for how to generate a key that complies in software would be helpful. There used to be a wiki article on it, but it is marked with various warnings saying it isn't recommended to follow it, and has seemingly vanished with a note that it moved somewhere. Aside from that, it seems a bit odd to issue devs Nitrokeys (at significant expense to both the Foundation and a kind sponsor), then require a key design that can't actually be stored on the Nitrokey. A Nitrokey-generated key is going to be way more secure than the way 99% of devs will actually implement what seems to be intended, which is to just generate their keys on a regular online host, and probably just leave it there. If it is the case that Nitrokeys can't support a separate primary key, I'd suggest modifying the GLEP to remove that requirement when a smartcard is in use. Its main purpose is to keep a key component offline, and if the key is generated on the card that is already accomplished. Maybe somebody has a suggestion for how to make the two work together, otherwise I'll go ahead and suggest a GLEP revision for the next Council meeting. -- Rich
[gentoo-dev] Last rites: games-rpg/eternal-lands
Removing this one as the maintainer, but would be happy to work with a proxy maintainer if somebody wants to take over. I suspect it isn't actually in-use... # Richard Freeman (24 Apr 2019) # Masked for removal in 30 days. Likely does not work # and is essentially unmainted. Please comment in # bug 548926 if you want to give this some care. games-rpg/eternal-lands games-rpg/eternal-lands-bloodsucker games-rpg/eternal-lands-data -- Rich
[gentoo-dev] Last rites: games-rpg/eternal-lands
Removing this one as the maintainer, but would be happy to work with a proxy maintainer if somebody wants to take over. I suspect it isn't actually in-use... # Richard Freeman (24 Apr 2019) # Masked for removal in 30 days. Likely does not work # and is essentially unmainted. Please comment in # bug 548926 if you want to give this some care. games-rpg/eternal-lands games-rpg/eternal-lands-bloodsucker games-rpg/eternal-lands-data -- Rich
[gentoo-dev] Best way to create a GLEP 63 compliant GPG key on Nitrocard?
I just generated a gpg key on my nitrocard using the default options, as I could not find any other official recommendations for how to create a key. However, it appears that the default config uses the same signing and primary key, and thus generates a warning when pushing to the main repo. What is the recommended way to create an on-card key? Or is the only supported way to do this to create a local key and then export it to the card? If so, what is the current recommended way to do that? -- Rich
Re: [gentoo-dev] [PATCH] glep-0063: Require encryption subkey, and make primary certify-only
On Sun, Feb 24, 2019 at 3:35 AM Michał Górny wrote: > > Following the recent mailing list discussion indicating that developers > are taking GLEP 63 as only source of truth about OpenPGP keys, and can > make assumption that if encryption key is not listed there they should > not have one. Amend the specification to extend it beyond the previous > limited scope of commit signing, and require an encryption key > appropriately. This matches the GnuPG defaults. Does GLEP 63 actually match the gpg defaults? That is, if you generate a gpg key and accept every default value will the key be acceptable? If not, could we get some updated documentation as to how to generate a minimally compliant key, similar to: https://www.gentoo.org/glep/glep-0063.html#bare-minimum-requirements -- Rich
Re: [gentoo-dev] rfc: cron.* and modern cron implementations
On Sat, Mar 2, 2019 at 8:25 PM Michael Orlitzky wrote: > > Using run-parts in /etc/crontab also has its problems, but I don't have > a solution handy for that one. Using run-parts runs those daily, weekly, > etc. jobs as root, which may not be what you want if you're operating on > user-controlled data (e.g. bug 662438). The best way to solve this would > be to let packages install their own crontab entries into a directory > like /etc/cron.d, but that location isn't standard. > So, the problem with cron.d is that you're now using crontab syntax, and for compatibility you have to use the lowest common denominator which is vixie. That means your jobs are STILL running as root, so the only problem you had with run-parts isn't solved. In addition you lose the ability to cover the desktop use case of non-24x7 systems running infrequent tasks. If you used vixie cron syntax for a monthly job it might never run at all on a typical desktop, as it would have one opportunity to run in a month, at one time of day. The crontab syntax also forces each package maintainer to pick the time of day their jobs run at, vs just letting the sysadmin choose the time the entire set of scripts is run. I'm sure there are alternatives like adding a compatibility layer (which is basically what run-crons already is), or some kind of helper where an ebuild can give it a set of parameters and it installs the task for whatever cron implementation eselect points it at. I'm just not sure that they are worth the complexity or provide much more value than the existing solutions. This is also somewhat orthogonal to run-crons, where you still are left with the choice around whether to use it with vixie or other implementations that don't support more desktop-oriented use cases. This is of course why that bug has been fairly intractable. -- Rich
Re: [gentoo-dev] rfc: cron.* and modern cron implementations
On Sat, Mar 2, 2019 at 7:26 PM Michael Orlitzky wrote:
>
> On 3/2/19 7:05 PM, William Hubbs wrote:
> >
> > Is there a reason we still use run-parts and the
> > /etc/cron.{hourly,daily,weekly,monthly} structure to run repeating cron
> > jobs?
> >
> > From what I read in the chat earlier, it sounds like the modern crons
> > might be able to handle this without that structure, but I'm not sure.
>
> https://bugs.gentoo.org/69777
>
> Totally. We should replace run-parts with something much simpler and
> more predictable. Then, if that doesn't work for you, all modern crons
> can do the things that run-parts tries to do, but better.
>
I'm not sure I see the connection here. All run-parts does is run all
the scripts in a directory. That seems pretty simple and
deterministic.
The bug is about cronbase, which contains run-crons, along with
installing the cron.d directories. I could see an argument for
splitting that package though obviously the package is already pretty
simple.
I imagine most cron implementations do not use run-crons. Whether any
particular one (like vixie-cron) should seems like a matter of taste.
Are we just talking about not having vixie-cron use run-crons? And
instead having it just have time-scheduled jobs for run-parts on the
various cron.* directories? That seems a bit narrower in scope than
what was originally suggested, though it isn't clear to me what is
being suggested...
--
Rich
Re: [gentoo-dev] rfc: cron.* and modern cron implementations
On Sat, Mar 2, 2019 at 7:05 PM William Hubbs wrote:
>
> someone brought this up on the chat channel today, so I'm bringing it
> here to ask for information.
>
> Is there a reason we still use run-parts and the
> /etc/cron.{hourly,daily,weekly,monthly} structure to run repeating cron jobs?
>
> From what I read in the chat earlier, it sounds like the modern crons
> might be able to handle this without that structure, but I'm not sure.
>
Are you proposing getting rid of run-parts? Or are you proposing
getting rid of /etc/cron.*?
run-parts is part of debianutils, and ca-certificates apparently uses
it, so trying to purge that might not go far. I don't think it is
directly in @system so it would go away on its own if it wasn't used.
Some of the cron implementations also use it, and some don't, and each
one can pull it in as needed I suppose.
I don't think you can get rid of the cron.* directories, since that is
the least-common-denominator way for packages to install scripts for
cron to run. If we wanted to do something else we'd probably need
some kind of eclass that knows how to install a cron script for any of
the various cron implementations out there. We can't really even just
go to generic cron syntax for some kind of crontab.d handler because I
don't think that is standardized for tasks that are to run if their
scheduled time is missed.
I suspect that maintainers of cron implementations that don't require
run-parts probably already avoid using it.
Maybe you had something specific in mind that I missed?
--
Rich
Re: [gentoo-dev] adding app-crypt/gentoo-keys to @system
On Fri, Feb 22, 2019 at 9:58 PM Matthew Thode wrote: > > Ok, after setting that up portage wants to update pgp keys, which fail > because keyservers suck. It doesn't look like we can change the > keyservers or disable the update entirely but we can set the retries to > 0 (which better disable it...). Robbat2 had a patch to allow disabling > the update but it doesn't look like it was applied. I assume that it proceeds after some timeout? Or does it completely bail? IMO failing successful makes more sense though it is less secure. It definitely makes sense to attempt a keyserver update since that is going to be the mechanism to catch key revocations. It also will make life easier on users using an older stage3 that happens to have expired keys. Well, assuming the keyserver works... -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 5:49 PM Michał Górny wrote: > > On Tue, 2018-11-27 at 16:01 -0500, Rich Freeman wrote: > > On Tue, Nov 27, 2018 at 3:42 PM Kent Fredric wrote: > > > > > > That git manages not to die every day based on what we throw at it is > > > frankly a miracle of engineering. > > > > Our repo is a linked list being constantly manipulated from the head > > backed by a hashed object store for the contents. For that use case > > it is probably the ideal data structure. Since our use case is > > actually the typical use case, it isn't a surprise that this was the > > design that was chosen... :) > > > > Computers are pretty fast when you actually use the correct algorithm... > > > > Yes, computers are fast and their work is cheap. On the other hand, > humans are not fast and their time is expensive. Now use the power of > human thinking to infer this to what you're doing to this thread. > Not wasting everybody's time with personal attacks? -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 4:50 PM Matt Turner wrote: > > Or, I don't know. Come up with something better and continue > bikeshedding. I won't. > I think antarus already came up with something better - let Sony explain its thinking, rather than trying to guess at what they're trying to accomplish and whether it is something we want to support or not. Or just stick with what we've already been doing for the last 15 years, which is completely compatible with the new GLEP. -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 3:42 PM Kent Fredric wrote: > > That git manages not to die every day based on what we throw at it is > frankly a miracle of engineering. Our repo is a linked list being constantly manipulated from the head backed by a hashed object store for the contents. For that use case it is probably the ideal data structure. Since our use case is actually the typical use case, it isn't a surprise that this was the design that was chosen... :) Computers are pretty fast when you actually use the correct algorithm... -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 3:34 PM Michał Górny wrote: > > Please don't forget we're talking about Gentoo, so brace for 2 weeks of > bikeshedding over whether first or last name should come first. > Thank goodness there isn't a Mike Gordon on the rolls or we could discuss the intricacies of UTF-8 ordinality. -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 3:23 PM William Hubbs wrote: > > On Tue, Nov 27, 2018 at 09:15:08PM +0100, Michał Górny wrote: > > On Tue, 2018-11-27 at 14:07 -0600, William Hubbs wrote: > > > All, > > > I just picked a random msg to reply to on the thread. > > > > > > Here is the updated AUTHORS file I would like to commit. > > > > > > > You should include at least all current developers with commit access. > > Because otherwise, this really looks like all Gentoo work was done by > > the Foundation (which didn't do any work at all) and Sony, entirely > > neglecting the huge effort done by many individuals. > > No, that is definitely not the case. all devs aren't required to be > listed; only those who want to be [1]. > Uh, I'll see your [1] (a random non-Gentoo website) and raise you the ACTUAL Gentoo policy on the matter [2]. If that policy is inappropriate you might want to revise it. > There is no way to contact everyone and see who wants to be listed, if > someone wants to be listed they can let us know. > > [1] https://opensource.google.com/docs/releasing/authors/ [2] https://www.gentoo.org/glep/glep-0076.html#simplified-attribution -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 3:15 PM Michał Górny wrote: > > On Tue, 2018-11-27 at 14:07 -0600, William Hubbs wrote: > > All, > > I just picked a random msg to reply to on the thread. > > > > Here is the updated AUTHORS file I would like to commit. > > > > You should include at least all current developers with commit access. > Because otherwise, this really looks like all Gentoo work was done by > the Foundation (which didn't do any work at all) and Sony, entirely > neglecting the huge effort done by many individuals. > > I get you can't be expected to figure out all the people who ever done > any major contribution to Gentoo. However, that's no excuse to skip > the process entirely and just put your employer there. > I did not realize that williamh is on Sony's payroll. For all the talk of conflicts of interest I've seen regarding comrel decisions (where there is no financial conflict of interest as just about every company on the planet recognizes), I sincerely hope that williamh intends to recuse himself from this matter as he has an obvious financial conflict of interest here. Ditto for any other employees of companies that wish to be acknowledged in our copyright notices. -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 2:58 PM Matt Turner wrote: > > What Copyright-owner header are you talking about? We would create one, just as we've created bugzilla tags in git for closing bugs/etc. Surely putting one line in a commit is no more difficult than putting one file in a repository? Indeed anybody can start sticking such a tag in commits today without any involvement from anybody else. > You've been the > most outspoken opponent of using what appears to be the standard > attribution form specified in GLEP-76 When have I been opposed to anything in GLEP 76? I'll admit that I don't 100% agree with everything in there, but I'm fine with following the GLEP as it is written. Multi-line copyright notices aren't in there, and the intent was never to be accumulating copyright holders on the single notice line. An authors file was a compromise I wasn't a huge fan of, but I'm suggesting that if we have one it ought to be auto-generated (presumably with the work being done by somebody who actually wants the file to be there). Also, GLEP 76 as it is written says: "Projects using this scheme must track authorship in a VCS, unless they list all authors of copyrightable contributions in an AUTHORS file." So, a VCS is the PREFERRED way of doing it. The alternative is listing ALL authors in the authors file. Right now it seems like people are advocating for only listing some authors. > I know mailing list debates are your personal pastime but this is a > real wasteoftime. You're the one advocating for changing the status quo. I'm happy if we drop the whole topic entirely. You certainly can't point fingers at others when you're proposing doing something differently. We wouldn't be having this discussion if some contributors were unwilling to contribute under our current standards. -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 12:31 PM William Hubbs wrote: > > Agreed, we should not add every developer to this file by default. > Isn't this basically giving the most credit to the most difficult-to-work-with entities by default? As others have pointed out, it seems like other projects are moving away from AUTHORS files in favor of git. If we want to go in the opposite direction, why wouldn't we give credit to those who aren't creating drama? -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 11:59 AM Matt Turner wrote: > > On Tue, Nov 27, 2018 at 7:41 AM Rich Freeman wrote: > > It makes sense to ensure that the solution actually solves the problem > > before we simply implement it. > > > > If we really need such a file it would probably also make more sense > > to have it auto-generated from git commit headers > > And how do you want to determine whether William's contributions are > copyright Sony or now? Do you want to look up his timezone and check > whether they were made during work hours? No, you look at the Copyright-owner header or whatever we want to call it, and use that. Companies that care about labeling what they own can take the time to properly document this. -- Rich
Re: [gentoo-dev] AUTHORS file for portage repository
On Tue, Nov 27, 2018 at 10:16 AM Michał Górny wrote: > > Will that actually solve any problem, or just add another half-baked > solution with no benefit to the resulting status? In other words, would > SIE suddenly stop requiring you to alter copyright in ebuilds? ++ It makes sense to ensure that the solution actually solves the problem before we simply implement it. If we really need such a file it would probably also make more sense to have it auto-generated from git commit headers, which could use a standardized format. This would actually provide more useful data around authorship/copyright than a generic file with a list of names anyway. Certainly standards like SPDX should be leveraged. These tags could be optional in git, but anybody who wants to use these tags could do so and tools that parse them could be created by those interested in this information. -- Rich -- Rich
Re: [gentoo-dev] [pre-GLEP r1] Gentoo binary package container format
On Mon, Nov 19, 2018 at 2:40 PM Zac Medico wrote: > > On 11/19/18 11:33 AM, Rich Freeman wrote: > > On Mon, Nov 19, 2018 at 2:21 PM Roy Bamford wrote: > >> > >> "The archive members support optional OpenPGP signatures. > >> The implementations must allow the user to specify whether OpenPGP > >> signatures are to be expected in remotely fetched packages." > >> > >> Or can the user specify that only some elements need to be signed? > >> > >> Is it a problem if not all elements are signed with the same key? > >> That could happen if one person makes a binpackage and someone > >> else updates the metadata. > >> > > > > IMO this is going a bit into PM details for a GLEP that is about > > container formats. > > > > Presumably any package manager is going to need to figure out what > > keys are/aren't valid and allow the user to configure this behavior. > > Users who want to go editing package innards will presumably adjust > > their package manager settings to accept their modifications, whether > > it means accepting their own sigs or disabling them. > > With the GLEP as it is, the user *must* use a local signing key to sign > installed packages during the installation process if they want to be > able to verify signatures for installed packages at some point in the > future, since the binary package format does not provide a way to use > binary package signatures for this purpose. I think we might be talking about different signatures? I think you're referring to signatures of the package files after they are installed on the local filesystem, while I'm talking about verifying the integrity of the package file themselves. If these signatures are applied to different data then obviously you couldn't just have the one signature serve double duty (unless you hung onto the binary package, verified the signature on it, then verified the package contents against the live filesystem). The simplest solution would be to do as you seem to be suggesting - verify the signature on the package before installing it, and then during installation capture whatever metadata is already supported by portage and sign that using a user's trusted key. This seems like the most practical solution in any case since we aren't likely to ever go down the route of using a single signed squashfs for /usr like a release-based binary distro might. -- Rich
Re: [gentoo-dev] [pre-GLEP r1] Gentoo binary package container format
On Mon, Nov 19, 2018 at 2:21 PM Roy Bamford wrote: > > "The archive members support optional OpenPGP signatures. > The implementations must allow the user to specify whether OpenPGP > signatures are to be expected in remotely fetched packages." > > Or can the user specify that only some elements need to be signed? > > Is it a problem if not all elements are signed with the same key? > That could happen if one person makes a binpackage and someone > else updates the metadata. > IMO this is going a bit into PM details for a GLEP that is about container formats. Presumably any package manager is going to need to figure out what keys are/aren't valid and allow the user to configure this behavior. Users who want to go editing package innards will presumably adjust their package manager settings to accept their modifications, whether it means accepting their own sigs or disabling them. -- Rich
Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format [gen...@jonesmz.com]
On Sun, Nov 18, 2018 at 5:40 PM Zac Medico wrote: > > On 11/18/18 1:55 PM, Rich Freeman wrote: > > > > My idea is to basically have portage generate a tag with all the info > > needed to identify the "right" package, take a hash of it, and then > > stick that in the filename. Then when portage is looking for a binary > > package to use at install time it generates the same tag using the > > same algorithm and looks for a matching hash. > > We've already had this handled for a couple years now, via > FEATURES=binpkg-multi-instance. According to the make.conf manpage this simply numbers builds. So, if you build something twice with the same config you end up with two duplicate files (wasteful). Presumably if you had a large collection of these packages portage would have to read the metadata within each one to figure out which one is appropriate to install. That would be expensive if IO is slow, such as when fetching packages online on-demand. But, it obviously is somewhat of an improvement for Roy's use case. IMO using a content-hash of certain metadata would eliminate duplication, and based on filename alone it would be clear whether the sought-after binary package exists or not. As with the build numbers you couldn't tell from filename inspection what packages you have, but if you know what you want you could immediately find it. IMO trying to cram all that metadata into a filename to make them more transparent isn't a good idea, and using hashes lets the user set their own policy regarding flexibility. Heck, you could auto-gen symlinks for subsets of metadata (ie, the same file could be linked from a file that specifies its USE flags but not its CFLAGS, so it would be found if either an exact hit on CFLAGS was sought or if CFLAGS were considered unimportant). But, I'm certainly not suggesting that you're not allowed to go to bed until you've built it. :) -- Rich
Re: Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format [gen...@jonesmz.com]
On Sun, Nov 18, 2018 at 4:10 PM Roy Bamford wrote: > > Replying off list because I am not on the whitelist. That seems odd. > 1) append a uuid to each filename. Generated when the bin package file is > generated. > 2) encode the hostname of the machine that generated the file > 3) encode the use flags in the filename. So, I brought up this same issue in the earlier discussion and it was considered out of scope, and I think this is fair. The GLEP does not specify filename, and IMO the standard for what goes INSIDE the file will work just fine with any future enhancements that address exactly this use case. Besides your case of building for a cluster, another use case is having a central binary repo that portage could check and utilize when a user's preferences happen to match what is pre-built. I suggest we start a different thread for any additional discussion of this use case. I was thinking and it probably wouldn't be super-hard to actually start building something like this. But, I don't want to derail this GLEP as I don't see any reason designing something like this needs to hold up the binary package format. Both the existing and proposed binary package formats will encode any metadata needed by the package manager inside the file, and the only extension we need is to encode identifying info in the filename. My idea is to basically have portage generate a tag with all the info needed to identify the "right" package, take a hash of it, and then stick that in the filename. Then when portage is looking for a binary package to use at install time it generates the same tag using the same algorithm and looks for a matching hash. If a hit is found then it reads the complete metadata in the file and applies all the sanity checks it already does. Generating of binary packages with the hash cold be made optional, and portage could also be configured to first look for the matching hash, then fall back to the existing naming convention, so that it would be compatible with existing generic names. So, users would get a choice as to whether they want to build up a library of these packages, or just have each build overwrite the last. Then the next step would be to allow these files to be fetched from a binary repo optionally, and then finally we'd need tools to create the repo. But, this step isn't needed for your use case. With the proper optional switches you could utilize as much of this scheme as you like. Also, you could optionally choose how much you want portage to encode in the tag and look for. Are you very fussy and only want a binary package with matching CFLAGS/USE/whatever? Or is just matching USE/arch/etc enough? Some of the existing portage options could potentially be re-used here. Please make any replies in a new thread. -- Rich
Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format
On Sat, Nov 17, 2018 at 9:05 AM Roy Bamford wrote: > > Does this proposal allow for installing the payload without > the use of the Gentoo package manager from some random > distro being used as a rescue media? > Yes, it is a tarball of tarballs. There would be an extra step, but a vanilla tarball containing the files to be extracted could be extracted as long as you have tar and the appropriate decompressor (not specified and could change, but I imagine it will remain bzip2 for now). -- Rich
Re: [gentoo-dev] Unmasking >=media-video/ffmpeg-4.0
On Tue, Nov 6, 2018 at 10:57 AM Alexis Ballier wrote: > > On Tue, 06 Nov 2018 17:08:22 +0200 > Mart Raudsepp wrote: > > > It is not GStreamer fault that ffmpeg breaks API and ABI without > > parallel installability, much less so the distro maintainers of it. > > If you/upstream don't make it parallel installable, then this is what > > you get. > > Are you, seriously, suggesting this is the solution to all problems > here ? > It isn't the only solution, but it is one sane upgrade path. You can't expect everybody to update their software overnight when the API changes. That means you have to support the old API for a while when you introduce a new one, otherwise you end up with some software that doesn't work with the old version, and some software that doesn't work with the new version. Inevitably you get some program that is either simple or has a ton of manpower that refactors to the new API very quickly and abandons versions using the old API, and then you have some other program that is very complex or low-manpower that takes forever to adopt it. Now, parallel installation isn't the only way to accomplish this inter-compatibility, but it is one way. It seems like everybody is determined to go down the appimage/container path and just make dll-hell the new standard for linux. When every program gets bundled with its on zlib you don't have to worry about compatibility. Now you just get to update all 47 copies of it when a vulnerability turns up, and hope that the fix is backported to all 13 versions you're implicitly using. -- Rich
Re: [gentoo-dev] net-dns/dnssec-root: Blind stable on arm, critical bug 667774
On Sat, Oct 20, 2018 at 8:19 AM Andreas Sturmlechner wrote: > > On Freitag, 12. Oktober 2018 14:50:55 CEST Rich Freeman wrote: > > ARM is not a Gentoo security supported arch. > > > > If the ARM maintainers feel that stable keywords make the lives of > > their users better, and it isn't causing problems for anybody else, > > I'm not sure why we should be interfering with this. > > That's interesting. If it's not security supported, does that mean we can > simply clean up vulnerable versions and drop every arm revdep to ~arm? > > Or are we supposed to keep vulnerable versions around and drop every keyword > except arm? > Setting aside the security supported flag that was already discussed, there is also a council decision regarding this general topic [1]. The only issue is that I'm not certain if it was intended to apply to ARM, or only to specific arches [2]. The last policy was: "If a maintainer has an open STABLEREQ, or a KEYWORDREQ blocking a pending STABLEREQ, for 90 days with archs CCed and otherwise ready to be stabilized, the maintainer can remove older versions of the package at their discretion. A package is considered ready to be stabilized if it has been in the tree for 30 days, and has no known major flaws on arches that upstream considers supported." [1] IMO that was written generically enough that it could apply anywhere, but that is up to the Council. In theory it could even be safely applied to x86/amd64, especially since maintainers can self-stabilize/keyword on those arches typically. [1] - https://projects.gentoo.org/council/meeting-logs/20131119-summary.txt [2] - https://projects.gentoo.org/council/meeting-logs/20130917-summary.txt -- Rich
Re: [gentoo-dev] Is there any way I can help with pull requests?
On Tue, Oct 16, 2018 at 3:14 PM Ralph Seichter wrote: > > I followed the PHP team's recommendation, as can be seen from the PR > conversation and from the underlying Bugzilla report. While I respect > Michał Górny's opinion, I understand that he does not have a deciding > vote in this case. He said as much himself in his comment - he wasn't acting on behalf of QA or anything like that. My guess is that both of these requests are probably just waiting for a maintainer/etc to do something, which is probably a typical situation. I'm not sure which projects typically let proxy-maintainers directly commit vs wanting to do it themselves. I don't think mgorny's feedback is going to hold anything up except to the degree that others on the appropriate team happen to agree, but if that were the case I'd expect them to just change the commit accordingly. -- Rich
Re: [gentoo-dev] Is there any way I can help with pull requests?
On Sat, Oct 13, 2018 at 8:59 AM James Le Cuirot wrote: > > On Sat, 13 Oct 2018 13:28:02 +0200 > Ralph Seichter wrote: > > > Is there any way I could help the Gentoo team? Any vacancies that need > > to be filled, or work that needs to be done? > > Following what I said above, we need more actual bona fide developers > who have done the quizzes, understand the issues, can most important > can take responsibility for their own contributions. > > There is the proxy maintainers project, where we can accept > contributions with a little less scrutiny, but I personally feel that > these contributors really should just become developers. It doesn't > matter if you're only interested in one or two packages, any help is > very welcome. > I have to imagine that while waiting to become a dev (or not), it would still be useful if contributors would like to skim pull requests and find ways to contribute to them. If a dev has pointed out an issue with a pull request, consider offering a patch to fix the issue. If a pull request hasn't been looked at, consider looking at it and offering patches to resolve any issues you see. Sure, ultimately a dev still has to review/commit, but I imagine any help with triage will just make their life easier, which means they'll be able to get more commits into the actual tree. And of course all the skills you would use to clean up pull requests are for the most part the same ones you'll need to become a dev. Also, reading a bunch of dev comments about quality issues with pull requests will make you aware of the kinds of QA issues people routinely run into, and you'll be less likely to create the same problems. In my experience the best way to get accepted as a committer in just about any kind of FOSS project is to basically do just this. Sooner or later you'll be noticed and short-listed. You'll also rub shoulders with people who are likely to become mentors, and since recruiter availability is limited they're going to be bound to focus on applicants who are active already. -- Rich
Re: [gentoo-dev] net-dns/dnssec-root: Blind stable on arm, critical bug 667774
On Thu, Oct 11, 2018 at 1:14 PM Thomas Deutschmann wrote: > > But that's not the point here. The point was to get some attention that > again we have a lacking architecture (net-dns/dnssec-root is not the > only package where ARM arch team is lacking behind) which affects anyone > "trusting" somehow in STABLE keywords. ARM is not a Gentoo security supported arch. If the ARM maintainers feel that stable keywords make the lives of their users better, and it isn't causing problems for anybody else, I'm not sure why we should be interfering with this. > > If everyone is using ~ARCH and don't care about STABLE keywords, well, > we could save a bunch of time, energy... > Is this costing YOU any time/energy? If not, why do you care? This thread seems to be devolving into another debate about the purpose of stable, and I'm mainly seeing arguments that have come up countless times already. Most of these arguments tend to point out things that are perceived as being wrong with stable as it currently exists. Most of these arguments are probably posted by people who don't even run stable, let alone maintain it. If somebody wants to actually "fix" stable IMO the best way to go about that is to create a proposal for something new that people will get behind. Gentoo tends to move forward by creating new things, not by arguing about what is broken with old things. What solution is even being proposed? Tell devs they're not allowed to work on stable? That doesn't mean that their next thoughts will be "wow, since I'm not allowed to spend 10 hours per week working on something I cared about, I guess I'll spend those 10 hours per week working on something that somebody else cares about." If it makes Gentoo less useful to them personally, they're just as likely to just drop other contributions they make on the side as they look for better solutions. IMO when stable teams create issues for maintainers by not being responsive to bugs then this needs to be dealt with. However, the Council has already allowed maintainers to drop stable keywords when this happens, and I imagine they'd be responsive to dealing with issues that are more sustained. -- Rich
Re: [gentoo-dev] Signed-off-by verification incoming
On Sat, Sep 29, 2018 at 9:25 AM Dirkjan Ochtman wrote: > > Some kind of repoman support would make this much easier to handle. As > it is, doing this by hand for the trivial case (only my own changes) > is a PITA. repoman could grow some kind of --sign-off option that > appends this to the commit message presumably? > Does "repoman commit" no longer do the right thing? -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky wrote: > > On 09/14/2018 03:58 PM, Richard Yao wrote: > >> > >> No one has answered the question: what do you do when a stable package > >> breaks because of a new warning? > >> > >> ...> > > Wouldn’t this be largely covered as part of GCC stabilization? We could > > reserve the right to kill -Werror in a package where it blocks GCC > > stabilization if the maintainer does not handle it in a timely manner. > >> > > They would be uncovered during GCC stabilization, but then you're right > back in the original situation: how do you fix the stable package? The > only answer that doesn't violate some other policy is to patch it in a > new revision and wait for it to stabilize again. > > Other questions arise: Do we block stabilization of clang et al.? > Presumably we could make it a blocker, so then portage won't install the new stable toolchain. That buys time and only affects users of that particular package. But, as I pointed out before you can do that without using -Werror - just block installation with an unqualified toolchain. You would only use an approach like this for packages where QA was fairly important, so the inconvenience would be worth it. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 1:33 PM Michał Górny wrote: > > On Fri, 2018-09-14 at 20:22 +0300, Alon Bar-Lev wrote: > > Let's do this the other way around and be react based on facts and not > > speculations. > > Let's change the policy for a year for selected packages as I > > outlined, monitor bugs and after a year see response times, affected > > users and if downstream patches are accumulated. Then we can decide if > > we need to patch upstream packages. > > If we need to patch upstream package anyway, not follow upstream > > policy and not accepting input for various of permutations and > > architecture from all users, this discussion is nearly void. > > > ...and for how long did you exactly ignore the standing policy that > suddenly we need a new testing period? How about we do the opposite > and you prove a *single* bug found downstream using this method so far? Wouldn't the flip side of this be demonstrating that this has actually caused issues? If following upstream discovers no bugs and also causes no issues, why not leave it to maintainer discretion? I'm not talking about hypothetical issues. I'm talking about specific issues with this specific example, that supposedly has already done all the testing necessary... -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Fri, Sep 14, 2018 at 1:22 PM Alon Bar-Lev wrote: > > Let's do this the other way around and be react based on facts and not > speculations. > Let's change the policy for a year for selected packages as I > outlined, monitor bugs and after a year see response times, affected > users and if downstream patches are accumulated. Then we can decide if > we need to patch upstream packages. Obviously up to QA/Council, but I think this sounds like a practical approach. No need to "change the policy" so much as approve a temporary exception, perhaps for a defined period of time. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Thu, Sep 13, 2018 at 9:29 AM Mike wrote: > > And I apologize for writing that commit rights were requested to be > removed. My mistake, bugzilla access rights were asked to be removed. >... > > I'm not a fan of more and more and more regulation that I see. Sorry if > you don't like that opinion. > What regulation? No action was taken. We can't exactly stop people from asking governance bodies to do things. At most we can say no when they ask. Unless we want to make people ask if they can ask. Now, that would be regulation... :) -- Rich
Re: [gentoo-dev] acceptable alternatives to -Werror, was: Changing policy about -Werror
On Wed, Sep 12, 2018 at 8:23 PM Chí-Thanh Christopher Nguyễn wrote: > > Rich Freeman schrieb: > >> Requirements: > >> > >> * Do not fail to build/install when a warning is encountered > > > > On a particularly critical package like a filesystem, wouldn't we want > > to still fail to install when a warning is encountered? > > Installation will proceed, but the user will get a big fat warning that the > sys-fs/zfs package is potentially broken. > > > I get that users might quit if packages don't install, but I'm not > > sure that a filesystem corruption is going to make them any happier... > > If the user recognizes this as a critical package, then they can do the > research before deciding on whether to use the package as is, attempt to > downgrade, or wait until a fix is released. But, you've ALREADY overwritten the previous version of the package that presumably didn't have this problem. What if downgrading doesn't cause the problem to go away? What if it is due to some dependency or toolchain change? Users would presumably want to roll back to the binaries that were working just a few minutes ago, not build new ones that might or might not have the same issue. -- Rich
Re: [gentoo-dev] acceptable alternatives to -Werror, was: Changing policy about -Werror
On Wed, Sep 12, 2018 at 7:35 PM Chí-Thanh Christopher Nguyễn wrote: > > > Requirements: > > * Do not fail to build/install when a warning is encountered On a particularly critical package like a filesystem, wouldn't we want to still fail to install when a warning is encountered? > Also possible is to introduce FEATURES="strict-warnings" or similar, which > will cause the build to fail if warnings are encountered in a warningfree > ebuild. I guess this depends on whether warningfree is only used where -Werror would otherwise be used. Packages could presumably also warn users when installing without this feature set not to complain too much if our defaults destroy their data and to consider setting the flag... :) I get that users might quit if packages don't install, but I'm not sure that a filesystem corruption is going to make them any happier... -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Wed, Sep 12, 2018 at 7:52 PM Matt Turner wrote: > > On Wed, Sep 12, 2018 at 4:03 PM Rich Freeman wrote: > > Now, I could buy that -Werror turns NEW warnings into fatal errors, > > due to the use of a newer toolchain, since upstream probably didn't > > test with that toolchain and thus wouldn't have seen the warning. > > Yes, exactly. This is one of the major things people have said repeatedly. > > Werror makes moving gcc forward much more difficult. > Sure, but wouldn't a block on newer versions of gcc cause similar headaches? -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Wed, Sep 12, 2018 at 7:32 PM Chí-Thanh Christopher Nguyễn wrote: > > Alon Bar-Lev schrieb: > > We > > are unique as permutations and architectures that are used by Gentoo > > users are so diverse that we find issues that nobody else finds. > > This needs to be highlighted more, as it is why suggestions that the > maintainer can simply put -Werror back on their own system are insufficient. > Wouldn't running into new runtime issues be exactly the sort of reason why we'd want to build with -Werror on packages where these issues are unacceptable? -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Wed, Sep 12, 2018 at 6:55 PM Thomas Deutschmann wrote: > > On 2018-09-12 16:50, Rich Freeman wrote: > > There is also the case where we want these warnings to block > > installation, because the risk of there being a problem is too great. > > I really disagree with that. So many devs have already said multiple > times in this thread that "-Werror" is only turning existing warnings > into fatal errors but "-Werror" itself doesn't add any new checks and > more often requires "-O3" to be useful. > This seems unlikely. If upstream is using -Werror in their build system, then they'd be getting build errors if these warnings already existed at the time they released the version. Now, I could buy that -Werror turns NEW warnings into fatal errors, due to the use of a newer toolchain, since upstream probably didn't test with that toolchain and thus wouldn't have seen the warning. If the warning only appears with -O3, and the package isn't built with -O3, then -Werror is a no-op and is harmless. But, I think there is somewhat of a legitimate point in pointing out that -Werror is in some sense just a proxy for detecting when a toolchain is being used which wasn't tested by upstream. You could accomplish the same by sticking a ton of toolchain blockers in the package. Of course, I can't imagine the toolchain team would be terribly happy about that, but if you want to avoid using newer toolchains with older versions of a package that would probably the the appropriate way to do it. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Wed, Sep 12, 2018 at 4:56 AM Jason Zaman wrote: > > Replying to a somewhat random post. There are two separate things here > that people are discussing here but are not the same thing. Three, really... > > 1) We want to know when a package has terrible warnings when installing > it so we can report upstream and know that something might have gone > wrong. There is also the case where we want these warnings to block installation, because the risk of there being a problem is too great. > > Stick this in your make.conf: > PORTAGE_ELOG_SYSTEM="echo save" > PORTAGE_ELOG_CLASSES="warn error log qa" > I'm pretty sure toralf's tinderbox already has these enabled but all > devs should too. The problem is that this will make the warnings non-fatal. Do we still tell users not to report these kinds of warnings in bugzilla? If they're the sort we consider serious then we would want to know about it so that we can address it, vs just waiting for upstream to fix them in a future release. There might be a better solution than -Werror, such as a flag in an ebuild that makes the existing QA warning fatal and tells the user to log a bug. -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Mon, Sep 10, 2018 at 5:01 PM Mike Gilbert wrote: > > On Mon, Sep 10, 2018 at 4:56 PM Kristian Fiskerstrand wrote: > > > > On 9/10/18 10:51 PM, Matt Turner wrote: > > > Consider again the bug that started this. The maintainer had not built > > > this configuration. None of the arch teams had built this > > > configuration until I did for the last architecture Cc'd. The patch > > > committed doesn't change anything installed on the system, if not for > > > Werror preventing the code from building. > > > > one way to look at it though, is that it is a valuable upstream > > contribution that this configuration produces the error, so Gentoo is > > contributing to upstream development because of it. > > As an end user of Gentoo, I may not care about "contributing to > upstream"; I just want the software to compile and install. > For more critical packages (like the example of zfs) whether it compiles and installs isn't 1/10th as important as whether it eats my data... -- Rich
Re: [gentoo-dev] Changing policy about -Werror
On Sun, Sep 9, 2018 at 1:50 PM Michael Orlitzky wrote: > > So if you're using -Werror to prevent a > "vulnerable" package from being installed, it doesn't work, and can > actually be harmful if it prevents me from using a better compiler. > Whether or not the new compiler is better, it is clearly untested with the package-version in question (otherwise these warnings would have been addressed). For something critical like a filesystem (zfs) that could be useful to know. I'm not convinced that this rule ought to be absolute. That said, I do agree with your later comments that this creates a messy situation by painting a user into a corner. Other than sticking painful ranged version dependencies on the toolchain into the package I'm not sure if there is a cleaner solution that guarantees that the package won't be built with a version of gcc that is untested with that specific package without a user override. I guess we could just have sensitive ebuilds output that it might eat your data if you didn't add -Werror to your CFLAGS and then leave it to the users to decide how much they care about build errors vs unlikely sorry-I-lost-your-10PiB-array errors. -- Rich
Re: [gentoo-dev] [RFC] Solving the problem of huge number of wrong LICENSES=*GPL-[23]
On Mon, Aug 27, 2018 at 6:46 PM Michael Mol wrote: > > I can say that if the licenses are habitually misidentified, I could not use > Gentoo's portage tree in my job without extensive and ongoing revalidation of > the license metadata. > Keep in mind that we're just talking about GPL-2 vs 2+ and GPL-3 vs 3+. We're not talking about GPL vs BSD vs something that involves a EULA. Do you actually accept one of these without the other, such that you would have problems if they had gotten confused? -- Rich
Re: [gentoo-dev] [RFC] Solving the problem of huge number of wrong LICENSES=*GPL-[23]
On Sun, Aug 26, 2018 at 7:15 AM Michał Górny wrote: > > On Sun, 2018-08-26 at 13:09 +0200, Paweł Hajdan, Jr. wrote: > > On 26/08/2018 12:53, Mart Raudsepp wrote: > > > The common issue here is that upstream COPYING files really do only > > > talk about one of the versions. And then you get to validate or source > > > files to be sure that they do have a "or later" clause in them. And > > > then on each bump you ideally should validate it again, etc, that no > > > sources without "or later" allowance are in there... > > > > Yup, precise tracking of license metadata can be a pain. > > > > I'm not really sure if that level of it is worth for us as a distro. For > > _importing_ other project's source code directly into one's project > > precise license compatibility matters a lot. That's not the scenario > > we're in. I see LICENSES as mostly a mechanism for end users to accept > > or reject EULAs etc, and I'm curious what are other common scenarios. > > > > Michał, could you elaborate on why not distinguishing more precisely > > between these GPL variants in LICENSES is a _problem_ ? I can certainly > > see the information is not always accurate, but it's not obvious to me > > how severe is the downside, what are the consequences in practice. > > > > I'm not aware of any major implications. However, I think that if we > provide for the distinction, the distinction should be used correctly. > IMO QA policy ought to be that the license is correct. How much time/effort goes into policing the policy in the case of 2/3/2+/3+ is a different matter. If people want to do it, great, but IMO it isn't adding tremendous value. I doubt we have any users relying on license filtering to distinguish between GPL2/2+. If somebody files a bug pointing out an incorrect license it should be fixed as a matter of policy, but I'm not sure more than that is necessary in this particular case. If we were talking about nonfree licenses being missed that would be more critical. -- Rich
Re: [gentoo-dev] Gentoo i486 support
On Fri, Aug 24, 2018 at 9:57 AM Mike Gilbert wrote: > > On Fri, Aug 24, 2018 at 9:19 AM Kent Fredric wrote: > > > > On Wed, 22 Aug 2018 07:26:24 -0500 > > Ben Kohler wrote: > > > > > Thoughts? > > > > Is there a good reason we can't have a legacy profile for this? > > > > Or perhaps, a new (exp) arch entirely dedicated to legacy x86? > > Sounds like a lot of work for something that will be used by very few people. > I think an exp arch is also overkill. How many packages simply can't be built for i486? I think a profile+masking makes a lot more sense than an entire new level of QA that touches every ebuild in the tree because there might be a few packages that don't work on 25 year old hardware. -- Rich
Re: [gentoo-dev] Gentoo i486 support
On Wed, Aug 22, 2018 at 4:27 PM R0b0t1 wrote: > > Even newer embedded i586 and i686 hardware isn't cost effective > considering power consumption. When considering power it often does > not even make sense to run donated hardware ~5 years old. > I was referring to running the x86 arch on hardware that is otherwise-capable of running the amd64 arch. I'm not really sure how important that would be, but I wanted to at least consider the possibility. In any case, this is becoming moot it seems. -- Rich
Re: [gentoo-dev] Gentoo i486 support
On Wed, Aug 22, 2018 at 8:26 AM Ben Kohler wrote: > > 1) Adjust x86 profile defaults to drop the problematic -march=i686. > This would be more in line with amd64 profiles (et al), which set no > -march value so it can run on any hardware for this arch. > My knee-jerk reaction was that this is a bad idea, but after a bit of thought there are some arguments in favor of this: First, the argument against: i386 is VERY old. Most distros moved their defaults to i686 because it had significant improvements, and i686 was still mainstream but i386 was ancient. In contrast with amd64 the entire architecture is fairly new and the baseline doesn't suffer from many of the issues i386 suffers compared to i686. This is a really short synopsis - if you go to any distro list archive you can find long passionate arguments from ~10 years ago that elaborated on this. In that sense, going back to i386 is turning back the clock. HOWEVER, I think there is an argument for i386 that wasn't so valid back then. x86 in general is starting to look a bit like i386. What are the main use cases for it in this day and age? I don't use x86, so I'm not the best person to answer that. However, I'd broadly split it into two categories (mostly by tautology): 1. Museum hardware. People have systems that are running simply BECAUSE they are old, not because they are cost-effective/etc. I'm not sure I'd even lump used hardware into this category any longer, as I'm sure there are plenty of i686+ used PCs at rock-bottom prices already out there, and maintaining pre-Y2K hardware is going to be fairly painful. For this use case i386 as the baseline makes a LOT of sense. 2. Non-museum hardware. People have x86 hardware because it is the most cost-effective solution for a task, and not merely because it is old. IMO for this use case i686 makes a lot more sense as a baseline. However, I'm honestly not sure in this day and age what these use cases even are, unless it is something you can buy for $10 at a flea market. Even if you're talking about a container running one application that only needs 500kB of RAM, is there really that much benefit to not building it for amd64? The other argument for i386 would be that in Gentoo nobody is stuck with the defaults. So, a default that works more widely as an entry point makes a lot of sense, since anybody can set CFLAGs and do an emerge -e world to get the benefits. Then again, if we're talking about older but not ancient hardware that is still quite a bit of build time. IMO the best thing here would be for people to actually RUN x86 to chime in. I've been amd64-only on Gentoo since not long after I started using Gentoo (and that was back when mplayer barely could be made to work on amd64). Once upon a time I could have bought the pointer size argument around RAM use, but if that were really a big concern I think more people would be running x32. Is there a large population that actually runs x86 on modern hardware, or is ancient hardware a significant use case? -- Rich
Re: [gentoo-dev] Idea for a new project: gentoo-libs
On Sun, Aug 5, 2018 at 2:12 PM Richard Yao wrote: > > > Prestige is good. We have prestige from our (myself and a few others) work in > upstream ZFS and Gentoo is well respected there. Sure, but ZFS on Linux isn't a Gentoo project. I'm not saying people who are Gentoo devs can't also do other things. Lots of Gentoo devs are involved in lots of stuff that gets a fair bit of respect. OpenRC is a big example of this. From time to time I bump into signs of Gentoo on upstream projects (just today that included SoapySDRPlay). Also, this gives us an opportunity to set an example, in helping to ensure the upstream project is maintained in a distro-friendly manner where Gentoo is a first class citizen. I'm not saying that stuff like this should be banned from infra. It just seems like an unnecessary constraint all-around. It means that the project has to generally follow Gentoo policies/etc, and is limited to the tools we provide. Just something to think about... -- Rich
Re: [gentoo-dev] Idea for a new project: gentoo-libs
On Sun, Aug 5, 2018 at 1:01 PM Alec Warner wrote: > > > Part of my frustration is that seemingly "anything open source related > can be held in Gentoo" and I'm somewhat against that as I feel it > dilutes the Gentoo mission. We are here to make a distribution, not > maintain random libraries. If you want to do that feel free; but I > don't see a need for that work to be associated with Gentoo. > Honestly, other than maybe some prestige I don't really get the point of hosting random software in Gentoo either. These days getting a repo on github or any of its 47 competitors is a few clicks. You have zero overhead from a governance standpoint, and a dev can of course stick ebuilds in the main repository with zero interference. It seems a lot cleaner from a copyright/etc standpoint as well. Even openrc is hosted outside of Gentoo these days, which makes perfect sense. With the distro as a whole it is a bit more complex, though honestly I'd love to see us get to a point where the whole thing can be SECURELY hosted entirely off-infra as well, even if we still chose to run our own infra. I just see it as a way to both provide options to our users and ourselves. For the latter, being able to host anything on an outside service means that if some component of infra goes down we could have mirrors already running and pulling from infra, or if for some reason somebody sues us or roots us or whatever we can pick up and move without much fuss. Running your own wiki/bugzilla/lists/etc was about the only way to do things in the 90s/etc, but these days there are other options... -- Rich
Re: [gentoo-dev] rfc: moving default location of portage tree (was: [gentoo-project] Call for agenda items - Council meeting 2018-07-29)
On Sun, Jul 29, 2018 at 3:56 PM Matt Turner wrote:
>
> On Fri, Jul 27, 2018 at 1:32 AM, Ulrich Mueller wrote:
> >
> > So, considering all the feedback from mailing list and IRC:
> >
> >/usr/portage -> /var/db/repos/gentoo
> >/usr/portage/distfiles -> /var/cache{,/gentoo}/distfiles
> >/usr/portage/packages -> /var/cache{,/gentoo}/binpkgs
> >
> > Open question: Should we have the additional "gentoo" path component
> > for the ones in /var/cache? The tradeoff is between a path that is
> > easier to type, or slightly easier usage if someone wants to NFS mount
> > distfiles and binpkgs.
>
> That proposal has by vote of support. No strong preference on whether
> to include gentoo/ or not. It's one NFS mount vs two so not a big deal
> either way.
>
Why not stick the repos in /var/repos and not /var/db/repos? If we're
just making up paths, why not make up a shorter one? I don't think
any other linux distros use /var/db...
--
Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Tue, Jul 24, 2018 at 5:11 PM Dennis Schridde wrote: > > On Tuesday, 24 July 2018 20:57:09 CEST Rich Freeman wrote: > > On Tue, Jul 24, 2018 at 2:32 PM Ian Stakenvicius wrote: > > > I don't think the process needs to be simplified much more than this; > > > each layer above has its purpose. However I do very much want to > > > caution on making it more complicated, especially with the addition of > > > syntax that allows setting or ignoring useflag state changes in a way > > > that will jumble up these layers. > > > > I think as long as it is a heirarchy it will be straightforward enough. > > > > If we introduce a ^ operator that unsets a flag, the only question is > > how far that propagates down the layers, and into what kinds of > > layers: > > > > Does a profile ^flag undo an IUSE +flag? > > Does a make.conf ^flag undo a profile +flag? An IUSE +flag? A > > profile flag mask? > > Does a package.use ^flag undo a make.conf +flag? A profile +flag, an > > IUSE +flag? Etc... > > I guess the question here is: Is there an official order in which the use > settings from the different profiles and config files have to be applied? > > I think my initial assumption of this order was wrong, USE flags only have two > states and indeed it seems that the ^ USE operator is not necessary, because > the - operator already serves the same purpose. > Kinda sorta. In the flag is either set or it isn't. However, I think the intent here was to strip out the effects of profiles, but keep the effects of IUSE. The question then becomes what other contexts can it be used in, and in each case what does it undo? -- Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Tue, Jul 24, 2018 at 2:32 PM Ian Stakenvicius wrote: > > I don't think the process needs to be simplified much more than this; > each layer above has its purpose. However I do very much want to > caution on making it more complicated, especially with the addition of > syntax that allows setting or ignoring useflag state changes in a way > that will jumble up these layers. > I think as long as it is a heirarchy it will be straightforward enough. If we introduce a ^ operator that unsets a flag, the only question is how far that propagates down the layers, and into what kinds of layers: Does a profile ^flag undo an IUSE +flag? Does a make.conf ^flag undo a profile +flag? An IUSE +flag? A profile flag mask? Does a package.use ^flag undo a make.conf +flag? A profile +flag, an IUSE +flag? Etc... This matters more than with +/-, since these just bluntly set the flag to one setting or the other regardless of what happened below (I think - I'd have to check all the different possibilities even for that). When you're just unsetting things the question is how many of these flags do you unset going down how far? Obviously if you unset everything up to that point then it is no different from -flag as we default to off on everything. -- Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Tue, Jul 24, 2018 at 12:49 PM Michael Orlitzky wrote: > > On 07/24/2018 12:37 PM, Rich Freeman wrote: > >> harder to customize, because you can't turn it off. > > > > This was already addressed in a previous comment - PMS can be modified > > to nullify flags > Saying that hypothetically we could modify the PMS and wait for a new > EAPI and wait for all package managers to catch up and wait a year after > that for the upgrade path and then update the profiles to a new EAPI and > then implement this new feature is just an indirect way of saying "you > can't do it." > > Changing the PMS won't let me set USE="-udev" in make.conf, anyway. It > would only affect sub-profiles. Certainly it should be implemented so that you could use it in make.conf. I also said nothing about the order of changes here. The PMS change could be made and be effective before the profile change is made. -- Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Tue, Jul 24, 2018 at 12:27 PM Michael Orlitzky wrote: > > On 07/24/2018 12:14 PM, Rich Freeman wrote: > > > > I don't believe anybody suggested making Gentoo harder to customize. > > This is just about setting reasonable defaults. > > For the (N+1)th time: Well, if it was already said, why did you add another reply? > enabling this flag by default does make Gentoo > harder to customize, because you can't turn it off. This was already addressed in a previous comment - PMS can be modified to nullify flags, and arguably it should already work this way (though that means subprofiles can't actually use -flags). > > You can run a server without bash, openrc, sysvinit, or glibc. Should > > these also be removed from the base profile? > > Three of those four aren't in the base profile. Yeah, it was a good idea > to remove them. It includes virtual/service-manager, which pulls in sysvinit and openrc by default. You can run just fine without any service manager. Just point the kernel at your daemon and it will launch it as PID 1. Also, you can use sysvinit without openrc and have it launch your daemon, but that also won't satisfy service-manager. But, none of this matters, because people who REALLY want to run without a service manager can do that with package.provided. Again, this is about sane defaults, and having a service manager is a sane default. -- Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Tue, Jul 24, 2018 at 12:06 PM Michael Orlitzky wrote: > > On 07/24/2018 11:39 AM, Mike Gilbert wrote: > > > > You can run any system without udev, but you need to be very careful > > about what Linux features you utilize and how you have the system > > configured. > > > > Most Linux servers out in the wild are running udev; your > > configuration is an exception to the common case. > > > > Gentoo users are Gentoo users because Gentoo is easy to customize. I don't believe anybody suggested making Gentoo harder to customize. This is just about setting reasonable defaults. You can run a server without bash, openrc, sysvinit, or glibc. Should these also be removed from the base profile? -- Rich
Re: [gentoo-dev] Adding USE=udev to linux profiles
On Sat, Jul 21, 2018 at 5:33 AM Zac Medico wrote: > > Sure, why not? So ^flag would mean that the flag state propagates from > the settings in IUSE. Presumably this could be overridden in subsequent profiles, or /etc/portage. That is, one profile might set a flag, and another profile could unset it, and then the final one could re-set it. > It's also conceivable that we could add a way for > profiles to modify the effective IUSE defaults, via new operators in > package.use or by introducing a new file such as package.use.default. That makes sense, or the syntax could be available in the ebuild. I imagine the better approach to take would depend on the nature of the incompatibility. -- Rich
