Re: [gentoo-dev] Breakage and frustration
Rich Freeman wrote: > a big question is how to make it happen without just throwing > complaints on the folks who are trying their best to keep it all going. The answer to this is the same as it has always been: Demonstrate that you are capable and reliable and given social compatibility then after some time you too can become a part of the team. This is the way most open source projects, actually most human projects work. Maintainers are not picked at random when there is high load, but trusted long-time contributors are promoted to maintainers when they shine. The more you complain instead of send patches, the less likely you are to ever become part of the solution. The key point to remember is that it is NOT neccessary to be part of the team in order to contribute solutions. You *first* contribute solutions and only *then* have a chance of becoming part of the team. I for one am working in my non-existant spare time on a fast ChangeLog generator. //Peter
Re: [gentoo-dev] Breakage and frustration
On Mon, Dec 14, 2015 at 5:48 AM, Peter Stugewrote: > > The key point to remember is that it is NOT neccessary to be part of > the team in order to contribute solutions. You *first* contribute > solutions and only *then* have a chance of becoming part of the team. > > I for one am working in my non-existant spare time on a fast > ChangeLog generator. > ++, and thanks. I was actually trying to think beyond this point though. Right now we have lots of infra stuff that the infra team would probably love to publish except there is no easy way to do it without creating security issues (intermixing of credentials and code, etc). We have a few devs who have access to all of it, but their time needs to be split between keeping the current state working, and trying to build some future state which is easier to contribute to. For the rest of our part, we do need to put up and start building things. So, how do we get from that to a future state where our infra servers have public backups or whatever minus /etc/credentials.include and /var/spool/private-email and so on? That is, a future state where the default is open with necessary exceptions? And how do we do it with what we have, or are able to get? I just want to be constructive. Clearly the first step is to acknowledge that the onus is on those who really want to see change to chip in to make it happen. We have to be here to serve - this is a volunteer FOSS project. -- Rich
Re: [gentoo-dev] Breakage and frustration
On 12/14/2015 02:58 PM, Rich Freeman wrote: > On Mon, Dec 14, 2015 at 5:48 AM, Peter Stugewrote: >> The key point to remember is that it is NOT neccessary to be part of >> the team in order to contribute solutions. You *first* contribute >> solutions and only *then* have a chance of becoming part of the team. >> >> I for one am working in my non-existant spare time on a fast >> ChangeLog generator. >> > ++, and thanks. > > I was actually trying to think beyond this point though. Right now we > have lots of infra stuff that the infra team would probably love to > publish except there is no easy way to do it without creating security > issues (intermixing of credentials and code, etc). We have a few devs > who have access to all of it, but their time needs to be split between > keeping the current state working, and trying to build some future > state which is easier to contribute to. For the rest of our part, we > do need to put up and start building things. > > So, how do we get from that to a future state where our infra servers > have public backups or whatever minus /etc/credentials.include and > /var/spool/private-email and so on? That is, a future state where the > default is open with necessary exceptions? And how do we do it with > what we have, or are able to get? > > I just want to be constructive. Clearly the first step is to > acknowledge that the onus is on those who really want to see change to > chip in to make it happen. We have to be here to serve - this is a > volunteer FOSS project. > Y'know ... If I had access to things I could help. But I don't, so I can't. The rather obvious solutions seem to be politically impossible, and the politically possible solutions are not satisfactory.
Re: [gentoo-dev] Breakage and frustration
On Mon, Dec 14, 2015 at 03:03:53PM +0100, Patrick Lauer wrote: > If I had access to things I could help. But I don't, so I can't. The scripts of the git->rsync process are already open. https://gitweb.gentoo.org/infra/mastermirror-scripts.git/ (also in there are the related scripts for the other distfiles/releases/experimental mirroring). The only parts not there (from memory): - credentials to connect to rsync endpoints And, as I noted, the Manifest issues are problems in Portage, which is also open code... -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Breakage and frustration
On Sun, 13 Dec 2015 18:36:41 +0100 Patrick Lauerwrote: > Broken breakage > > > tl;dr: Stuff is broken, and no one seems to care > ... > [1] https://bugs.gentoo.org/show_bug.cgi?id=557184 RESOLVED > [2] https://bugs.gentoo.org/show_bug.cgi?id=557192 RESOLVED > [3] https://bugs.gentoo.org/show_bug.cgi?id=557344 RESOLVED > [4] https://bugs.gentoo.org/show_bug.cgi?id=557400 RESOLVED > [5] https://bugs.gentoo.org/show_bug.cgi?id=557826 RESOLVED > [6] https://bugs.gentoo.org/show_bug.cgi?id=565574 RESOLVED > [7] https://bugs.gentoo.org/show_bug.cgi?id=565694 RESOLVED > [8] https://bugs.gentoo.org/show_bug.cgi?id=567074 RESOLVED > [9] https://bugs.gentoo.org/show_bug.cgi?id=567830 RESOLVED Tried to follow the email and failed to find unfixed problems. I believe focusing on actual issues would help. What is broken currently? Do relevant people know they are broken (= are there open bugs)? Or is it an email thread about how not to break things ever in future? -- Sergei signature.asc Description: PGP signature
Re: [gentoo-dev] Breakage and frustration
TL;DR summary: Yes, stuff has broken, but I'd call them reasonable teething issues well distributed through the stack, and to be compared to the CVS server moves from a decade ago, rather than CVS just before the Git switch. On Sun, Dec 13, 2015 at 06:36:41PM +0100, Patrick Lauer wrote: ... (mail re-ordered for related issues) > Once that was 'fixed' there was the fun of [2], which made emerge --sync > very expensive because it refetched lots of files. Every time. The fix for this caused these two bugs: > And fixing that introduces [7] some more regressions that broke updating > @system for about 3.5 days. > - a few days of grub being uninstallable (iow, making installing > impossible for many users) > And the manifest issues are still [9] making life exciting. Bug #567920 describes the issue very succinctly (mtime of a deleted file needs to be included in the new Manifest mtime calculation). Both of them can be worked around if the entire path (all staging nodes, servers and end users) uses --checksum, but that's even more expensive. I have another work-around idea, and that's simply appending a comment of the latest commit per directory to the changelog, because that will trigger the manifest being different ;-). > The fix to that fix (notice a pattern here?) broke rsync for *all* users > [8]. Almost as if no one ever tests things in a test environment ... but > hey, we're agile, let's fix stuff in production! We still never figured out how .git came to be added to the outgoing data. It was NOT the rsync into staging directory, because it was only the directory structure, and none of the files. --exclude=.git WAS used in most of the ryncs, but not the final ones from staging to tree distribution. > - about 3 months of emerge --changelog being broken, just to be broken > in a different way This change (order of changelog entries) was explicitly to reduce your complaint of prior excess traffic. Why Portage's parsing of the ChangeLogs is still not handling it is an open question. > - 3.5 days of emerge @system being broken > - about a day of emerge --sync needing manual interaction to be able to > update again You missed one: - rsync generation now halts if somebody committed some breakage to the tree (missing DIST entry, bad eclass). > > So all in all emerge --sync && emerge -uND @system being down for >10% > of the time. > > Now, I don't know if you use Gentoo, but I do, and I use it at work, so > having this level of randomization happen is not really useful. > > Tell me then, please - what can I/we do so that this kind of breakage > stops, and we can actually aim at having a most excellent distro? In the > long run I am considering just creating my own clone of all > infrastructure bits so I can fix things, but it's an option that is > needlessly braindead, wasting effort, and not really useful to users > that are not me. Your own infra option would NOT have fixed [2]/[7]/[9]. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] Breakage and frustration
On Sun, Dec 13, 2015 at 12:36 PM, Patrick Lauerwrote: > In the long run I am considering just creating my own clone of all > infrastructure bits so I can fix things I just wanted to comment that things like this should never be viewed as a bad thing. Many contributions to Gentoo arose because somebody basically took something they saw and made an unofficial fork they improved upon. That may or may not have later become official, but it is often beneficial all the same. I'd just suggest that anybody doing this document what they do and publish their sources, so that others can do the same more easily. My ideal state for Gentoo would be one where very little of what we do is centrally housed at all, and infra is just a bunch of servers we tend to use as a matter of convention, but anybody can clone one at any time and set up their own. Federated authentication could let people login to various services without having to actually have access to anything sensitive. This would make it much easier for anybody to offer new services, or to offer patches to infra, and then infra could perhaps be more in the role of a trusted gatekeeper and spend less time having to implement every little thing. That said, it is something that would need to be worked towards. My understanding is that infra is already trying to keep the sensitive stuff separate from the mundane in their newer work, but I'm sure they have a ton of old stuff that hasn't been migrated in this fashion. As with many teams they're probably a bit overloaded, so a big question is how to make it happen without just throwing complaints on the folks who are trying their best to keep it all going. -- Rich