On Sat, Oct 29, 2016 at 5:42 AM, Neil Bothwick <[email protected]> wrote: > On Fri, 28 Oct 2016 23:53:03 -0700, Daniel Campbell wrote: > >> > Anyone seeing this again? I've just sync'd to two servers in >> > Australia, and then, for the hell of it, one in Canada and am getting >> > it for: >> > >> > dev-libs/botan >> > app-arch/tar >> > media-video/libav >> > app-crypt/qca >> > net-print/cups-filters >> > >> > I suppose time will sort it out..... >> > >> > Andrew >> > >> This shouldn't happen unless the distfiles aren't found, or someone (a >> dev) didn't use repoman to commit, leaving an old Manifest around. > > It looks like repoman is the culprit > > https://bugs.gentoo.org/show_bug.cgi?id=598376 >
This is probably not the issue here, since Gentoo uses thin manifests (there is nothing for repoman to update). The manifests that are causing the problem aren't created by regular Gentoo developers. They're created by a script that runs as a part of the rsync mirror process. This is a fairly well-known problem that has been around for over a year. You will only run into this problem if you use rsync to update your repository, since the problem is created when creating the master rsync mirror. The original git repository doesn't contain the error, and the git mirror on github doesn't mess with the Manifests. The issue apparently has to do with Changelog generation. In April the Council gave Infra the option to stop generating Changelogs, which would eliminate the problem. I suspect those maintaining the scripts prefer to keep them around, and I don't think anybody on the Council has access to change the scripts. I switched to git syncing eons ago, so I've never seen this bug. I recognize it has been a source of frustration for a lot of users, and a bit of frustration for the Council, since there doesn't seem to be a lot we can do to change it in practice. zlg is of course right that these kinds of problems can also be caused by maintainer failure to use repoman/etc or if an upstream distfile changes. If that is the problem then you'll see it no matter how you sync your repo. However, when you get a bunch of these after syncing it is almost always a result of the mirror creation process. I can't remember the last time I saw a manifest error (granted, I'm also using mgorny's stable mirror branch, which I think screens for these kinds of errors). While there can be some latency I do in general recommend syncing from https://github.com/gentoo-mirror/gentoo . This is a mirror of the Gentoo developer git repository with two changes: 1. Metadata is added to the mirror, which greatly speeds things up compared to using the raw git repository (you can do this yourself, it is one of the steps done by the rsync generation process as well, but this one is not buggy). 2. The default stable branch of this mirror screens for numerous issues before accepting commits. That means it is generally a little behind the main branch (at this moment the main branch is 2 minutes old, and the default stable branch is 20min old), but a lot of the really annoying issues that are caused by devs skipping repoman won't be seen. Now, if a maintainer breaks a package then this mirror will quickly get out of date until the problem is corrected, but Gentoo QA gets warnings when this is happening and usually the maintainer is being pestered or somebody else is fixing it. I suspect this process has probably reduced the error rate for everybody. I have seen this get a few days old though, which is something to keep in mind. It does not contain Changelogs, though if you use it you'll have a full history so you can just run git whatchanged <path> to get something pretty close to a changelog. To use it just put this in /etc/portage/repos.conf/gentoo.conf: [DEFAULT] main-repo = gentoo [gentoo] location = /usr/portage sync-type = git sync-uri = https://github.com/gentoo-mirror/gentoo.git auto-sync = yes If you want to git-sync from some other mirror, just change the url accordingly. If you switch your mirror I suggest just renaming /usr/portage and letting portage re-create it. The other big benefit of git syncing is that if you sync every day it is a lot faster. If you sync less often it will become slower compared to rsync. git is much more efficient at finding what has changed, but rsync is not burdened with transferring a complete history. If you only sync once every few months rsync will be a lot faster. -- Rich
