The two most common errors that occur in the rsync-gen process are missing DIST entries in the Manifest committed to git, and metadata.xml that fails to validate against the DTD. Two much less common errors that have occurred in the last 24 hours which prompted this email are FILESDIR items that are way too large (>60 KiB), and completely missing metadata.xml. All four of these errors are fatal to the rsync-gen process, and for good reason.
Anything fatal to rsync-gen completely stops the git to rsync conversion, and the rsync tree remains at the state it was before the failure until the failure is resolved. If it isn't apparent, this completely blocks all updates, GLSA distribution, news item distribution, and anything else distributed by the rsync tree either now or in the future, until the problem is fixed. If you're curious about the current age of the rsync tree due to failures, you can retrieve the metadata/timestamp or metadata/timestamp.x (not timestamp.chk) file from rsync.gentoo.org and inspect its contents (or do a verbose listing of the file, and look at the mtime). The rsync.gentoo.org rotation only contains servers managed by infra, which sync against the private master rsync server very frequently, so you can be reasonably sure they're about as up to date as possible. Explanation of the various timestamp files: * timestamp: created by rsync-gen script in the staging directory immediately after md5-cache generation and manifest thickening is complete. * timestamp.x: created by cron job periodically (script comment says every 5 minutes, but I think it's less often than that) in the staging directory * timestamp.chk: created by cron job periodically (again, script comment says every 5 minutes, but I think it's every 15 minutes) in the *final* directory The final directory is what is served by the private master rsync server, so you can use timestamp.chk to see the age of your local mirror relative to the private master rsync server. The staging directory is transferred to the final directory as part of successful completion of the rsync-gen script (it's the last step), so if anything bails before that, it doesn't happen (thus, timestamp and timestamp.x are mostly equivalent in meaning). Why do these happen? The answer is very simple: you forgot to run repoman in the package directory before pushing your changes. Alternately, the metadata.xml failing to validate error can happen if your repoman is old. So what can you do about it? First, make sure repoman is installed and up to date. As of portage 2.3.0, repoman is a separate package (app-portage/repoman) with its own release schedule, so once you've updated portage past that version, you need to install repoman yourself. Repoman needs to be at least version 2.3.0 as well (which it will be if you've installed portage 2.3.0 or newer), because the metadata.xml validation check does not exist in any previous version, and as mentioned, that check is fatal to the rsync-gen process. Second, and I cannot stress this enough, *run repoman.* Before you push any changes to a package, run repoman manifest followed by at least repoman manifest-check. Repoman manifest will ensure that all your DIST entries are there, and repoman manifest-check will cover the other three issues I mentioned. Repoman's scan and full commands also include the checks done by manifest-check, so if you're running one of those already prior to pushing your changes, you don't need to do repoman manifest-check as well. Note: pkgcore's repoman-alike (I believe it's called pcheck) does not behave the same as repoman as of this writing, and some checks it does not consider fatal are fatal to repoman (eg the FILESDIR item larger than 60 KiB), so if you're relying on pcheck for your QA checks, you should also do a repoman manifest-check for good measure. This should go without saying, but if you modify an eclass in a way that affects SRC_URI for any package, you should go through the packages that use that eclass and run repoman manifest on them. Remember, QA includes YOU! (This also applies to all of you submitting pull requests or otherwise proxy maintaining packages, and all members of proxy-maint project; you should all be doing these QA steps before you submit or push changes.) Sincerely, The guy who gets emailed every half hour because you broke the rsync conversion until it's fixed, also known as Doug or dwfreed.
