Re: Sorry state of the rsync based CVS,replication
On Sun, 2010-11-14 at 12:13 +0100, Simon L. B. Nielsen wrote: There is nothing which prevents mirror sites from providing access to the CVS repo via rsync, even if they get it via CVSup... I went ahead with adding this to ftp2.freebsd.org: % rsync ftp2.freebsd.org::FreeBSD-CVS/ drwxr-xr-x 512 2010/08/02 13:35:40 . drwxr-xr-x 512 2010/08/02 22:09:36 gnats drwxr-xr-x 512 2010/08/03 02:34:06 mail drwxr-xr-x 512 2010/08/02 21:50:04 ncvs drwxr-xr-x 512 2010/08/03 00:51:26 www % -- Ken Smith - From there to here, from here to | kensm...@buffalo.edu there, funny things are everywhere. | - Theodor Geisel | signature.asc Description: This is a digitally signed message part
Re: Sorry state of the rsync based CVS,replication
On 12 Nov 2010, at 09:47, Patrick Bihan-Faou wrote: Don't take this as flamebait, because I have no intention in starting a war on this particular issue, but as good as cvsup is, this is unfortunately a fairly isolated tool that, from my prospective (which is necessarily biaised and incomplete), does not offer any feature compelling enough to prefer it over rsync in our case. That position is by essence just a personal view, applicable to me only and not to anybody else. Also I have to admit that now that the m3 dependency is gone with csup, it becomes easier to return to it. The issue is not to remove CVS via rsync - just to remove it from the FTP collection where it doesn't belong. There is nothing which prevents mirror sites from providing access to the CVS repo via rsync, even if they get it via CVSup... If it's useful (IE, any of the primary mirrors requests it) we can probably rather easily set up rsync access via cvsup-master. That said, I think rsync access is likely not too interesting for most master mirrors as they likely provide access to the repo via CVSup already, so they have cvsup installed already. -- Simon L. B. Nielsen Hat: FreeBSD.org clusteradm ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Sorry state of the rsync based CVS,replication
Hi, [...] Regarding the permission of the Attic subdirs in place The development/ section of the FTP site is something I hadn't looked at before so it took me a little time to find what populates it and investigate a little. I *think* the issue with the Attic directories not including world-read permissions was either an issue with a badly formed chmod(1) done a long time ago or an issue with the mechanism that populates that portion of the FTP site missing a umask setting in the script that does it some time back in history (it's there now). Not all of the Attic directories had the wrong permissions, it seemed to stop some time in 2007. I adjusted the permissions on ftp-master so hopefully this issue is fixed. However ... Great news. I'll check various rsync source later and see if the situation improves. We are moving to svn and svnsync for the freebsd source tree (and I am happy with this), but the ports do not seem to be available using SVN (or not in a documented way). Can something be done to restore RSYNC mirroring of the CVS tree to a working state ? The FTP site desperately needs to go on a diet so we're poking around to see if there is some stuff that can be dropped. This section of the site is a candidate for being removed. As you say the ports are not available in SVN but I'm curious why you use the content from the FTP site instead of just using a CVSUP mirror. Is there some benefit to it? We would sort of like to stop providing this as part of the FTP site if there really isn't any benefit to it over using the cvsup mirror infrastructure which won't be going away any time soon. The benefit is organisational to us. We did use cvsup in the past but it has been a pain to maintain as all the other external sources we keep in sync with use rsync. That combined with the requirement for m3 etc. for the sole purpose of syncing the CVS tree when there are alternatives (rsync for the freebsd cvs tree and more recently svnsync) made the switch to rsync a no brainer (note that this decision was taken long before csup came to life). Don't take this as flamebait, because I have no intention in starting a war on this particular issue, but as good as cvsup is, this is unfortunately a fairly isolated tool that, from my prospective (which is necessarily biaised and incomplete), does not offer any feature compelling enough to prefer it over rsync in our case. That position is by essence just a personal view, applicable to me only and not to anybody else. Also I have to admit that now that the m3 dependency is gone with csup, it becomes easier to return to it. Now if the plan is to eliminate rsync CVS mirroring (and I am in no position to criticize such a decision that is very well justified in your mail), we will of course adapt and go to one of the supported methods for CVS tree mirroring. If this means cvsup only, then we will do it without any hard feeling. I already figured that we must be almost the only ones to use rsync for CVS tree mirroring (otherwise that issue would have been detected and fixed a long time ago), and I don't expect (nor demand) the project to maintain a service for just one user, this would be ridiculous and unreasonable. Anyways, thank you very much for taking the time to look at this issue and hopefully having fixed it. This is really greatly appreciated. Best regards, Patrick. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Sorry state of the rsync based CVS,replication
On Wed, 2010-10-27 at 14:45 +0200, Patrick Bihan-Faou wrote: We use FreeBSD extensively and keep a local mirror of the CVS repository. Up until recently things where working properly with the various servers listed in http://www.freebsd.org/doc/handbook/mirrors-rsync.html, but sometime during the summer ftp13.freebsd.org did not respond anymore and since then rsync replication is broken. The main issue (besides the removal of ftp13.freebsd.org) is that most rsync sources refuse to replicate the content of the .Attic directories in the CVS tree. This means that performing a check-out on ports using a tag usually won't work as some files will not be there anymore. Here are the typical logs I get using most rsync servers: rsync: opendir /3/freebsd-core/development/FreeBSD-CVS/ports/chinese/pcmanx/files/Attic (in vol) failed: Permission denied (13) At this moment the only rsync server that provides an adequate replication of the CVS repository is ftp2.tw.FreeBSD.org. As others have reported this was caused by the permissions on the Attic directories not including world read permission. For sites where it was working it's actually an indication they're not following best practices for a mirror site. It's typically a bad idea to have the thing that allows access to the content of the mirror site running with the same credentials as what keeps the mirror site up to date. We don't use the 'feature' that allow for (pre-staging content that the world shouldn't have access to for a period of time, allowing the mirror sites to get fully populated before the release date) but I know of other projects that do. The ftp-master machines don't have that in place because they're not public and they need to allow the blessed mirrors access to everything (for the purposes of pre-staging, if we were actually using that feature...). The development/ section of the FTP site is something I hadn't looked at before so it took me a little time to find what populates it and investigate a little. I *think* the issue with the Attic directories not including world-read permissions was either an issue with a badly formed chmod(1) done a long time ago or an issue with the mechanism that populates that portion of the FTP site missing a umask setting in the script that does it some time back in history (it's there now). Not all of the Attic directories had the wrong permissions, it seemed to stop some time in 2007. I adjusted the permissions on ftp-master so hopefully this issue is fixed. However ... We are moving to svn and svnsync for the freebsd source tree (and I am happy with this), but the ports do not seem to be available using SVN (or not in a documented way). Can something be done to restore RSYNC mirroring of the CVS tree to a working state ? The FTP site desperately needs to go on a diet so we're poking around to see if there is some stuff that can be dropped. This section of the site is a candidate for being removed. As you say the ports are not available in SVN but I'm curious why you use the content from the FTP site instead of just using a CVSUP mirror. Is there some benefit to it? We would sort of like to stop providing this as part of the FTP site if there really isn't any benefit to it over using the cvsup mirror infrastructure which won't be going away any time soon. Thanks. -- Ken Smith - From there to here, from here to | kensm...@buffalo.edu there, funny things are everywhere. | - Theodor Geisel | signature.asc Description: This is a digitally signed message part
Sorry state of the rsync based CVS,replication
Hi, We use FreeBSD extensively and keep a local mirror of the CVS repository. Up until recently things where working properly with the various servers listed in http://www.freebsd.org/doc/handbook/mirrors-rsync.html, but sometime during the summer ftp13.freebsd.org did not respond anymore and since then rsync replication is broken. The main issue (besides the removal of ftp13.freebsd.org) is that most rsync sources refuse to replicate the content of the .Attic directories in the CVS tree. This means that performing a check-out on ports using a tag usually won't work as some files will not be there anymore. Here are the typical logs I get using most rsync servers: rsync: opendir /3/freebsd-core/development/FreeBSD-CVS/ports/chinese/pcmanx/files/Attic (in vol) failed: Permission denied (13) At this moment the only rsync server that provides an adequate replication of the CVS repository is ftp2.tw.FreeBSD.org. We are moving to svn and svnsync for the freebsd source tree (and I am happy with this), but the ports do not seem to be available using SVN (or not in a documented way). Can something be done to restore RSYNC mirroring of the CVS tree to a working state ? Best regards, Patrick Bihan-Faou ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
