Re: Long delay updating xenocara source tree?
Hi, Jiri B wrote on Thu, Feb 09, 2012 at 07:52:44AM -0500: Somebody suggested: find . -path '*CVS/Root' | xargs rm Never ever! Well, certainly not when you are working in a mixed tree intentionally using different servers for different subdirectories. However, on a laptop i'm carrying around, in a clean tree which i'm using together with various mirrors of the same repository (whichever mirror happens to be closest at any given place), having directories bound to specific servers is merely a nuisance and regularly pruning Root files makes my life easier. Of course, that precludes using cvs -d, and it requires using the CVSROOT environment variable instead, in which case new Root files only pop up in new directories, not in existing directories. I don't even deny that verges abuse of cvs and is hardly how it was intended to be used - but it works well enough for this specific scenario. Hence, not never ever. Yours, Ingo ischwarze@isnote $ env | grep CVS | sort CVSROOT=:ext:schwa...@cvs.openbsd.org:/cvs CVSROOT_CA=anon...@anoncvs.ca.openbsd.org:/cvs CVSROOT_CVS=:ext:schwa...@cvs.openbsd.org:/cvs CVSROOT_DE=anon...@openbsd.cs.fau.de:/cvs CVSROOT_EU=anon...@anoncvs.eu.openbsd.org:/cvs CVSROOT_OSN=anon...@mirror.osn.de:/cvs CVSROOT_SI=anon...@s2k11.obsd.si:/cvs
Re: Long delay updating xenocara source tree?
Stuart Henderson wrote [2012-02-10 01:01+0100]: On 2012-02-09, Steffen Daode Nurpmeso sdao...@googlemail.com wrote: On the long run cvs(1) will die, and be replaced by Schily SCCS oh please don't, even as a joke. It's not that bad! It's from 1972, and only good things can come from 1972. And my daily dose of Hypericum; highly recommendet. we will need a bulk order if we have to use SCCS ;) Somehow the lights have to become brighter! --steffen
Re: Long delay updating xenocara source tree?
Otto Moerbeek wrote [2012-02-03 12:47+0100]: I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Otto, could you please elaborate? How does this slow down a cvs update? Is it really due to the fact that there are multiple different *values* stored in the CVS/Root files? So that cvs needs to consult multiple cvs servers, and *that* is what gets slow? I just checked, and I have three different servers mentioned in my CVS/Root files; a cvs update is slow, but when I have just one server mentioned in the CVS/Root files, it is not faster. Also, specifying 'cvs -d mirror' explicitly should get rid of this problem then (right?), and speed things up, which it doesn't. Or is the slowdown possibly due to the sheer *number* of CVS/Root files that are consulted? Would that be remedied by cvs -d too? Actually, cvs(1) says: -d CVS_root_directory Use CVS_root_directory as the root directory pathname of the master source repository. Overrides the setting of the CVSROOT environment variable. This value should be specified as an absolute pathname. CVS/Root Pathname to the repository ( CVSROOT ) location at the time of checkout. This file is used instead of the CVSROOT environment variable if the environment variable is not set. A warning message will be issued when the contents of this file and the CVSROOT environment variable differ. The file may be over- ridden by the presence of the CVS_IGNORE_REMOTE_ROOT environment variable. It doesn't state explicitly whether 'cvs -d' makes the CVS/Root files ignored. Does it? Also, the wording for 'CVS/Root' doesn't seem to be entirely correct: If I untar, say, sys.tar.gz, then my local tree contains no CVS/Root files, and if I later do a 'cvs -d mirror up -Pd', the tree gets populated with CVS/Root files (mentioning mirror) -- that's not a time of checkout, that's an update. Also, having CVS_IGNORE_REMOTE_ROOT doesn't override CVS/Root, it just makes it ignored, right? Those different entries can be created when doing a cvs up -d that creates a new dir. Not only. A 'cvs -d mirror up -d' of a freshly untared sys.tar.gz creates CVS/Root files all over the place, *not* just in the new directories (that got created due to the 'up -d'). For example it creates ./sys/arch/i386/CVS/Root, while the directory /sys/arch/i386/ definitely was there before (so it didn't get created with 'up -d'). If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. Yes, but it will also change the content of CVS/Root in directories that existed before. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. So, using an explicit cvs root does not ignore the CVS/Root files? Because if it does, then the 'find | xargs rm' should make no difference, right? Now this is really another important issue of scattered information, is it, and it's not noted in anoncvs.html! I've slightly modified your command, i think my version is more secure for use on a webpage. -Otto --steffen Index: anoncvs.html === RCS file: /cvs/www/anoncvs.html,v retrieving revision 1.363 diff -a -p -u -r1.363 anoncvs.html --- anoncvs.html 24 Jan 2012 09:57:35 - 1.363 +++ anoncvs.html 3 Feb 2012 13:33:28 - @@ -542,6 +542,15 @@ add the em-d anon...@anoncvs.ca.openbs # strongcd /usr/src/strong # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong /pre + +p And because cvs(1) stores the name of the server which is in use once +a directory gets created in the file CVS/Root inside this new directory, ^^^ (not only) +it maybe wise to issue a command sequence like the following: +pre + # strongcd /usr/src/strong + # strongfind . -path '*CVS/Root' | xargs rm/strong + # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong +/pre /ul I think this is blind typing without understanding what is going on (not that I know), contrary to the ways of our brilliant FAQ. Let's understand it first. Can someone with internal knowledge of cvs shed some light on this please? Thank you for your time Jan
Re: Long delay updating xenocara source tree?
On Thu, Feb 09, 2012 at 01:16:10PM +0100, Jan Stary wrote: +it maybe wise to issue a command sequence like the following: +pre + # strongcd /usr/src/strong + # strongfind . -path '*CVS/Root' | xargs rm/strong + # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong +/pre /ul Never ever! For example if we would talk about ports tree, 'mystuff' could get taken from different cvs repo. CVS works like this, you can have multiple cvs repos inside the tree, it is by design. Instead of this silly command above, better to tell people to start learning cvs :) jirib
Re: Long delay updating xenocara source tree?
On Thu, Feb 09, 2012 at 01:16:10PM +0100, Jan Stary wrote: Otto Moerbeek wrote [2012-02-03 12:47+0100]: I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Otto, could you please elaborate? How does this slow down a cvs update? Is it really due to the fact that there are multiple different *values* stored in the CVS/Root files? So that cvs needs to consult multiple cvs servers, and *that* is what gets slow? I just checked, and I have three different servers mentioned in my CVS/Root files; a cvs update is slow, but when I have just one server mentioned in the CVS/Root files, it is not faster. Also, specifying 'cvs -d mirror' explicitly should get rid of this problem then (right?), and speed things up, which it doesn't. Or is the slowdown possibly due to the sheer *number* of CVS/Root files that are consulted? Would that be remedied by cvs -d too? I have no idea, I just observerd the slowdown multiple times. -Otto Actually, cvs(1) says: -d CVS_root_directory Use CVS_root_directory as the root directory pathname of the master source repository. Overrides the setting of the CVSROOT environment variable. This value should be specified as an absolute pathname. CVS/Root Pathname to the repository ( CVSROOT ) location at the time of checkout. This file is used instead of the CVSROOT environment variable if the environment variable is not set. A warning message will be issued when the contents of this file and the CVSROOT environment variable differ. The file may be over- ridden by the presence of the CVS_IGNORE_REMOTE_ROOT environment variable. It doesn't state explicitly whether 'cvs -d' makes the CVS/Root files ignored. Does it? Also, the wording for 'CVS/Root' doesn't seem to be entirely correct: If I untar, say, sys.tar.gz, then my local tree contains no CVS/Root files, and if I later do a 'cvs -d mirror up -Pd', the tree gets populated with CVS/Root files (mentioning mirror) -- that's not a time of checkout, that's an update. Also, having CVS_IGNORE_REMOTE_ROOT doesn't override CVS/Root, it just makes it ignored, right? Those different entries can be created when doing a cvs up -d that creates a new dir. Not only. A 'cvs -d mirror up -d' of a freshly untared sys.tar.gz creates CVS/Root files all over the place, *not* just in the new directories (that got created due to the 'up -d'). For example it creates ./sys/arch/i386/CVS/Root, while the directory /sys/arch/i386/ definitely was there before (so it didn't get created with 'up -d'). If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. Yes, but it will also change the content of CVS/Root in directories that existed before. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. So, using an explicit cvs root does not ignore the CVS/Root files? Because if it does, then the 'find | xargs rm' should make no difference, right? Now this is really another important issue of scattered information, is it, and it's not noted in anoncvs.html! I've slightly modified your command, i think my version is more secure for use on a webpage. -Otto --steffen Index: anoncvs.html === RCS file: /cvs/www/anoncvs.html,v retrieving revision 1.363 diff -a -p -u -r1.363 anoncvs.html --- anoncvs.html24 Jan 2012 09:57:35 - 1.363 +++ anoncvs.html3 Feb 2012 13:33:28 - @@ -542,6 +542,15 @@ add the em-d anon...@anoncvs.ca.openbs # strongcd /usr/src/strong # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong /pre + +p And because cvs(1) stores the name of the server which is in use once +a directory gets created in the file CVS/Root inside this new directory, ^^^ (not only) +it maybe wise to issue a command sequence like the following: +pre + # strongcd /usr/src/strong + # strongfind . -path '*CVS/Root' | xargs rm/strong + # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong +/pre /ul I think this is blind typing without understanding what is going on (not that I know), contrary to the ways of our brilliant FAQ. Let's understand it first. Can someone with internal knowledge of cvs shed some light on this please? Thank you for your time Jan
Re: Long delay updating xenocara source tree?
On Thu, Feb 09, 2012 at 07:52:44AM -0500, Jiri B wrote: On Thu, Feb 09, 2012 at 01:16:10PM +0100, Jan Stary wrote: +it maybe wise to issue a command sequence like the following: +pre + # strongcd /usr/src/strong + # strongfind . -path '*CVS/Root' | xargs rm/strong + # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong +/pre /ul Never ever! For example if we would talk about ports tree, 'mystuff' could get taken from different cvs repo. CVS works like this, you can have multiple cvs repos inside the tree, it is by design. Instead of this silly command above, better to tell people to start learning cvs :) jirib Still the observation stands. -Otto
Re: Long delay updating xenocara source tree?
1. On the long run cvs(1) will die, and be replaced by Schily SCCS (oh god, why does v6 not have any branch name info, too??), which will rule the world. Without having the need to use GNU autoconf at all, so that software will compile with make(1) not gmake(1). Make bootstrapping easy. Freedom! Peace!! Love!!! Yup. 2. I like my patch. And my daily dose of Hypericum; highly recommendet. Jan Stary wrote [2012-02-09 13:16+0100]: [.] Actually, cvs(1) says: [.] This is not true for cvs(1) on MacOS X (Snow Leopard), btw. And heaven knows if a Linux system does have any manpages at all! Right?? (TinyCore definitely has none.) Yes, but it will also change the content of CVS/Root in directories that existed before. Huh? But that's right, push the people to it! The current FAQ does not massage CVS/Root related brain cells! I think this is blind typing without understanding I quote myself (quotes removed from sites though, so it may not be remembered 100 percent correctly): I use version control daily, but i don't want to think about it. (It was due to our massive internal hg(1)/git(1) war, and my favorite hg(1) - the user interface is so much better and easier to understand - 'lost due to .. Python and the .. backend.) That is to say: i agree with you. I am psychologically experienced. Can someone with internal knowledge of cvs shed some light on this please? Thank you for your time Jan --steffen
Re: Long delay updating xenocara source tree?
there aren't all that many repositories the size of ours out there. My upload-traffic problem never occurred with binutils, which is also an *incredible* large repository, especially if you up -d. 10% there and my monthly traffic would exhaust, and no begging would help. I have no idea, I just observerd the slowdown multiple times. --steffen
Re: Long delay updating xenocara source tree?
Replying to myself,
On Feb 09 13:16:10, Jan Stary wrote:
Also, specifying 'cvs -d mirror' explicitly should get rid of this
problem then (right?), and speed things up, which it doesn't.
it does; see below.
Or is the slowdown possibly due to the sheer *number* of CVS/Root
files that are consulted? Would that be remedied by cvs -d too?
Actually, cvs(1) says:
-d CVS_root_directory
Use CVS_root_directory as the root directory pathname of
the master source repository. Overrides the setting of the
CVSROOT environment variable. This value should be specified
as an absolute pathname.
CVS/Root
Pathname to the repository ( CVSROOT ) location at the
time of checkout. This file is used instead of the CVSROOT
environment variable if the environment variable is not
set. A warning message will be issued when the contents
of this file and the CVSROOT environment variable differ.
The file may be over- ridden by the presence of the
CVS_IGNORE_REMOTE_ROOT environment variable.
It doesn't state explicitly whether 'cvs -d'
makes the CVS/Root files ignored. Does it?
I think it does: I tested with ktrace.
Without the explicit -d cvsroot, the kdump of 'ktrace cvs -q up -Pd'
contains a lot of sequences like this (ommiting sigprocmask and mprotect):
27461 cvs CALL stat(0x8a0af780,0xcfbf3da8)
27461 cvs NAMI CVS
27461 cvs STRU struct stat {dev=1028, ino=27355, mode=drwxr-xr-x ,
nlink=2, uid=0, gid=0, rdev=117141, atime=1313508893,
stime=1328714573.403561510, ctime=1328714573.403561510, size=512, blocks=4,
blksize=16384, flags=0x0, gen=0x7c8f81d }
27461 cvs RET stat 0
27461 cvs CALL access(0x8a0aff10,0x4R_OK)
27461 cvs NAMI CVS/Root
27461 cvs RET access 0
27461 cvs CALL open(0x8a0aff10,0O_RDONLY,unused0x1b6)
27461 cvs NAMI CVS/Root
27461 cvs RET open 3
27461 cvs CALL fstat(0x3,0xcfbf3cd0)
27461 cvs STRU struct stat {dev=1028, ino=28235, mode=-rw-r--r-- ,
nlink=1, uid=0, gid=0, rdev=113711, atime=1328714573.403561510,
stime=1328714573.403561510, ctime=1328714573.403561510, size=21, blocks=4,
blksize=16384, flags=0x0, gen=0xbde86d5b }
27461 cvs RET fstat 0
27461 cvs CALL read(0x3,0x85d34000,0x4000)
27461 cvs GIO fd 3 read 21 bytes
/home/cvsync/openbsd
27461 cvs RET read 21/0x15
I haven't looked at the cvs source code, but this is
undoubtedly cvs checking if there is a readable CVS/Root,
opening it, and reading it (indeed, /home/cvsync/openbsd
is my locally cvsync'd cvsroot).
Now, with the explicit cvs -d cvsroot, a kdump of
'ktrace cvs -q -d /home/cvsync/openbsd up -Pd' says
3670 cvs CALL access(0x3c00f315,0F_OK)
3670 cvs NAMI CVS/Root
3670 cvs RET access 0
which is just a check whether the file exists.
It is never opened, read from, or written to.
So it seems that 'cvs -d mirror up -d' creates the CVS/Root file
(and writes mirror into it) if the file doesn't exist; if it does
exist, it is ignored.
The exact cause of the slowdown is not known to me. But when you are
switch repositories once in a while it's easy to get this case.
I repair this by find . -name Root | xargs rm and using a explicit cvs
root.
So, using an explicit cvs root does not ignore the CVS/Root files?
Because if it does, then the 'find | xargs rm' should make no difference,
right?
Given the above, doing
find . -name Root | xargs rm
cvs -d mirror up -Pd
actually means *more* work, as it creates and writes all the CVS/Root files.
If they are left there (skip the find | xargs rm), it is only checked that
they exist and left intact -- which is less work, right?
(Then again, I am just reading a kdump. I don't know what CVS really does.)
Here are the tests I did on /usr/src, in the simplest case
of having a local cvsroot, uniform for all files in the tree:
With the CVS/Root files present, all containing /home/cvsync/openbsd:
time cvs -q up -Pd
4m44.15s real 0m4.64s user 0m8.49s system
4m43.67s real 0m4.25s user 0m8.84s system
4m11.57s real 0m4.36s user 0m8.77s system
4m20.84s real 0m4.55s user 0m8.46s system
4m24.09s real 0m4.43s user 0m8.82s system
4m52.91s real 0m4.32s user 0m8.40s system
4m42.52s real 0m4.53s user 0m8.36s system
4m33.40s real 0m4.64s user 0m8.35s system
4m55.80s real 0m4.43s user 0m8.51s system
time cvs -q -d /home/cvsync/openbsd up -Pd
4m24.48s real 0m4.28s user 0m8.35s system
4m21.63s real 0m4.27s user 0m8.43s system
4m14.73s real 0m4.44s user 0m8.43s system
4m13.98s real 0m4.26s user 0m7.93s system
4m17.07s real 0m4.38s user 0m7.90s system
4m18.60s real 0m4.35s user 0m8.77s system
Re: Long delay updating xenocara source tree?
On Thu, Feb 09, 2012 at 06:15:50PM +0100, Jan Stary wrote: Replying to myself, [snip] Things might change if (1) the cvsroot is remote (2) it is not the same for all files. I will try that next. Jan That last case (remote repo's and different Roots) is the actual case I have observerd multipe times. There might be playing other facts though, but I'm curious what that test case shows. BTW, AFAIK, cvs -d up does not write Root files. -Otto
Re: Long delay updating xenocara source tree?
On Feb 09 18:54:45, Otto Moerbeek wrote: BTW, AFAIK, cvs -d up does not write Root files. It does: cd /usr/src ftp -o - ftp://MIRROR/pub/OpenBSD/5.0/sys.tar.gz | tar xzpf cd sys find . -name Root cvs -q -d MIRROR up find . -name Root Jan
Re: Long delay updating xenocara source tree?
On Feb 09 20:10:11, Jan Stary wrote: On Feb 09 18:54:45, Otto Moerbeek wrote: BTW, AFAIK, cvs -d up does not write Root files. It does: cd /usr/src ftp -o - ftp://MIRROR/pub/OpenBSD/5.0/sys.tar.gz | tar xzpf cd sys find . -name Root cvs -q -d MIRROR up find . -name Root Ah, you mean 'write' as opposed to 'create'? What I showed above is that 'cvs -d MIRROR up' _creates_ the CVS/Root in the directories where they did not exist. But you are right, running 'cvs -d OTHERMIRROR up' does _not_ rewrite those CVS/Root files that already exist. Jan
Re: Long delay updating xenocara source tree?
On Thu, Feb 09, 2012 at 08:16:25PM +0100, Jan Stary wrote: On Feb 09 20:10:11, Jan Stary wrote: On Feb 09 18:54:45, Otto Moerbeek wrote: BTW, AFAIK, cvs -d up does not write Root files. It does: cd /usr/src ftp -o - ftp://MIRROR/pub/OpenBSD/5.0/sys.tar.gz | tar xzpf cd sys find . -name Root cvs -q -d MIRROR up find . -name Root Ah, you mean 'write' as opposed to 'create'? What I showed above is that 'cvs -d MIRROR up' _creates_ the CVS/Root in the directories where they did not exist. But you are right, running 'cvs -d OTHERMIRROR up' does _not_ rewrite those CVS/Root files that already exist. Jan I meant 'create' but a test clearly shows that it does create missing Root files. Sorry for the confusion, I must be mixing this up with some other weird cvs behaviour I've seen in the past. -Otto
Re: Long delay updating xenocara source tree?
On 2012-02-09, Steffen Daode Nurpmeso sdao...@googlemail.com wrote: On the long run cvs(1) will die, and be replaced by Schily SCCS oh please don't, even as a joke. anyway they are still releasing new versions of CVS. (they have to, they add new features with security holes...) I like my patch. And my daily dose of Hypericum; highly recommendet. we will need a bulk order if we have to use SCCS ;)
Re: Long delay updating xenocara source tree?
On 02/09/12 12:54, Otto Moerbeek wrote: On Thu, Feb 09, 2012 at 06:15:50PM +0100, Jan Stary wrote: Replying to myself, [snip] Things might change if (1) the cvsroot is remote (2) it is not the same for all files. I will try that next. Jan That last case (remote repo's and different Roots) is the actual case I have observerd multipe times. There might be playing other facts though, but I'm curious what that test case shows. ... cvs does some weird s**t from time to time. I've had the chance to watch it on a public server..it only gets weirder when you watch it there. SOMETIMES -- and I do not know why -- it builds a tree in the cvs server's tmp dir (which is probably something like /repo/anoncvs/tmp in a good mirror), which is as almost as big as the repo it is serving -- I've seen 600+M in size for a single user. That's a lot of disk thrashing, 'specially if it is on the same disk with a few other people doing checkouts or updates, and maybe serving install, too. Yeah, I was able to throw massive amounts of disk I/O and a MFS tmp dir at it, but not all mirrors have that kind of throughput under those circumstances. (keep in mind, on i386, MFS can be no bigger than a hair under 1GB, and 600MB is a non-trivial chunk of that. And if a CVS process disconnects prematurely, the tmp file never gets purged by cvs). I do not know why it does those 600M temp dirs...sometimes (usually?), it does just a few MB. Combine a few people having a nasty update (with the 600MB tmp dirs), slow disk I/O on the server and a lot of disk thrashing...well, uh...no, I STILL can't explain a nine (plus!) hour update, but I do think a large chunk of the problem may be on the server's side. and possibly, if you have multiple servers in different parts of the tree, maybe multiple servers. But no, I'm NOT going to advise people to blow away all their CVS/Root files routinely, for the most part, I find them more useful than harm, but there are a FEW times when it would be appropriate to do so, so a FAQ entry will be coming on this. As trouble resolving thing...great. I've thought about CHANGING all my Root files (as I'm going to have to change my mirror soon...and I'm hoping, soon again after that), simply deleting them and having them recreate is quite nifty, wish I had thought of that. :) Nick.
Re: Long delay updating xenocara source tree?
On Fri, Feb 03, 2012 at 12:47:02PM +0100, Otto Moerbeek wrote: On Thu, Feb 02, 2012 at 03:15:29PM +0100, Steffen Daode Nurpmeso wrote: Henning Brauer wrote: there aren't all that many repositories the size of ours out there. That's true. But no Henning, i don't believe it's that; you know, it's just that i don't have anything to say, because i have no knowledge about the internals of cvs(1). I always thought of this as some kind of misbehaviour in between OpenCVS and GNU cvs, because i would think of cvs(1) as something like this: cvs up . | read CVS/Entries | for those files with diff. timestamps, checksum file | send list [+ checksums] to server | server compare revision/timestamp/[checksum] - client unmodified: send diff (expected final checksum?) - client modified: send full file (if size treshold), otherwise do blockwise checksumming etc. (i.e. rsync-like) [I don't really believe cvs(1) does the latter though.] | integrate diffs / replace locally modified files Wether cvs(1) does do some rsync-like block-checksumming for locally modified files or not, uploading 10% of the repositories size or more before any data is sent from the server just can't be correct anyhow. Even more for my usage case because there were no locally modified files at all. And also the problem goes away if you do specify files directly, as with a file glob, so it makes a difference wether you say $ cvs -fz9 up -PAC . or $ cvs -fz9 up -PAC *.* I don't remember wether i've used -d or not. So for me this turned out as either look into the code, instrument some functions and try to fix it or turn over to cvsync. And GNU cvs is hard to look at, with a lot of comments which refer to some (numeric or so) error reports. But it would surely be interesting to know what is going wrong. --steffen I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Those different entries can be created when doing a cvs up -d that creates a new dir. If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. -Otto My experience is identical to Otto's. Including his fix. :-) Ken
Re: Long delay updating xenocara source tree?
On Thu, Feb 02, 2012 at 03:15:29PM +0100, Steffen Daode Nurpmeso wrote: Henning Brauer wrote: there aren't all that many repositories the size of ours out there. That's true. But no Henning, i don't believe it's that; you know, it's just that i don't have anything to say, because i have no knowledge about the internals of cvs(1). I always thought of this as some kind of misbehaviour in between OpenCVS and GNU cvs, because i would think of cvs(1) as something like this: cvs up . | read CVS/Entries | for those files with diff. timestamps, checksum file | send list [+ checksums] to server | server compare revision/timestamp/[checksum] - client unmodified: send diff (expected final checksum?) - client modified: send full file (if size treshold), otherwise do blockwise checksumming etc. (i.e. rsync-like) [I don't really believe cvs(1) does the latter though.] | integrate diffs / replace locally modified files Wether cvs(1) does do some rsync-like block-checksumming for locally modified files or not, uploading 10% of the repositories size or more before any data is sent from the server just can't be correct anyhow. Even more for my usage case because there were no locally modified files at all. And also the problem goes away if you do specify files directly, as with a file glob, so it makes a difference wether you say $ cvs -fz9 up -PAC . or $ cvs -fz9 up -PAC *.* I don't remember wether i've used -d or not. So for me this turned out as either look into the code, instrument some functions and try to fix it or turn over to cvsync. And GNU cvs is hard to look at, with a lot of comments which refer to some (numeric or so) error reports. But it would surely be interesting to know what is going wrong. --steffen I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Those different entries can be created when doing a cvs up -d that creates a new dir. If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. -Otto
Re: Long delay updating xenocara source tree?
Otto Moerbeek wrote [2012-02-03 12:47+0100]: I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Those different entries can be created when doing a cvs up -d that creates a new dir. If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. Now this is really another important issue of scattered information, is it, and it's not noted in anoncvs.html! I've slightly modified your command, i think my version is more secure for use on a webpage. -Otto --steffen Index: anoncvs.html === RCS file: /cvs/www/anoncvs.html,v retrieving revision 1.363 diff -a -p -u -r1.363 anoncvs.html --- anoncvs.html24 Jan 2012 09:57:35 - 1.363 +++ anoncvs.html3 Feb 2012 13:33:28 - @@ -542,6 +542,15 @@ add the em-d anon...@anoncvs.ca.openbs # strongcd /usr/src/strong # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong /pre + +p And because cvs(1) stores the name of the server which is in use once +a directory gets created in the file CVS/Root inside this new directory, +it maybe wise to issue a command sequence like the following: +pre + # strongcd /usr/src/strong + # strongfind . -path '*CVS/Root' | xargs rm/strong + # strongcvs -d anon...@anoncvs.ca.openbsd.org:/cvs -q up -Pd/strong +/pre /ul p
Re: Long delay updating xenocara source tree?
On Fri, 3 Feb 2012, Otto Moerbeek wrote: On Thu, Feb 02, 2012 at 03:15:29PM +0100, Steffen Daode Nurpmeso wrote: Henning Brauer wrote: there aren't all that many repositories the size of ours out there. That's true. But no Henning, i don't believe it's that; you know, it's just that i don't have anything to say, because i have no knowledge about the internals of cvs(1). I always thought of this as some kind of misbehaviour in between OpenCVS and GNU cvs, because i would think of cvs(1) as something like this: cvs up . | read CVS/Entries | for those files with diff. timestamps, checksum file | send list [+ checksums] to server | server compare revision/timestamp/[checksum] - client unmodified: send diff (expected final checksum?) - client modified: send full file (if size treshold), otherwise do blockwise checksumming etc. (i.e. rsync-like) [I don't really believe cvs(1) does the latter though.] | integrate diffs / replace locally modified files Wether cvs(1) does do some rsync-like block-checksumming for locally modified files or not, uploading 10% of the repositories size or more before any data is sent from the server just can't be correct anyhow. Even more for my usage case because there were no locally modified files at all. And also the problem goes away if you do specify files directly, as with a file glob, so it makes a difference wether you say $ cvs -fz9 up -PAC . or $ cvs -fz9 up -PAC *.* I don't remember wether i've used -d or not. So for me this turned out as either look into the code, instrument some functions and try to fix it or turn over to cvsync. And GNU cvs is hard to look at, with a lot of comments which refer to some (numeric or so) error reports. But it would surely be interesting to know what is going wrong. --steffen I like to say that long delays I have seen when using cvs had to do with multiple different values of CVS/Root files in my local tree. Those different entries can be created when doing a cvs up -d that creates a new dir. If a cvs -d option is used at the same time, the CVS/Root entry for tht dir wil be different than the other's. The exact cause of the slowdown is not known to me. But when you are switch repositories once in a while it's easy to get this case. I repair this by find . -name Root | xargs rm and using a explicit cvs root. -Otto Hmmm. That doesn't seem to [fully] explain the slowdowns I've seen, since I always use an explicit cvs root (following the FAQ) though I certainly have switched repositories from time to time. Dave -- Dave Anderson d...@daveanderson.com
Re: Long delay updating xenocara source tree?
On 2012-02-02, Brett brett.ma...@gmx.com wrote: Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker. And even more speedy when I added a scanfile /root/cvsync-scanfile line in the collection part of the config file. You're running it as root? It makes a lot more sense to use a separate user.
Re: Long delay updating xenocara source tree?
On Thu, 2 Feb 2012 11:40:22 + (UTC) Stuart Henderson s...@spacehopper.org wrote: On 2012-02-02, Brett brett.ma...@gmx.com wrote: Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker. And even more speedy when I added a scanfile /root/cvsync-scanfile line in the collection part of the config file. You're running it as root? It makes a lot more sense to use a separate user. I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time.
Re: Long delay updating xenocara source tree?
On 2012/02/02 23:00, Brett wrote: On Thu, 2 Feb 2012 11:40:22 + (UTC) Stuart Henderson s...@spacehopper.org wrote: On 2012-02-02, Brett brett.ma...@gmx.com wrote: Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker. And even more speedy when I added a scanfile /root/cvsync-scanfile line in the collection part of the config file. You're running it as root? It makes a lot more sense to use a separate user. I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time. I use a dedicated user for this.
Re: Long delay updating xenocara source tree?
And even more speedy when I added a scanfile /root/cvsync-scanfile line in the collection part of the config file. You're running it as root? It makes a lot more sense to use a separate user. I have been updating as root (the few times I have used it so far) but thanks for the reminder, will chown and switch to normal user next time. I use a dedicated user for this. Sounds like a good idea.
Re: Long delay updating xenocara source tree?
Henning Brauer wrote: there aren't all that many repositories the size of ours out there. That's true. But no Henning, i don't believe it's that; you know, it's just that i don't have anything to say, because i have no knowledge about the internals of cvs(1). I always thought of this as some kind of misbehaviour in between OpenCVS and GNU cvs, because i would think of cvs(1) as something like this: cvs up . | read CVS/Entries | for those files with diff. timestamps, checksum file | send list [+ checksums] to server | server compare revision/timestamp/[checksum] - client unmodified: send diff (expected final checksum?) - client modified: send full file (if size treshold), otherwise do blockwise checksumming etc. (i.e. rsync-like) [I don't really believe cvs(1) does the latter though.] | integrate diffs / replace locally modified files Wether cvs(1) does do some rsync-like block-checksumming for locally modified files or not, uploading 10% of the repositories size or more before any data is sent from the server just can't be correct anyhow. Even more for my usage case because there were no locally modified files at all. And also the problem goes away if you do specify files directly, as with a file glob, so it makes a difference wether you say $ cvs -fz9 up -PAC . or $ cvs -fz9 up -PAC *.* I don't remember wether i've used -d or not. So for me this turned out as either look into the code, instrument some functions and try to fix it or turn over to cvsync. And GNU cvs is hard to look at, with a lot of comments which refer to some (numeric or so) error reports. But it would surely be interesting to know what is going wrong. --steffen
Re: Long delay updating xenocara source tree?
Hi Amit, Amit Kulkarni wrote [2012-02-01 00:57+0100]: this motivated me to use cvsync from anoncvs and use a CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster and while initial checkout from cvsync takes 5-10 hrs Yes, that's a real pity. I think it would be great if tarballs would be offered for those, too. I.e. this is *all* of OpenBSD (src, xenocara and www, the latter however refuses papers/, slides/ and songs/ because i also have a checkout of the Steelix translation project), as has been burned onto my last full-backup DVD: $ ls -latr -rw-r--r-- 1 steffen wheel 516407350 26 Jan 15:41 cvsync_openbsd.tbz I think plain gzip wouldn't be that much larger. And hey, it's worth noting that this is smaller than .git/objects/pack of src.git alone! the subsequent updates are lightning quick. Yes. And how easy it now is to do 'cvs up -PAC' to overwrite my messy patches. --amit Ciao, --steffen
Re: Long delay updating xenocara source tree?
Dave Anderson wrote [2012-01-28 15:13+0100]: [.] I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. If you're doing your updates in such a regular manner, i think your best bet is cvsync(1), even if that means additional local storage - but it is *far* more efficient in both, time and traffic. (Not to talk about the possibility to do 'cvs log' and the like locally, without internet connection; if that's an issue.) Pears similar ciao, --steffen thanks... this motivated me to use cvsync from anoncvs and use a CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster and while initial checkout from cvsync takes 5-10 hrs, the subsequent updates are lightning quick. --amit Yes, I also finally got around to trying cvsync because of this thread and its a lot quicker. And even more speedy when I added a scanfile /root/cvsync-scanfile line in the collection part of the config file.
Re: Long delay updating xenocara source tree?
2012/1/31 Dave Anderson d...@daveanderson.com: I do have a slowish ADSL link (384Kbps/1536Kbps) which would limit me to very roughly 1MB/min outbound, so I took advice to use '-z 9' to compress data and that reduced the total time for a xenocara source tree update from about 11 hours to about 2.5 hours. (Though I discovered that not all servers support compression.) if they do anoncvs over ssh, you can ask ssh to compress the data instead of having cvs do it, for those servers. -- To our sweethearts and wives. May they never meet. -- 19th century toast
Re: Long delay updating xenocara source tree?
On Sat, Jan 28, 2012 at 8:43 AM, Steffen Daode Nurpmeso sdao...@googlemail.com wrote: Hey, Dave Anderson wrote [2012-01-28 15:13+0100]: [.] I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. Also true for me. I've run into this problem perhaps a dozen times over the past several months [.] I've noted a lot of upload network traffic when doing 'cvs up' on OpenBSD repos; i.e., before anything else happened about ~70 MB (www) and ~150 MB (src) *upstream* traffic were produced, and it took more than an hour before the download of data began (src). I never had similar problems with anoncvs.mindrot.org, nor cvs.savannah.gnu.org and *.cvs.sourceforge.net etc. If you're doing your updates in such a regular manner, i think your best bet is cvsync(1), even if that means additional local storage - but it is *far* more efficient in both, time and traffic. (Not to talk about the possibility to do 'cvs log' and the like locally, without internet connection; if that's an issue.) Pears similar ciao, --steffen Steffen, i also used to suffer from this unexplained stuff, and yes i used comstyle/spacehopper thanks... this motivated me to use cvsync from anoncvs and use a CVSROOT=/home/amit/MYLOCALREPO to update from cvs. Much much faster and while initial checkout from cvsync takes 5-10 hrs, the subsequent updates are lightning quick. --amit
Re: Long delay updating xenocara source tree?
On Sat, 28 Jan 2012, Nick Holland wrote: On 01/28/12 09:12, Dave Anderson wrote: Thanks for the info. I've been using -Pd because http://www.openbsd.org/anoncvs.html says to use them; I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. and please continue to use them. -Pd is the RIGHT way. I plan to. Dropping them felt kind of iffy; thanks for the confirmation that it isn't the way to go. Apparently, Philip gets away with it, but he's a developer and he knows this stuff pretty well, we don't expect ordinary users to clean up the mess it can make. I'll defer to his expertise on coding and probably CVS, but there are many things in many parts of the tree where a lack of a -Pd will hurt you in ways other than slow updates. There are thousands of ways to use cvs incorrectly, -Pd is the correct way to do updates (or maybe -PAd under some circumstances). And none of this has anything to do with your real problem. I run far slower hardware than most people, and xenocara updates don't take nine hours (and if I understand you, that was nine hours then you gave up and killed it). This has NOTHING TO DO WITH COMMAND LINE OPTIONS. I wrote the FAQ you used, I use that FAQ, and I do it on hardware like mac68k and sparc, and it works, it does not take nine hours to update xenocara (it just feels like it...) No, it was about 9 hours from issuing the cvs update command until there was any visible action; the update ran to completion in a total of about 11 hours. I've killed some other update attempts which ran even longer without any visible action. If you could...next time you see this, use a CVSROOT=anon...@obsd.cec.mtu.edu:/cvs and see if things run better. NOTE: DO NOT GET USED TO USING THIS MIRROR, IT IS BEING SHUT DOWN VERY SOON. But, being it's been being advertised as being shut down, it's pretty lightly loaded, and it handles the CVS temp directory as an mfs, which really really helps (this is on the server end. Nothing you can do about it on your side). My hunch, as a soon-to-be former mirror operator is that you are having a problem with your mirror of choice, not a problem on your end, and it may be a problem with multiple mirrors. I've tried 3 or 4 different servers, and have had this problem with all of them (at least some of the time). I just checked out xenocara from that mirror, and then did an update on my amd64 system, the update took less than one minute. Your results will vary, but not to nine hours, unless you are using dialup. :) I do have a slowish ADSL link (384Kbps/1536Kbps) which would limit me to very roughly 1MB/min outbound, so I took advice to use '-z 9' to compress data and that reduced the total time for a xenocara source tree update from about 11 hours to about 2.5 hours. (Though I discovered that not all servers support compression.) Then I did a test update of xenocara against your server (still using -z 9), and the entire process took barely 1 minute. I then retried that upgrade against the server I've been using (anoncvs.comstyle.com), and the total time was just under 3 minutes. As a final (for the moment) test I did (against my usual server and with -z 9) an update of my entire source tree and the total times were src: 7:37, xenocara: 3:55, ports: 41:58, and www: 2:39 -- for a total of about 55 minutes. I've no idea why I'm suddenly getting so much faster responses. Does cvs update send a potentially large but extremely variable amount of data from my system to the server? If so, that (plus my slowish uplink) might explain some of these timings. But the cause of these massive variations is not at all obvious from where I sit. Thanks for any further info. Dave -- Dave Anderson d...@daveanderson.com
Re: Long delay updating xenocara source tree?
* Steffen Daode Nurpmeso sdao...@googlemail.com [2012-01-28 15:44]: I've noted a lot of upload network traffic when doing 'cvs up' on OpenBSD repos; i.e., before anything else happened about ~70 MB (www) and ~150 MB (src) *upstream* traffic were produced, and it took more than an hour before the download of data began (src). I never had similar problems with anoncvs.mindrot.org, nor cvs.savannah.gnu.org and *.cvs.sourceforge.net etc. there aren't all that many repositories the size of ours out there. If you're doing your updates in such a regular manner, i think your best bet is cvsync(1), even if that means additional local storage - but it is *far* more efficient in both, time and traffic. (Not to talk about the possibility to do 'cvs log' and the like locally, without internet connection; if that's an issue.) Pears similar ciao, that's what many of us do - full repository, synced somehow (i personally rsync from another machine which in turn speaks cvsync to bob) locally. Especially convenient when you sit in an airplane or the like and want to diff... -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de, Full-Service ISP Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully Managed Henning Brauer Consulting, http://henningbrauer.com/
Re: Long delay updating xenocara source tree?
On Fri, 27 Jan 2012, Philip Guenther wrote: On Fri, Jan 27, 2012 at 12:10 PM, Dave Anderson d...@daveanderson.com wrote: I've run into this problem perhaps a dozen times over the past several months while running amd64-current, most recently at 15:53 2012/1/26 EST while running a system built from source updated at about 14:30 2012/1/21 EST: when trying to update the xenocara source tree there is a very long (perhaps infinite) delay between issuing the 'cvs ...' command and the start of any visible activity. In this most recent case the delay was about 9 hours. Updating the src and ports source trees at about the same time and using the same CVSROOT has always worked OK; there's some delay but not a really long one. And sometimes the xenocara update has worked without any problem. When it doesn't, 'rm -rf /usr/xenocara' followed by reloading from the 5.0-release CD has always allowed a subsequent cvs update to work. The command I'm using is # cd /usr/xenocara cvs -d$CVSROOT -q up -Pd (except for the working directory, exactly the same as the command for updating the src or ports tree). I bet it'll be faster if you don't use the -P or -d options. The -d option to cvs up requires the cvs server to walk directories that are present on the server but not present on the client. That includes directories which are now empty because all their files have been removed (ala cvs rm)...of which there are a bunch in the xenocara tree. The -P option requires extra work too, though it's not as bad as the -d option, IIRC. Personally, I use the rule of thumb of only using -d and -P when I have reason to believe directories have been added or removed, either from seeing the commit email or from a build failing because a directory is missing... Thanks for the info. I've been using -Pd because http://www.openbsd.org/anoncvs.html says to use them; I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. Dave -- Dave Anderson d...@daveanderson.com
Re: Long delay updating xenocara source tree?
Hey, Dave Anderson wrote [2012-01-28 15:13+0100]: [.] I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. Also true for me. I've run into this problem perhaps a dozen times over the past several months [.] I've noted a lot of upload network traffic when doing 'cvs up' on OpenBSD repos; i.e., before anything else happened about ~70 MB (www) and ~150 MB (src) *upstream* traffic were produced, and it took more than an hour before the download of data began (src). I never had similar problems with anoncvs.mindrot.org, nor cvs.savannah.gnu.org and *.cvs.sourceforge.net etc. If you're doing your updates in such a regular manner, i think your best bet is cvsync(1), even if that means additional local storage - but it is *far* more efficient in both, time and traffic. (Not to talk about the possibility to do 'cvs log' and the like locally, without internet connection; if that's an issue.) Pears similar ciao, --steffen
Re: Long delay updating xenocara source tree?
On 2012-01-28, Philip Guenther guent...@gmail.com wrote: I've run into this problem perhaps a dozen times over the past several months while running amd64-current, most recently at 15:53 2012/1/26 EST while running a system built from source updated at about 14:30 2012/1/21 EST: when trying to update the xenocara source tree there is a very long (perhaps infinite) delay between issuing the 'cvs ...' command and the start of any visible activity. In this most recent case the delay was about 9 hours. Updating the src and ports source trees at about the same time and using the same CVSROOT has always worked OK; there's some delay but not a really long one. And sometimes the xenocara update has worked without any problem. When it doesn't, 'rm -rf /usr/xenocara' followed by reloading from the 5.0-release CD has always allowed a subsequent cvs update to work. The command I'm using is # cd /usr/xenocara cvs -d$CVSROOT -q up -Pd (except for the working directory, exactly the same as the command for updating the src or ports tree). I bet it'll be faster if you don't use the -P or -d options. If people try this, please don't report any build problems unless you've re-run with both of these options. Also note it's not going to work well for ports..
Re: Long delay updating xenocara source tree?
On 01/28/12 09:12, Dave Anderson wrote: Thanks for the info. I've been using -Pd because http://www.openbsd.org/anoncvs.html says to use them; I haven't yet had a chance to look into how cvs works beyond reading the man page, faq, etc. and please continue to use them. -Pd is the RIGHT way. Apparently, Philip gets away with it, but he's a developer and he knows this stuff pretty well, we don't expect ordinary users to clean up the mess it can make. I'll defer to his expertise on coding and probably CVS, but there are many things in many parts of the tree where a lack of a -Pd will hurt you in ways other than slow updates. There are thousands of ways to use cvs incorrectly, -Pd is the correct way to do updates (or maybe -PAd under some circumstances). And none of this has anything to do with your real problem. I run far slower hardware than most people, and xenocara updates don't take nine hours (and if I understand you, that was nine hours then you gave up and killed it). This has NOTHING TO DO WITH COMMAND LINE OPTIONS. I wrote the FAQ you used, I use that FAQ, and I do it on hardware like mac68k and sparc, and it works, it does not take nine hours to update xenocara (it just feels like it...) If you could...next time you see this, use a CVSROOT=anon...@obsd.cec.mtu.edu:/cvs and see if things run better. NOTE: DO NOT GET USED TO USING THIS MIRROR, IT IS BEING SHUT DOWN VERY SOON. But, being it's been being advertised as being shut down, it's pretty lightly loaded, and it handles the CVS temp directory as an mfs, which really really helps (this is on the server end. Nothing you can do about it on your side). My hunch, as a soon-to-be former mirror operator is that you are having a problem with your mirror of choice, not a problem on your end, and it may be a problem with multiple mirrors. I just checked out xenocara from that mirror, and then did an update on my amd64 system, the update took less than one minute. Your results will vary, but not to nine hours, unless you are using dialup. :) Nick.
Long delay updating xenocara source tree?
I've run into this problem perhaps a dozen times over the past several months while running amd64-current, most recently at 15:53 2012/1/26 EST while running a system built from source updated at about 14:30 2012/1/21 EST: when trying to update the xenocara source tree there is a very long (perhaps infinite) delay between issuing the 'cvs ...' command and the start of any visible activity. In this most recent case the delay was about 9 hours. Updating the src and ports source trees at about the same time and using the same CVSROOT has always worked OK; there's some delay but not a really long one. And sometimes the xenocara update has worked without any problem. When it doesn't, 'rm -rf /usr/xenocara' followed by reloading from the 5.0-release CD has always allowed a subsequent cvs update to work. The command I'm using is # cd /usr/xenocara cvs -d$CVSROOT -q up -Pd (except for the working directory, exactly the same as the command for updating the src or ports tree). This has happened with several different values for CVSROOT, currently # echo $CVSROOT anon...@anoncvs1.usa.openbsd.org:/cvs I am very confused by this. Any clues would be greatly appreciated. Dave -- Dave Anderson d...@daveanderson.com
Re: Long delay updating xenocara source tree?
On Fri, Jan 27, 2012 at 9:10 PM, Dave Anderson d...@daveanderson.com wrote: I've run into this problem perhaps a dozen times over the past several months while running amd64-current, most recently at 15:53 2012/1/26 EST while running a system built from source updated at about 14:30 2012/1/21 EST: when trying to update the xenocara source tree there is a very long (perhaps infinite) delay between issuing the 'cvs ...' command and the start of any visible activity. B In this most recent case the delay was about 9 hours. B Updating the src and ports source trees at about the same time and using the same CVSROOT has always worked OK; there's some delay but not a really long one. B And sometimes the xenocara update has worked without any problem. B When it doesn't, 'rm -rf /usr/xenocara' followed by reloading from the 5.0-release CD has always allowed a subsequent cvs update to work. The command I'm using is B # cd /usr/xenocara cvs -d$CVSROOT -q up -Pd (except for the working directory, exactly the same as the command for updating the src or ports tree). How about: # cd /usr ; cvs -d$CVSROOT -q up -Pd xenocara This has happened with several different values for CVSROOT, currently B # echo $CVSROOT B anon...@anoncvs1.usa.openbsd.org:/cvs I am very confused by this. B Any clues would be greatly appreciated. B B B B Dave -- Dave Anderson d...@daveanderson.com
Re: Long delay updating xenocara source tree?
On Fri, Jan 27, 2012 at 12:10 PM, Dave Anderson d...@daveanderson.com wrote: I've run into this problem perhaps a dozen times over the past several months while running amd64-current, most recently at 15:53 2012/1/26 EST while running a system built from source updated at about 14:30 2012/1/21 EST: when trying to update the xenocara source tree there is a very long (perhaps infinite) delay between issuing the 'cvs ...' command and the start of any visible activity. In this most recent case the delay was about 9 hours. Updating the src and ports source trees at about the same time and using the same CVSROOT has always worked OK; there's some delay but not a really long one. And sometimes the xenocara update has worked without any problem. When it doesn't, 'rm -rf /usr/xenocara' followed by reloading from the 5.0-release CD has always allowed a subsequent cvs update to work. The command I'm using is # cd /usr/xenocara cvs -d$CVSROOT -q up -Pd (except for the working directory, exactly the same as the command for updating the src or ports tree). I bet it'll be faster if you don't use the -P or -d options. The -d option to cvs up requires the cvs server to walk directories that are present on the server but not present on the client. That includes directories which are now empty because all their files have been removed (ala cvs rm)...of which there are a bunch in the xenocara tree. The -P option requires extra work too, though it's not as bad as the -d option, IIRC. Personally, I use the rule of thumb of only using -d and -P when I have reason to believe directories have been added or removed, either from seeing the commit email or from a build failing because a directory is missing... Philip Guenther
Re: xenocara source
On Fri, Feb 29, 2008 at 05:19:00PM -0800, Marco S Hyman wrote: For the sake of new folks it may be wise to put a .cvsignore in our /usr/src tree to prevent unintended cosequences of using the (also suggested) prune switch on cvs (-P). -P will only remove EMPTY directories that cvs knows about. Putting xenocara (or in my case zenocara and ports) in /usr/src is pretty much a no-op when it comes to cvs. A cd /usr/src; cvs up -Pd displays ? xenocara. I can live with that. Still, if we keep recommending /usr/src/xenocara, the ignore would get rid of the ? xenocara noise. I routinely read through the output of cvs update to figure out which files I've got which do not belong. Any extra line will make me go `huh ?' and think about it.
Re: xenocara source
On Sat, 1 Mar 2008, Marc Espie wrote: Still, if we keep recommending /usr/src/xenocara, the ignore would get rid of the ? xenocara noise. Yeah, I agree. I already proposed this several times, but obviously that wasn't an interested for anyone before. -- Antoine
Re: xenocara source
On 2008-03-01, Marc Espie [EMAIL PROTECTED] wrote: Still, if we keep recommending /usr/src/xenocara, the ignore would get rid of the ? xenocara noise. I don't see any ? xenocara noise here, just the normal output from cvs update as it recurses into the xenocara tree.. I thought it was useful to be able to update all the source in one command, but I'm certain there will be other opinions :-)
Re: xenocara source
You can simply add a newline like /xenocara/// # I don't really remember the syntax in the list of directories in a file under /usr/src/CVS thus making all of your sources updated at once without any error. _ Ne gardez plus qu'une seule adresse mail ! Copiez vos mails vers Yahoo! Mail http://mail.yahoo.fr
Re: xenocara source
On 2008-03-01, hyjial [EMAIL PROTECTED] wrote: You can simply add a newline like /xenocara/// # I don't really remember the syntax in the list of directories in a file under /usr/src/CVS thus making all of your sources updated at once without any error. Ah, perhaps I did that at some point then...
Re: xenocara source
On Sat, Mar 01, 2008 at 12:13:53PM +, Stuart Henderson wrote: On 2008-03-01, hyjial [EMAIL PROTECTED] wrote: You can simply add a newline like /xenocara/// # I don't really remember the syntax in the list of directories in a file under /usr/src/CVS thus making all of your sources updated at once without any error. Ah, perhaps I did that at some point then... If you checkout xenocara with cvs it will add the entry for you. But of course it won't happen if you just untar xenocara.tar.gz. -- Pierre Riteau
Re: xenocara source
On Friday 29 February 2008, Marco S Hyman wrote: For the sake of new folks it may be wise to put a .cvsignore in our /usr/src tree to prevent unintended cosequences of using the (also suggested) prune switch on cvs (-P). -P will only remove EMPTY directories that cvs knows about. Putting xenocara (or in my case zenocara and ports) in /usr/src is pretty much a no-op when it comes to cvs. A cd /usr/src; cvs up -Pd displays ? xenocara. I can live with that. When following anoncvs.html, if a new person goes to update their /usr/src tree, they would thwack their /usr/src/xenocara tree. If the directory isn't empty it isn't thwacked If the directory isn't known by CVS it isn't thwacked. You are correct about the thwacking and I failed to actually test it. While trying to build a new -stable box from CD and looking over the site docs/faq, I noticed the discrepancy and added a .cvsignore before running any cvs update commands -But I'm odd that way, because I actually enjoy going through the docs/faq at least once per release and thinking about the suggested commands. And yes, the ? xenocara error/message is really very trivial. But the question still stands; why does our documentation give suggested methods which result in an error/message without explaining it? I'm not concerned about long time users like you or me, or people who are already familiar with UNIX and it's tools. But if this was your very first adventure into CVS, both the docs and ways things work should be clear and correct. Yesterday I sent a diff to www@ to handle the XF4 - xenocara changes that were missed on release in anoncvs.html. Adding a one line statement about .cvsignore and/or ? xenocara to either anoncvs.html or faq5.html would solve this, or alternatively we could just add a .cvsignore to /usr/src It's not a big deal, and is not really a problem that stops stuff from working, but if we want our docs to be as clear and accuate as possible, then we need to make a change (either document the issue, or prevent it). kind regards, jcr
Re: xenocara source
I'm not concerned about long time users like you or me, or people who are already familiar with UNIX and it's tools. But if this was your very first adventure into CVS, both the docs and ways things work should be clear and correct. As a new user of CVS and maintaining an OpenBSD install, I'm certainly a bit confused about this. Even more so now.. what is the suggested method.. I understand that depends on what you're trying to accomplish: I'm not (yet) modifying any code under /usr/src.. so my goal is simply pulling the -stable repositories and updating my system as needed. thanks! ~Jason
Re: xenocara source
On Saturday 01 March 2008, Jason Beaudoin wrote: I'm not concerned about long time users like you or me, or people who are already familiar with UNIX and it's tools. But if this was your very first adventure into CVS, both the docs and ways things work should be clear and correct. As a new user of CVS and maintaining an OpenBSD install, I'm certainly a bit confused about this. Even more so now.. what is the suggested method.. I understand that depends on what you're trying to accomplish: I'm not (yet) modifying any code under /usr/src.. so my goal is simply pulling the -stable repositories and updating my system as needed. Historically, OpenBSD keeps the X server code in it's own branch. You can see the cvs layout here: http://www.openbsd.org/cgi-bin/cvsweb/ There's a description of the brances here: http://www.openbsd.org/anoncvs.html On OpenBSD 4.1 and eariler the suggested layout would be: /usr/src-The system source code /usr/XF4-The X source code /usr/obj-System build object files /usr/objx -The X build object files In 4.1 and earlier, to update both system source and X source, you'd run two separate cvs commands, one for each branch. With the new 4.2 release of OpenBSD the suggested layout has changed. /usr/src-The system source code /usr/src/xenocara -The X source code /usr/obj-System build object files /usr/objx -The X build object files In 4.2 and later, to update both system source and X source, you can supposedly run just one cvs command according to the claim by Owain Ainsworth in this thread, but I personally have not tested this, and the documentation in the FAQ-5 and anoncvs.html do not mention it. I need to test this... If the claim is true, then the docs need more work. The goal of the new layout is to simplify things but since this is new, we're still ironing out the rough spots, namely making sure docs match expected behavior and are consistent (steven@ corrected anoncvs.html just a few hours ago). As for the obvious qusetion of why keep X in it's own branch? the answer is because it makes things simple (surprisingly enough). Keeping the build of X separated from the rest of the system allows one a lot of flexibility... If you want to add a new X server (such as the recent addition of xenocara-X.org), it's a very straight forward change. If you have esoteric/old hardware and need to run XFree86-3 (the X11/ branch), or if you need/want XFree86-4 (the XF4/ branch), you can easily do it. Some folks don't even bother with X, so there's not point in forcing cvs downloads/updates on them when they are just trying to maintain a simple system. Additionally, like Dongsheng Song mentioned, there are people who like keeping their X source separated from their system source tree. Of course, there are people who keep their source trees Read-Only in a central location and export them over nfs to their build machine(s). In short, there's a lot of flexibility, and how *you* want to do it is *your* own choice. The only problem here is all this flexibility tends to be a bit confusing when you're first learning it, so keeping the docs clean, correct and clear is important. Kind Regards, JCR
Re: xenocara source
Yes, I'd like use /usr/xenocara, not /usr/src/xenocara. 2008/3/1, J.C. Roberts [EMAIL PROTECTED]: According to FAQ-5, the suggested place to put your xenocara source tree is under /usr/src/xenocara I know it really doesn't matter where you put your xenocara tree but... For the sake of new folks it may be wise to put a .cvsignore in our /usr/src tree to prevent unintended cosequences of using the (also suggested) prune switch on cvs (-P). When following anoncvs.html, if a new person goes to update their /usr/src tree, they would thwack their /usr/src/xenocara tree. Thoughts? -JCR
Re: xenocara source
For the sake of new folks it may be wise to put a .cvsignore in our /usr/src tree to prevent unintended cosequences of using the (also suggested) prune switch on cvs (-P). -P will only remove EMPTY directories that cvs knows about. Putting xenocara (or in my case zenocara and ports) in /usr/src is pretty much a no-op when it comes to cvs. A cd /usr/src; cvs up -Pd displays ? xenocara. I can live with that. When following anoncvs.html, if a new person goes to update their /usr/src tree, they would thwack their /usr/src/xenocara tree. If the directory isn't empty it isn't thwacked If the directory isn't known by CVS it isn't thwacked. // marc
Re: xenocara source
On Friday 29 February 2008, Dongsheng Song wrote: Yes, I'd like use /usr/xenocara, not /usr/src/xenocara. You can. The addition of a .cvsignore to /usr/src would not remove your ability to keep your xenocara where ever you like. The trouble is the project documentation is misleading. If a new person follows the documentation (faq5.html and anoncvs.html), they will unintentionally delete their /usr/src/xenocara tree which is the suggested location for it according to the documentation. -jcr
Re: xenocara source
On Fri, Feb 29, 2008 at 05:22:34PM -0800, J.C. Roberts wrote: On Friday 29 February 2008, Dongsheng Song wrote: Yes, I'd like use /usr/xenocara, not /usr/src/xenocara. You can. The addition of a .cvsignore to /usr/src would not remove your ability to keep your xenocara where ever you like. The trouble is the project documentation is misleading. If a new person follows the documentation (faq5.html and anoncvs.html), they will unintentionally delete their /usr/src/xenocara tree which is the suggested location for it according to the documentation. Incorrect. I put xenocara in /usr/src/xenocara. in fact a cvs up updates both. Since i reckon i do more in the xenocara tree than most people, I think it's safe to say that it's alright. Prune only removes EMPTY directories that are in cvs. -jcr -- Don't you feel more like you do now than you did when you came in?
