Re: Race condition in wget

2005-04-01 Thread Hrvoje Niksic
Behdad Esfahbod [EMAIL PROTECTED] writes: If I use the 1.8.2 version, although I get 100 different log files, but get only 14 index.html files. And this was a bug, because those HTML files are likely to be both overwritten and concurrently written to by, on average, 7.14 Wget processes per

Re: Race condition in wget

2005-04-01 Thread Hrvoje Niksic
Behdad Esfahbod [EMAIL PROTECTED] writes: Thanks. I tried the CVS version and the 1.8.2 version, on NFS, using a loop like yours, couldn't reproduce the problem. I am told that O_EXCL has worked just fine on NFS for many years now. The open(2) man page on Linux is either outdated or assumes

Re: Race condition in wget

2005-04-01 Thread Behdad Esfahbod
On Fri, 1 Apr 2005, Hrvoje Niksic wrote: Behdad Esfahbod [EMAIL PROTECTED] writes: Thanks. I tried the CVS version and the 1.8.2 version, on NFS, using a loop like yours, couldn't reproduce the problem. I am told that O_EXCL has worked just fine on NFS for many years now. The open(2)

Re: Race condition in wget

2005-04-01 Thread Hrvoje Niksic
Behdad Esfahbod [EMAIL PROTECTED] writes: I am told that O_EXCL has worked just fine on NFS for many years now. The open(2) man page on Linux is either outdated or assumes ancient or broken NFS implementations. Well, the network I'm on is the Computer Science department's graduate students

Re: Race condition in wget

2005-03-31 Thread Behdad Esfahbod
On Wed, 30 Mar 2005, Hrvoje Niksic wrote: Behdad Esfahbod [EMAIL PROTECTED] writes: Well, sorry if it's all nonsense now: Last year I sent the following mail, and got a reply confirming this bug and that it may be changed to use pid instead of a serial in log filename. Recently I was

Re: Race condition in wget

2005-03-31 Thread Behdad Esfahbod
Sorry for the noise, I just noticed: When testing with: for x in `seq 1 100`; do wget -b http://behdad.org/; done If I use the 1.8.2 version, although I get 100 different log files, but get only 14 index.html files. With the CVS, I see in the log that it's trying to find a nonexistence file:

Re: Race condition in wget

2005-03-30 Thread Hrvoje Niksic
Behdad Esfahbod [EMAIL PROTECTED] writes: Well, sorry if it's all nonsense now: Last year I sent the following mail, and got a reply confirming this bug and that it may be changed to use pid instead of a serial in log filename. Recently I was doing a project and had the same problem, I found

Re: Race condition in wget

2005-03-29 Thread Behdad Esfahbod
On Sun, 18 Jan 2004, Behdad Esfahbod wrote: Hi, I'm using wget from Fedora development snapshot, which introduces itself as GNU Wget 1.9+cvs-stable. I think there is a race condition in wget when allocating a log file. Here is a log to show that. What happens is that my system is under a bit

Re: Race condition in wget

2004-01-19 Thread Hrvoje Niksic
You're right, it is a race condition. You could avoid it by specifying explicit log file names which somehow depend on Wget's PID. Avoiding race conditions in the general case is non-trivial (there is O_EXCL, but it doesn't always work under NFS). Maybe a better solution would be to change the

Race condition in wget

2004-01-18 Thread Behdad Esfahbod
Hi, I'm using wget from Fedora development snapshot, which introduces itself as GNU Wget 1.9+cvs-stable. I think there is a race condition in wget when allocating a log file. Here is a log to show that. What happens is that my system is under a bit of load, so the first (almost) twenty wgets