On Fri, 1 Apr 2005, Hrvoje Niksic wrote:
> Behdad Esfahbod <[EMAIL PROTECTED]> writes:
>
> > Thanks. I tried the CVS version and the 1.8.2 version, on NFS,
> > using a loop like yours, couldn't reproduce the problem.
>
> I am told that O_EXCL has worked just fine on NFS for many years now.
> The open(2) man page on Linux is either outdated or assumes ancient or
> broken NFS implementations.
Well, the network I'm on is the Computer Science department's
graduate students and faculty's network, a large Linux and
Solaris based network. A couple months ago I wrote a distributed
crawler and ran them on some 100 machines with a shared NFS
file-system as the shared memory, for locking and all, and I
could easily observe all kind of problems with NFS, including
this O_EXCL one. The Python code I came up with was:
def get_lock_file_safe(file):
"""Acquire a lock file and return true if successful, NFS safe."""
# The protocol described in `man open' under O_EXCL entry
ans = False
try:
os.unlink(lock)
except OSError:
pass
os.close(os.open(lock, os.O_CREAT, 0600))
# Looks like NFS sometimes loses our created lock file, try
# twice with a small delay
time.sleep(0.05)
os.close(os.open(lock, os.O_CREAT, 0600))
try:
os.link(lock, file)
ans = True
except OSError:
if os.stat(lock).st_nlink == 2:
ans = True
os.unlink(lock)
return ans
Now if you say newer Linux kernels are fine with it, it's fair
enough to not bother in wget, just wanted to say that there are
those ancient systems still in large use.
Answering the other email here too, what's wrong with creating
the file before the actual download begins? I mean, leaving
empty (0 bytes) files around is not dirtier than leaving
incomplete downloads in case of an interrupted run. The why not
just openning it when promising?
--behdad
http://behdad.org/
