RE: [distcc] Local vs. NFS lock files.
--- Martin Pool [EMAIL PROTECTED] wrote: On Mon, 2005-04-04 at 22:25 -0500, [EMAIL PROTECTED] wrote: them locked. With file creation, instead of file locking, we can look at the date the file was created, and choose to ignore its existence if it is too old. These seems like an awefully kludged hack. All that is needed is a semaphore that is always correct. That's right. The question is, what mechanism to use to get it? A host server, like I've implemented... :-) nudge, nudge. Vic -- Martin __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc __ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
RE: [distcc] Local vs. NFS lock files.
Care to elaborate on implementing a semaphore that is always correct? Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Monday, April 04, 2005 8:26 PM To: Donohue, Michael Cc: Martin Pool; distcc@lists.samba.org; Daniel Kegel Subject: RE: [distcc] Local vs. NFS lock files. them locked. With file creation, instead of file locking, we can look at the date the file was created, and choose to ignore its existence if it is too old. These seems like an awefully kludged hack. All that is needed is a semaphore that is always correct. __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
Re: [distcc] Local vs. NFS lock files.
On Wed, 2005-03-30 at 12:49 -0700, Donohue, Michael wrote: This is enough to convince me that NFS locking isn't hurting us at PayPal, anyway. What exactly are the issues that arise elsewhere? The main problem is that many people have NFS half-working, so that file IO works but locks don't work. This might be because NFS locks are broken in your software (old Unix?) or because you forgot to run the lock server (easily done on linux). Because locks and IO are handled by different daemons and different protocols it's easy to have this and not notice. (In NFS4 they're a single protocol.) Furthermore it seems that if locks aren't working, the client OS will often just grant all locks. In general NetApp servers seem to be the one case where NFS does work pretty reliably... which is nice for netapp owners, but not so good for conservative design. :-) There is also this: http://www.mail-archive.com/distcc@lists.samba.org/msg01325.html If you know that NFS locks work then it's fine for you to put DISTCC_DIR there. It's not a great long-term solution for coordinating between machines because obviously not everyone has a shared disk. -- Martin signature.asc Description: This is a digitally signed message part __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
RE: [distcc] Local vs. NFS lock files.
It looks like a common workaround to broken or missing NFS file locks is the use of dot-lockfiles. The claim is that file creation on NFS is atomic, so creating a file with a well known name is a suitable replacement for advisory locking. Given that distcc isn't actually writing data into the lock files, then they are just advisory locks. Can we perhaps substitute file creation for file locks, as a more conservative way of handling this?Has this been considered before? This may also solve one of our outstanding problems with lock files on NFS, where they remain locked even though there are no processes holding them locked. With file creation, instead of file locking, we can look at the date the file was created, and choose to ignore its existence if it is too old. With file locks, you get no other information besides the fact that it is locked. Michael -Original Message- From: Martin Pool [mailto:[EMAIL PROTECTED] Sent: Monday, April 04, 2005 6:18 AM To: Donohue, Michael Cc: Daniel Kegel; distcc@lists.samba.org Subject: Re: [distcc] Local vs. NFS lock files. On Wed, 2005-03-30 at 12:49 -0700, Donohue, Michael wrote: This is enough to convince me that NFS locking isn't hurting us at PayPal, anyway. What exactly are the issues that arise elsewhere? The main problem is that many people have NFS half-working, so that file IO works but locks don't work. This might be because NFS locks are broken in your software (old Unix?) or because you forgot to run the lock server (easily done on linux). Because locks and IO are handled by different daemons and different protocols it's easy to have this and not notice. (In NFS4 they're a single protocol.) Furthermore it seems that if locks aren't working, the client OS will often just grant all locks. In general NetApp servers seem to be the one case where NFS does work pretty reliably... which is nice for netapp owners, but not so good for conservative design. :-) There is also this: http://www.mail-archive.com/distcc@lists.samba.org/msg01325.html If you know that NFS locks work then it's fine for you to put DISTCC_DIR there. It's not a great long-term solution for coordinating between machines because obviously not everyone has a shared disk. -- Martin __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
RE: [distcc] Local vs. NFS lock files.
them locked. With file creation, instead of file locking, we can look at the date the file was created, and choose to ignore its existence if it is too old. These seems like an awefully kludged hack. All that is needed is a semaphore that is always correct. -Original Message- From: Martin Pool [mailto:[EMAIL PROTECTED] Sent: Monday, April 04, 2005 6:18 AM To: Donohue, Michael Cc: Daniel Kegel; distcc@lists.samba.org Subject: Re: [distcc] Local vs. NFS lock files. On Wed, 2005-03-30 at 12:49 -0700, Donohue, Michael wrote: This is enough to convince me that NFS locking isn't hurting us at PayPal, anyway. What exactly are the issues that arise elsewhere? The main problem is that many people have NFS half-working, so that file IO works but locks don't work. This might be because NFS locks are broken in your software (old Unix?) or because you forgot to run the lock server (easily done on linux). Because locks and IO are handled by different daemons and different protocols it's easy to have this and not notice. (In NFS4 they're a single protocol.) Furthermore it seems that if locks aren't working, the client OS will often just grant all locks. In general NetApp servers seem to be the one case where NFS does work pretty reliably... which is nice for netapp owners, but not so good for conservative design. :-) There is also this: http://www.mail-archive.com/distcc@lists.samba.org/msg01325.html If you know that NFS locks work then it's fine for you to put DISTCC_DIR there. It's not a great long-term solution for coordinating between machines because obviously not everyone has a shared disk. -- Martin __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
RE: [distcc] Local vs. NFS lock files.
On Mon, 2005-04-04 at 22:25 -0500, [EMAIL PROTECTED] wrote: them locked. With file creation, instead of file locking, we can look at the date the file was created, and choose to ignore its existence if it is too old. These seems like an awefully kludged hack. All that is needed is a semaphore that is always correct. That's right. The question is, what mechanism to use to get it? -- Martin signature.asc Description: This is a digitally signed message part __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
Re: [distcc] Local vs. NFS lock files.
--- Donohue, Michael [EMAIL PROTECTED] wrote: [... snip ...] This is enough to convince me that NFS locking isn't hurting us at PayPal, anyway. What exactly are the issues that arise elsewhere? Doing file locking over NFS simply didn't work for us here at Marconi -- with our older Solaris machines (running Solaris 2.5, 2.6, 2.7, 2.8, or 2.9), linux boxes (of varying sorts), and an older (I think) NetApps box. Vic Michael __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc __ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
