Re: Mandatory locking?
[EMAIL PROTECTED] (Terry Lambert) writes: I think this has been the basis of your objection so far. If so, it's a fundamental misunderstanding of "mandatory". In this context What I was objecting to were some of the arguments made by Greg Lehey and Wes Peters, both of whom explicitly stated that opening does not block. It had nothing to do with mandatory locking beyond that (quite possibly flawed) interpretation. By your definiton of explicit, no. However, explicit locking is voluntary, just as advisory locking is voluntary, in terms of whether programs participate (or not). This pretty much means that explicit locking degrades to advisory locking, in the presence of (un)intentionally non-participatory programs. That's basically what my objection was. The "deadlock prone" objection made by others applies more strongly to implicit locking, and is also valid. It can take quite a bit of care to ensure that there is always a maintenance path to the system that allows a sufficient environment to be used without blocking on locked files to allow root to get in and kill any processes causing problems. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 19:53:22 -0400, Christian Kuhtz wrote:
On Thu, Aug 26, 1999 at 09:09:33AM +0930, Greg Lehey wrote:
On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote:
All the files under Tandem's NSK has mandatory locking. The file cannot be
opened if another process has it opened. some thing like
* if the file is opened for reading, any one can open it for
reading but opening for writing gives error
* if the file is open for writing, it can't be opened for
read/write
* if the process holding the file is killed, the lock is gone
* it is possible to get the pid of the process(es) which has
a given file open (like which process has file "xyz" open?
kind of query). btw, is there any way to get this info now in FBSD?
This sounds interesting...
But - aren't there NFS issues? I mean, in stateless access to
a file - how do you know if the process holding the file is killed
if it's remote?
NSK is a prorietary operating system ("NonStop Kernel", previously
known as Guardian, previously known as TOS), not UNIX. There is no
NFS, and there is no distinction between network access and local
access: all goes over the message system. When a file is closed, its
locks are released.
How does this message system handle releasing stale locks when components
talking to the message system die unexpectedly? Is there some sort of aging,
override or timeout mechanism to purge stale locks?
When a process dies unexpectedly, the message system releases all
resources which were allocated to it. This includes, of course,
closing its files. If the process is on the local processor, the
locak processor cleans up. If the local processor dies, the remote
systems are informed and do the cleanup for all resources used by
processes on that processor.
Greg
--
See complete headers for address, home page and phone numbers
finger [EMAIL PROTECTED] for PGP public key
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 18:25:31 +, Terry Lambert wrote: And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. Har! 8-). I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Har again! 8-). I continue to be amazed. That's why I've been keeping out of this discussion. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. UNIX doesn't do this _at all_ well for "hosted OS's". The NetWare for UNIX product, which I worked on the attributed FS (with resource forks and multiple namespaces) and the lock coherency model on is one example. We had to add hooks into the fcntl() call in the FS to support this. Something that might be interesting to look at in this regard is the latest SAMBA code. The latest SAMBA code has an OS OPLOCK (Opportunity Locking) API that it will utilize, if it is present (currently only on IRIX), which deals with the lock coherency issues. Note that since SAMBA externalizes via SMB an interface that has to implement NetBIOS calls, and those, in turn, externalize via the DOS INT 2A/2C mechanisms into the file I/O INTs, that SAMBA has to support mandatory locking. How does it do this under FreeBSD? Does it implement it internally? In effect, it is an API which externalizes much of the same types of operations that implement LEASE operations used by the current FreeBSD NFS server implementation. I don't know if this would be quite sufficient (it's been a while since I did a lease and order of operation audit on the VFS code, and written up patches to support Jordan's class project of making the NFS server locking work), but it should be a healthy start down the right road, I think. Might even fix a couple of NFS bugs as a side benefit... Anyway, Sean's also right about needing mandatory file locks for binary emulation of other platforms (some of which Sean coded for). Are file locks sufficient for Vinum, Greg? To repeat myself again: none of this is relevant to Vinum. The original problem I was looking out turned out not to require any other locking method than was already present in Vinum. The current lock structures in FreeBSD are hung off the backing inodes (of which specfs has none available that are not themselves abstract), and the spec_advlock() function returns either EOPNOTSUPP or EINVAL, based on the arguments its given. IMO, unless the lock list is hung off the vnode (I guess you could hang the mandatory locks there, and leave the advisory ones alone, but that's kind of a waste, especially if the locking code can be reused), you aren't going to be able to do range locks on your stripes. Where are advisory locks kept nowadays? I'll discuss this in a separate message. Am I not understanding how Vinum's RAID 5 is implemented? Are you using files for the stripes, or are you laying out the disk as a true RAID 5 controller would lay it out? I'm laying out the disk in the same manner as a hardware RAID-5 controller would do it. Vinum's downward interface is to the disk driver, not the file system. I'll send a separate (hopefully last) message to try and sum up what I'm doing. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
tlamb...@primenet.com (Terry Lambert) writes: I think this has been the basis of your objection so far. If so, it's a fundamental misunderstanding of mandatory. In this context What I was objecting to were some of the arguments made by Greg Lehey and Wes Peters, both of whom explicitly stated that opening does not block. It had nothing to do with mandatory locking beyond that (quite possibly flawed) interpretation. By your definiton of explicit, no. However, explicit locking is voluntary, just as advisory locking is voluntary, in terms of whether programs participate (or not). This pretty much means that explicit locking degrades to advisory locking, in the presence of (un)intentionally non-participatory programs. That's basically what my objection was. The deadlock prone objection made by others applies more strongly to implicit locking, and is also valid. It can take quite a bit of care to ensure that there is always a maintenance path to the system that allows a sufficient environment to be used without blocking on locked files to allow root to get in and kill any processes causing problems. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 19:53:22 -0400, Christian Kuhtz wrote: On Thu, Aug 26, 1999 at 09:09:33AM +0930, Greg Lehey wrote: On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote: All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? NSK is a prorietary operating system (NonStop Kernel, previously known as Guardian, previously known as TOS), not UNIX. There is no NFS, and there is no distinction between network access and local access: all goes over the message system. When a file is closed, its locks are released. How does this message system handle releasing stale locks when components talking to the message system die unexpectedly? Is there some sort of aging, override or timeout mechanism to purge stale locks? When a process dies unexpectedly, the message system releases all resources which were allocated to it. This includes, of course, closing its files. If the process is on the local processor, the locak processor cleans up. If the local processor dies, the remote systems are informed and do the cleanup for all resources used by processes on that processor. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 18:25:31 +, Terry Lambert wrote: And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. Har! 8-). I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Har again! 8-). I continue to be amazed. That's why I've been keeping out of this discussion. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. UNIX doesn't do this _at all_ well for hosted OS's. The NetWare for UNIX product, which I worked on the attributed FS (with resource forks and multiple namespaces) and the lock coherency model on is one example. We had to add hooks into the fcntl() call in the FS to support this. Something that might be interesting to look at in this regard is the latest SAMBA code. The latest SAMBA code has an OS OPLOCK (Opportunity Locking) API that it will utilize, if it is present (currently only on IRIX), which deals with the lock coherency issues. Note that since SAMBA externalizes via SMB an interface that has to implement NetBIOS calls, and those, in turn, externalize via the DOS INT 2A/2C mechanisms into the file I/O INTs, that SAMBA has to support mandatory locking. How does it do this under FreeBSD? Does it implement it internally? In effect, it is an API which externalizes much of the same types of operations that implement LEASE operations used by the current FreeBSD NFS server implementation. I don't know if this would be quite sufficient (it's been a while since I did a lease and order of operation audit on the VFS code, and written up patches to support Jordan's class project of making the NFS server locking work), but it should be a healthy start down the right road, I think. Might even fix a couple of NFS bugs as a side benefit... Anyway, Sean's also right about needing mandatory file locks for binary emulation of other platforms (some of which Sean coded for). Are file locks sufficient for Vinum, Greg? To repeat myself again: none of this is relevant to Vinum. The original problem I was looking out turned out not to require any other locking method than was already present in Vinum. The current lock structures in FreeBSD are hung off the backing inodes (of which specfs has none available that are not themselves abstract), and the spec_advlock() function returns either EOPNOTSUPP or EINVAL, based on the arguments its given. IMO, unless the lock list is hung off the vnode (I guess you could hang the mandatory locks there, and leave the advisory ones alone, but that's kind of a waste, especially if the locking code can be reused), you aren't going to be able to do range locks on your stripes. Where are advisory locks kept nowadays? I'll discuss this in a separate message. Am I not understanding how Vinum's RAID 5 is implemented? Are you using files for the stripes, or are you laying out the disk as a true RAID 5 controller would lay it out? I'm laying out the disk in the same manner as a hardware RAID-5 controller would do it. Vinum's downward interface is to the disk driver, not the file system. I'll send a separate (hopefully last) message to try and sum up what I'm doing. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Not to jump down your throat, or anything, but you seem to be perpetuating some incorrct assumptions about both effect and proposed implementation details, and they must be stomped. 8-). I was assuming that mandatory locking, in the context of this discussion, does not mean automatic, forced exclusion on open, but rather explicit locks, applied by calls similar to those used for advisory locking, that are enforced by the kernel. The arguments presented by most people seem to rely on such an interpretation. To avoid confusion, I'll refer to the possible locking methods as advisory locking, explicit locking and implicit locking. Advisory locking lacks coherency for a NetWare, SMB, AppleTalk, or other file server running under FreeBSD as a hosted OS. It also has the problem that the hosted OS semantics, if they include mandatory locking, are not enforced against other processes, e.g. between an SMB server and an AppleTalk server running on the same machine, or beteen an SMB server and a UNIX program both needing access to the same database. Yes, if file service protocols don't provide locking (or if one of the operating systems involved doesn't provide locking) they obviously can't benefit from any locking that isn't done implicitly. Also, I believe your example is flawed. If a file is opened by a Not for explicit locking, I hope. process that requires mandatory locking, no process that does not also open the file with mandatory locking turned on can access the file. Neither can a program that requires mandatory locking semantics open the file if it is open by a process not using those same semantics. So if a process wishes to use explicit locking calls, it indicates that intent when opening the file - otherwise, the open implicitly locks the file. So multiple writers, or simultaneous readers and writers are only permitted for programs that indicate that they are going to use explicit locks on the file. This could actually make sense. But I don't think that is what is being suggested. Mandatory locking for things like database files is necessary, unless the underlying FS supports records (in which case, like FILES-11, it most likely supports record locking anyway, and may only decide not to support them if it seperately implements a transaction facility). You have to have mandatory locking to implement transactions... like updating the parity bits on a RAID 5 stripe. But you certainly don't want to use open/read/write/close cycles for such a purpose. This is why so many _real_ UNIX databases like to squat on their own raw disk partition. Locking entire files, in addition to ranges, would seem to me to be of further benefit, as it would allow properly locking programs to fully protect against any single non-locking program which, like Greg's cat example, would presumably be run interactively and thus would require explicit stupidity to create additional races. This is already possible, using O_EXCL. Likewise, it doesn't I think you mean O_EXLOCK. It sets an advisory lock, it does not help against programs that don't use locks. apply to device files, and can not be applied (via fcntl(2)) to any files whose vnodes indirect through other than the vfsops version of "struct fileops". It doesn't depend on the struct fileops selected, fcntl checks explicitly that f_type is DTYPE_VNODE before assuming that f_data points to a vnode. For SVR4 semantics, you can set the suid/sgid permission bits on a non-executable file. The document describing "mandatory" locking in Linux seems to indicate that setting sgid changes the behavior of locking calls to apply explicit locks rather than merely advisory ones, and that this is what is done by other operating systems as well. Actually an implementation could still use the existing (advisory) locks internally, but apply advisory locks in the kernel for the duration of operations that need them (read/write/some cases of open). The act of opening the file for O_RDONLY sets a read lock on the entire file, which allows multiple readers, and the act of opening the file O_RDWR sets a write lock on the file, which allows a single writer. I'm fairly certain this is not what is being discussed. Certainly not by more than half of the participants in the discussion. ;--) Again, there are no issues with badly behaved processes. There is no such thing as a badly behaved process. I agree, for implicit locks there isn't. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
I was assuming that mandatory locking, in the context of this discussion, does not mean automatic, forced exclusion on open, but rather explicit locks, applied by calls similar to those used for advisory locking, that are enforced by the kernel. It's not mandatory, if it's not implicit. You can't enforce locks against processes that don't make lock calls, if it's not implicit. I think this has been the basis of your objection so far. If so, it's a fundamental misunderstanding of "mandatory". In this context it means that no program has the option of not participating in the locking protocol. Also, I believe your example is flawed. If a file is opened by a Not for explicit locking, I hope. By your definiton of explicit, no. However, explicit locking is voluntary, just as advisory locking is voluntary, in terms of whether programs participate (or not). This pretty much means that explicit locking degrades to advisory locking, in the presence of (un)intentionally non-participatory programs. Since we already have advisory locking, this case is not really interesting to consider. So if a process wishes to use explicit locking calls, it indicates that intent when opening the file - otherwise, the open implicitly locks the file. So multiple writers, or simultaneous readers and writers are only permitted for programs that indicate that they are going to use explicit locks on the file. This is a case of "the excluded middle". It presumes, incorrectly, that there will always be at least one program engaged in explicit locking on a file to which explicit locking must be applied (the kernel can only enfoce exclusion of non-participating programs if it know that the file has explicit access semantices). This neglects the case of a file that requires these semantics, but which is not currently in use by a program which employs them. In that particular case, non-participating programs are (incorrectly) permitted access to the file. There are obscure ways around this. You could implement, for example, an access manager, and use file permissions to protect the file. Then you require all programs accessing the file to obtain the fd they will use via a transaction that results in an open, with the explicit locking semantics, which passes the resulting fd to the requesting program via descriptor passing. This is incredibly arcane, and doesn't work over a network (where descriptor passing won't work), but it's one method of turning explicit participation in a locking protocol into something useful. You have to have mandatory locking to implement transactions... like updating the parity bits on a RAID 5 stripe. But you certainly don't want to use open/read/write/close cycles for such a purpose. No. Which is why you would want to implement mandatory participation in a range locking protocol, and prohibit reads and writes. This is somewhat defeated by FreeBSD's unified VM and buffer cache, in that you would have to disallow mmap() of such files, seperate the access semantics into a semantic VFS stacking layer with its own idea of backing store, or hack the VM to make the pages virtually unmapped -- that is, take read and write faults, and enforce the lock semantics at fault time. It's ugly, but it's useful. My gut tells me that software RAID 5 belongs in kernel space, or if in user space, requires a seperate VFS stacking layer to proxy the user space requests into kernel space, at the expense of explicit backing store and underlying object coherency. This is already possible, using O_EXCL. Likewise, it doesn't I think you mean O_EXLOCK. It sets an advisory lock, it does not help against programs that don't use locks. Yes, for 4.4BSD, it's O_EXLOCK. For everything else, it's O_EXCL. apply to device files, and can not be applied (via fcntl(2)) to any files whose vnodes indirect through other than the vfsops version of "struct fileops". It doesn't depend on the struct fileops selected, fcntl checks explicitly that f_type is DTYPE_VNODE before assuming that f_data points to a vnode. The mechanics are irrelevant for the open call, which goes through the VFS to specfs for devices. It would be more relevant if we were talking about lock operations after the open. As shown above, though, these are not useful in the sense that mandatory locks are useful. For SVR4 semantics, you can set the suid/sgid permission bits on a non-executable file. The document describing "mandatory" locking in Linux seems to indicate that setting sgid changes the behavior of locking calls to apply explicit locks rather than merely advisory ones, and that this is what is done by other operating systems as well. Actually an implementation could still use the existing (advisory) locks internally, but apply advisory locks in the kernel for the duration of operations that need them (read/write/some cases of open). This
Re: Mandatory locking?
Not to jump down your throat, or anything, but you seem to be perpetuating some incorrct assumptions about both effect and proposed implementation details, and they must be stomped. 8-). I was assuming that mandatory locking, in the context of this discussion, does not mean automatic, forced exclusion on open, but rather explicit locks, applied by calls similar to those used for advisory locking, that are enforced by the kernel. The arguments presented by most people seem to rely on such an interpretation. To avoid confusion, I'll refer to the possible locking methods as advisory locking, explicit locking and implicit locking. Advisory locking lacks coherency for a NetWare, SMB, AppleTalk, or other file server running under FreeBSD as a hosted OS. It also has the problem that the hosted OS semantics, if they include mandatory locking, are not enforced against other processes, e.g. between an SMB server and an AppleTalk server running on the same machine, or beteen an SMB server and a UNIX program both needing access to the same database. Yes, if file service protocols don't provide locking (or if one of the operating systems involved doesn't provide locking) they obviously can't benefit from any locking that isn't done implicitly. Also, I believe your example is flawed. If a file is opened by a Not for explicit locking, I hope. process that requires mandatory locking, no process that does not also open the file with mandatory locking turned on can access the file. Neither can a program that requires mandatory locking semantics open the file if it is open by a process not using those same semantics. So if a process wishes to use explicit locking calls, it indicates that intent when opening the file - otherwise, the open implicitly locks the file. So multiple writers, or simultaneous readers and writers are only permitted for programs that indicate that they are going to use explicit locks on the file. This could actually make sense. But I don't think that is what is being suggested. Mandatory locking for things like database files is necessary, unless the underlying FS supports records (in which case, like FILES-11, it most likely supports record locking anyway, and may only decide not to support them if it seperately implements a transaction facility). You have to have mandatory locking to implement transactions... like updating the parity bits on a RAID 5 stripe. But you certainly don't want to use open/read/write/close cycles for such a purpose. This is why so many _real_ UNIX databases like to squat on their own raw disk partition. Locking entire files, in addition to ranges, would seem to me to be of further benefit, as it would allow properly locking programs to fully protect against any single non-locking program which, like Greg's cat example, would presumably be run interactively and thus would require explicit stupidity to create additional races. This is already possible, using O_EXCL. Likewise, it doesn't I think you mean O_EXLOCK. It sets an advisory lock, it does not help against programs that don't use locks. apply to device files, and can not be applied (via fcntl(2)) to any files whose vnodes indirect through other than the vfsops version of struct fileops. It doesn't depend on the struct fileops selected, fcntl checks explicitly that f_type is DTYPE_VNODE before assuming that f_data points to a vnode. For SVR4 semantics, you can set the suid/sgid permission bits on a non-executable file. The document describing mandatory locking in Linux seems to indicate that setting sgid changes the behavior of locking calls to apply explicit locks rather than merely advisory ones, and that this is what is done by other operating systems as well. Actually an implementation could still use the existing (advisory) locks internally, but apply advisory locks in the kernel for the duration of operations that need them (read/write/some cases of open). The act of opening the file for O_RDONLY sets a read lock on the entire file, which allows multiple readers, and the act of opening the file O_RDWR sets a write lock on the file, which allows a single writer. I'm fairly certain this is not what is being discussed. Certainly not by more than half of the participants in the discussion. ;--) Again, there are no issues with badly behaved processes. There is no such thing as a badly behaved process. I agree, for implicit locks there isn't. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
I was assuming that mandatory locking, in the context of this discussion, does not mean automatic, forced exclusion on open, but rather explicit locks, applied by calls similar to those used for advisory locking, that are enforced by the kernel. It's not mandatory, if it's not implicit. You can't enforce locks against processes that don't make lock calls, if it's not implicit. I think this has been the basis of your objection so far. If so, it's a fundamental misunderstanding of mandatory. In this context it means that no program has the option of not participating in the locking protocol. Also, I believe your example is flawed. If a file is opened by a Not for explicit locking, I hope. By your definiton of explicit, no. However, explicit locking is voluntary, just as advisory locking is voluntary, in terms of whether programs participate (or not). This pretty much means that explicit locking degrades to advisory locking, in the presence of (un)intentionally non-participatory programs. Since we already have advisory locking, this case is not really interesting to consider. So if a process wishes to use explicit locking calls, it indicates that intent when opening the file - otherwise, the open implicitly locks the file. So multiple writers, or simultaneous readers and writers are only permitted for programs that indicate that they are going to use explicit locks on the file. This is a case of the excluded middle. It presumes, incorrectly, that there will always be at least one program engaged in explicit locking on a file to which explicit locking must be applied (the kernel can only enfoce exclusion of non-participating programs if it know that the file has explicit access semantices). This neglects the case of a file that requires these semantics, but which is not currently in use by a program which employs them. In that particular case, non-participating programs are (incorrectly) permitted access to the file. There are obscure ways around this. You could implement, for example, an access manager, and use file permissions to protect the file. Then you require all programs accessing the file to obtain the fd they will use via a transaction that results in an open, with the explicit locking semantics, which passes the resulting fd to the requesting program via descriptor passing. This is incredibly arcane, and doesn't work over a network (where descriptor passing won't work), but it's one method of turning explicit participation in a locking protocol into something useful. You have to have mandatory locking to implement transactions... like updating the parity bits on a RAID 5 stripe. But you certainly don't want to use open/read/write/close cycles for such a purpose. No. Which is why you would want to implement mandatory participation in a range locking protocol, and prohibit reads and writes. This is somewhat defeated by FreeBSD's unified VM and buffer cache, in that you would have to disallow mmap() of such files, seperate the access semantics into a semantic VFS stacking layer with its own idea of backing store, or hack the VM to make the pages virtually unmapped -- that is, take read and write faults, and enforce the lock semantics at fault time. It's ugly, but it's useful. My gut tells me that software RAID 5 belongs in kernel space, or if in user space, requires a seperate VFS stacking layer to proxy the user space requests into kernel space, at the expense of explicit backing store and underlying object coherency. This is already possible, using O_EXCL. Likewise, it doesn't I think you mean O_EXLOCK. It sets an advisory lock, it does not help against programs that don't use locks. Yes, for 4.4BSD, it's O_EXLOCK. For everything else, it's O_EXCL. apply to device files, and can not be applied (via fcntl(2)) to any files whose vnodes indirect through other than the vfsops version of struct fileops. It doesn't depend on the struct fileops selected, fcntl checks explicitly that f_type is DTYPE_VNODE before assuming that f_data points to a vnode. The mechanics are irrelevant for the open call, which goes through the VFS to specfs for devices. It would be more relevant if we were talking about lock operations after the open. As shown above, though, these are not useful in the sense that mandatory locks are useful. For SVR4 semantics, you can set the suid/sgid permission bits on a non-executable file. The document describing mandatory locking in Linux seems to indicate that setting sgid changes the behavior of locking calls to apply explicit locks rather than merely advisory ones, and that this is what is done by other operating systems as well. Actually an implementation could still use the existing (advisory) locks internally, but apply advisory locks in the kernel for the duration of operations that need them (read/write/some cases of open). This is really insufficient. The unified VM and buffer cache
RE: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file "xyz" open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? Just curious... - Dave Rivers - To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
[EMAIL PROTECTED] (Wes Peters) writes: And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. It should only take one, as long as the arguments made are not bogus. IMHO Greg made some very silly arguments (or at least used some very stupid examples) for mandatory locking and never answered my points regarding them. (The arguments of some of the ones opposing mandatory locking have been equally silly.) I *do* agree that mandatory locking *can* be useful, but the usefulness is not nearly as broad as some people seem to be implying, and advisory locking is not as useless as some claim. The most significant advantage I see with mandatory locking over advisory locking is guaranteeing atomicity for things done by programs that do use locking. This only protects the data when programs that access the same data without locking don't need locking, which generally means that they either don't need to modify the data or that there can't be multiple instances of those other programs *and* the modifications made are themselves atomic (can't be read-modify-write, or even multiple writes if consistency is required). This is a somewhat limited set of cases. If anyone can come up with a counter-example, please present it. Locking entire files, in addition to ranges, would seem to me to be of further benefit, as it would allow properly locking programs to fully protect against any single non-locking program which, like Greg's cat example, would presumably be run interactively and thus would require explicit stupidity to create additional races. Locking entire files is also the only way to ensure that non-locking programs can even see the file in a consistent state. As a special case, mandatory locking could also be useful in ensuring long-term exclusive access to some set of data, but this seems like something that should be done using file permissions. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
[Cc's trimmed] On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: How 'bout "anyone who can kill the process holding the lock?" + file owner ( + root ). Which processes can't root kill? Zombies? :) Otherwise I would be able to lock ~wes/FreeBSDmarkers and you wouldn't be able to do anything about it until either notifying me or notifying root about the process I accidentally left hanging. Hey, I'm the one who gave you write access to it. If I didn't want you write-locking it, I wouldn't give you write access, now wood eye? Yes, but I wrote a program that knows when I move between here and Toronto. That program automatically updates ~wes/FreeBSDmarkers. Unfortunately, I left a small bug in that program (it doesn't unlock and it doesn't end itself). To make matters worse, since this was the iteration where I move to Toronto, I probably won't be reading mail for a couple days (or more). You'll have to try and contact root (or just crash the machine). Fortunately for you, [EMAIL PROTECTED] is fairly responsive... :) But in the meantime, everyone else, including you, is locked out of that file..which is pretty bad since my buggy program had another bug I forgot to mention: it accidentally removed all entries from the file except mine. I hope you don't use that file for anything important. Gosh, and I thought I was being smart by using mandatory locks so that your file would get badly damaged it someone else tried modifying it while my program also modified it. Or better yet, implement file ACLs so I can grant read/write to everyone BUT you. Of course, I can do that now by creating goup "nothoek" right? ;^) Well, that would work, too. :-) -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wed, 25 Aug 1999, Tim Vanderhoek wrote: :[Cc's trimmed] : :On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: : : How 'bout "anyone who can kill the process holding the lock?" : : + file owner ( + root ). : : Which processes can't root kill? : :Zombies? :) ps shows the parent. Kill the parent, and the zombified child dies. This ia a non issue. Jamie Bowden -- If we've got to fight over grep, sign me up. But boggle can go. -Ted Faber (on Hasbro's request for removal of /usr/games/boggle) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Greg Lehey wrote: On Wednesday, 25 August 1999 at 0:11:23 -0600, Wes Peters wrote: "Daniel C. Sobral" wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. Well, now, that message of mine quoted above was intended to lighten up the mood. I'spose so was Wes' message. -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. Har! 8-). I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Har again! 8-). Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. UNIX doesn't do this _at all_ well for "hosted OS's". The NetWare for UNIX product, which I worked on the attributed FS (with resource forks and multiple namespaces) and the lock coherency model on is one example. We had to add hooks into the fcntl() call in the FS to support this. Something that might be interesting to look at in this regard is the latest SAMBA code. The latest SAMBA code has an OS OPLOCK (Opportunity Locking) API that it will utilize, if it is present (currently only on IRIX), which deals with the lock coherency issues. Note that since SAMBA externalizes via SMB an interface that has to implement NetBIOS calls, and those, in turn, externalize via the DOS INT 2A/2C mechanisms into the file I/O INTs, that SAMBA has to support mandatory locking. In effect, it is an API which externalizes much of the same types of operations that implement LEASE operations used by the current FreeBSD NFS server implementation. I don't know if this would be quite sufficient (it's been a while since I did a lease and order of operation audit on the VFS code, and written up patches to support Jordan's class project of making the NFS server locking work), but it should be a healthy start down the right road, I think. Might even fix a couple of NFS bugs as a side benefit... Anyway, Sean's also right about needing mandatory file locks for binary emulation of other platforms (some of which Sean coded for). Are file locks sufficient for Vinum, Greg? The current lock structures in FreeBSD are hung off the backing inodes (of which specfs has none available that are not themselves abstract), and the spec_advlock() function returns either EOPNOTSUPP or EINVAL, based on the arguments its given. IMO, unless the lock list is hung off the vnode (I guess you could hang the mandatory locks there, and leave the advisory ones alone, but that's kind of a waste, especially if the locking code can be reused), you aren't going to be able to do range locks on your stripes. Am I not understanding how Vinum's RAID 5 is implemented? Are you using files for the stripes, or are you laying out the disk as a true RAID 5 controller would lay it out? I guess this all moderately re-begs the "struct fileops" question, yet again, too... 8-(. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Not to jump down your throat, or anything, but you seem to be
perpetuating some incorrct assumptions about both effect and
proposed implementation details, and they must be stomped. 8-).
And how many programmers with nearly (or more than) two decades of UNIX
experience it takes to convince someone it really is useful.
It should only take one, as long as the arguments made are not bogus.
IMHO Greg made some very silly arguments (or at least used some very
stupid examples) for mandatory locking and never answered my points
regarding them. (The arguments of some of the ones opposing mandatory
locking have been equally silly.)
I must have missed this...
I *do* agree that mandatory locking *can* be useful, but the
usefulness is not nearly as broad as some people seem to be implying,
and advisory locking is not as useless as some claim.
Advisory locking lacks coherency for a NetWare, SMB, AppleTalk, or
other file server running under FreeBSD as a hosted OS.
It also has the problem that the hosted OS semantics, if they
include mandatory locking, are not enforced against other
processes, e.g. between an SMB server and an AppleTalk server
running on the same machine, or beteen an SMB server and a UNIX
program both needing access to the same database.
The most significant advantage I see with mandatory locking over
advisory locking is guaranteeing atomicity for things done by programs
that do use locking. This only protects the data when programs that
access the same data without locking don't need locking, which
generally means that they either don't need to modify the data or that
there can't be multiple instances of those other programs *and* the
modifications made are themselves atomic (can't be read-modify-write,
or even multiple writes if consistency is required).
This is a somewhat limited set of cases. If anyone can come up with a
counter-example, please present it.
See above.
Also, I believe your example is flawed. If a file is opened by a
process that requires mandatory locking, no process that does not
also open the file with mandatory locking turned on can access the
file. Neither can a program that requires mandatory locking
semantics open the file if it is open by a process not using those
same semantics.
This means that your example applications that "access the same data
without locking" can't exist.
Mandatory locking for things like database files is necessary,
unless the underlying FS supports records (in which case, like
FILES-11, it most likely supports record locking anyway, and
may only decide not to support them if it seperately implements
a transaction facility).
You have to have mandatory locking to implement transactions...
like updating the parity bits on a RAID 5 stripe.
This is why so many _real_ UNIX databases like to squat on their
own raw disk partition.
Locking entire files, in addition to ranges, would seem to me to be of
further benefit, as it would allow properly locking programs to fully
protect against any single non-locking program which, like Greg's cat
example, would presumably be run interactively and thus would require
explicit stupidity to create additional races.
This is already possible, using O_EXCL. Likewise, it doesn't
apply to device files, and can not be applied (via fcntl(2)) to
any files whose vnodes indirect through other than the vfsops
version of "struct fileops".
Greg's "cat example", I believe, was intended to illustrate the
example of something which opens and closes the file between
operations. These operations are, themselves, atomic, so in
effect, the mandatory-lock-using program is dealing with a
"snapshot" of the file, as far as it is concerned, with there
being absolutely no danger of a process not obeying the access
protocol running amok during the operation, merely because a
range lock was not asserted.
Locking entire files is also the only way to ensure that non-locking
programs can even see the file in a consistent state.
For SVR4 semantics, you can set the suid/sgid permission bits on a
non-executable file.
The act of opening the file for O_RDONLY sets a read lock on the
entire file, which allows multiple readers, and the act of opening
the file O_RDWR sets a write lock on the file, which allows a single
writer.
Again, there are no issues with badly behaved processes. There
is no such thing as a badly behaved process.
For mandatory range locks, you would have to implement a seperate
open mode ("open for mandatory range locking"), and enforce against
writing or reading, which requires modifying the system calls, any
range which was not locked for writing or reading by the process
making the request.
Obviously, one mode could be the setgid bit being set on the file,
and the other the setuid bit, and/or you could define additional
open(2) flags and/or new chflags(1) arguments. The reason SVR4
uses the suid/sgid bits is that
Re: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file "xyz" open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? The remote lockd tells your local lockd about it. If the remote system dies entirely, your statd tells you about it. Mandatory locking via SGID/SUID bits works over NFS, so long as locking works over NFS, and so long as both the client and the server implement the same feature signalling mechanism. Yet another reason to not go the chflags/open parameter route, since neither can be communicated well over NFS. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
[Cc's trimmed] On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: How 'bout "anyone who can kill the process holding the lock?" + file owner ( + root ). Which processes can't root kill? Zombies? :) Kill their parents. Eventually, you will get init to reap the process status. Not that this matters, of course, since a zombie has already engaged in resource tracking cleanup via _exit(). 8-p. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
The thing about well-intentioned but incorrect locking code is that it will appear to work fine, until it trips over the one code path where it forgets to lock some file that it should have locked. And even then, the code will "work" just fine, until multiple processes are accessing that file at the same time. Which is why you perform unit testing, itegration testing, and full branch path validation, if you are a professional software engineer engaging in due dilligence. Unfortunately, most professional software engineers are only permitted by their management to engage in due dilligence when the softwar being worked on is part of a life-support system of some kind (e.g. the software that runs on the microcontroller in your cars anti-lock braking system). Most software malfunctions are the result of either software engineer unprofessionalism or management unprofessionalism, not because "software is magic" and not because "software is this amorphorous thing" and not because "the software is too complex for one mind to grasp all of it". Anyone who tells you any different is either lying or a hardware engineer. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote:
All the files under Tandem's NSK has mandatory locking. The file cannot be
opened if another process has it opened. some thing like
* if the file is opened for reading, any one can open it for
reading but opening for writing gives error
* if the file is open for writing, it can't be opened for
read/write
* if the process holding the file is killed, the lock is gone
* it is possible to get the pid of the process(es) which has
a given file open (like which process has file "xyz" open?
kind of query). btw, is there any way to get this info now in FBSD?
This sounds interesting...
But - aren't there NFS issues? I mean, in stateless access to
a file - how do you know if the process holding the file is killed
if it's remote?
NSK is a prorietary operating system ("NonStop Kernel", previously
known as Guardian, previously known as TOS), not UNIX. There is no
NFS, and there is no distinction between network access and local
access: all goes over the message system. When a file is closed, its
locks are released.
Greg
--
See complete headers for address, home page and phone numbers
finger [EMAIL PROTECTED] for PGP public key
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Thu, Aug 26, 1999 at 09:09:33AM +0930, Greg Lehey wrote:
On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote:
All the files under Tandem's NSK has mandatory locking. The file cannot be
opened if another process has it opened. some thing like
* if the file is opened for reading, any one can open it for
reading but opening for writing gives error
* if the file is open for writing, it can't be opened for
read/write
* if the process holding the file is killed, the lock is gone
* it is possible to get the pid of the process(es) which has
a given file open (like which process has file "xyz" open?
kind of query). btw, is there any way to get this info now in FBSD?
This sounds interesting...
But - aren't there NFS issues? I mean, in stateless access to
a file - how do you know if the process holding the file is killed
if it's remote?
NSK is a prorietary operating system ("NonStop Kernel", previously
known as Guardian, previously known as TOS), not UNIX. There is no
NFS, and there is no distinction between network access and local
access: all goes over the message system. When a file is closed, its
locks are released.
How does this message system handle releasing stale locks when components
talking to the message system die unexpectedly? Is there some sort of aging,
override or timeout mechanism to purge stale locks?
--
Christian Kuhtz, Sr. Network ArchitectBellSouth Corporation
[EMAIL PROTECTED] -wk, [EMAIL PROTECTED] -hmAdvanced Data Services
"Affiliation given for identification, not representation." Atlanta, GA, U.S.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
Sean Eric Fagan wrote: The fact that Greg thinks it's necessary and desirable (and he has considerably more OS experience than a lot of the people who have decided it's a stupid idea) should alone say a lot for the idea. I was waiting for someone else to bring up that point, because I might be accused of partiality. The fact that you and Terry both seem to agree with Greg makes it a very rare triumvirate. So, who's going to do it? ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Daniel C. Sobral wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Tim Vanderhoek wrote: On Tue, Aug 24, 1999 at 08:25:59AM -0600, Wes Peters wrote: I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... How 'bout anyone who can kill the process holding the lock? + file owner ( + root ). Which processes can't root kill? Otherwise I would be able to lock ~wes/FreeBSDmarkers and you wouldn't be able to do anything about it until either notifying me or notifying root about the process I accidentally left hanging. Hey, I'm the one who gave you write access to it. If I didn't want you write-locking it, I wouldn't give you write access, now wood eye? [Not that I'm likely to ever need more than an advisory lock on that particular file, but the principle... :-] Hm. ``chmod go-MayLock ~wes/Fre*'' The sticky bit could be used. Or better yet, implement file ACLs so I can grant read/write to everyone BUT you. Of course, I can do that now by creating goup nothoek right? ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
On Tuesday, 24 August 1999 at 22:28:10 -0700, Sean Eric Fagan wrote: In article 19990825113518.d83273.kithrup.freebsd.cvs-...@freebie.lemis.com you write: Correct. I lock a stripe at a time. What people need to realize is that Greg is doing this locking in user mode. As such, he has two real options: 1. Implement a vinum-specific ioctl that locks a region of a file at the device level, or 2. Implement standard mandatory region locking, which damned near every OS in existence has, and which OSes which have existed for decades longer than unix has existed have always had. Now, because this _is_ (currently) a vinum-specific requirement, doing (1) is not all that unpalatable. However, it's at the wrong level (device driver, instead of file), and it is merely putting off the inevitable. In fact, this confusion with Vinum is more historical than anything. I started thinking what tools are available for this as yet not clearly defined task that will run in user mode and require locking?. The obvious first question was do we support read (i.e. mandatory) locking?. Or does nobody wish to have working Linux and Solaris/x86 compatibility after all? That reason, if nothing else, is reason to design it, so it gets done correctly. Exactly, and that was why I started this thread. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 0:11:23 -0600, Wes Peters wrote: Daniel C. Sobral wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
RE: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? Just curious... - Dave Rivers - To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
w...@softweyr.com (Wes Peters) writes: And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. It should only take one, as long as the arguments made are not bogus. IMHO Greg made some very silly arguments (or at least used some very stupid examples) for mandatory locking and never answered my points regarding them. (The arguments of some of the ones opposing mandatory locking have been equally silly.) I *do* agree that mandatory locking *can* be useful, but the usefulness is not nearly as broad as some people seem to be implying, and advisory locking is not as useless as some claim. The most significant advantage I see with mandatory locking over advisory locking is guaranteeing atomicity for things done by programs that do use locking. This only protects the data when programs that access the same data without locking don't need locking, which generally means that they either don't need to modify the data or that there can't be multiple instances of those other programs *and* the modifications made are themselves atomic (can't be read-modify-write, or even multiple writes if consistency is required). This is a somewhat limited set of cases. If anyone can come up with a counter-example, please present it. Locking entire files, in addition to ranges, would seem to me to be of further benefit, as it would allow properly locking programs to fully protect against any single non-locking program which, like Greg's cat example, would presumably be run interactively and thus would require explicit stupidity to create additional races. Locking entire files is also the only way to ensure that non-locking programs can even see the file in a consistent state. As a special case, mandatory locking could also be useful in ensuring long-term exclusive access to some set of data, but this seems like something that should be done using file permissions. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
[Cc's trimmed] On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: How 'bout anyone who can kill the process holding the lock? + file owner ( + root ). Which processes can't root kill? Zombies? :) Otherwise I would be able to lock ~wes/FreeBSDmarkers and you wouldn't be able to do anything about it until either notifying me or notifying root about the process I accidentally left hanging. Hey, I'm the one who gave you write access to it. If I didn't want you write-locking it, I wouldn't give you write access, now wood eye? Yes, but I wrote a program that knows when I move between here and Toronto. That program automatically updates ~wes/FreeBSDmarkers. Unfortunately, I left a small bug in that program (it doesn't unlock and it doesn't end itself). To make matters worse, since this was the iteration where I move to Toronto, I probably won't be reading mail for a couple days (or more). You'll have to try and contact root (or just crash the machine). Fortunately for you, r...@freebsd.org is fairly responsive... :) But in the meantime, everyone else, including you, is locked out of that file..which is pretty bad since my buggy program had another bug I forgot to mention: it accidentally removed all entries from the file except mine. I hope you don't use that file for anything important. Gosh, and I thought I was being smart by using mandatory locks so that your file would get badly damaged it someone else tried modifying it while my program also modified it. Or better yet, implement file ACLs so I can grant read/write to everyone BUT you. Of course, I can do that now by creating goup nothoek right? ;^) Well, that would work, too. :-) -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wed, 25 Aug 1999, Tim Vanderhoek wrote: :[Cc's trimmed] : :On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: : : How 'bout anyone who can kill the process holding the lock? : : + file owner ( + root ). : : Which processes can't root kill? : :Zombies? :) ps shows the parent. Kill the parent, and the zombified child dies. This ia a non issue. Jamie Bowden -- If we've got to fight over grep, sign me up. But boggle can go. -Ted Faber (on Hasbro's request for removal of /usr/games/boggle) To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Greg Lehey wrote: On Wednesday, 25 August 1999 at 0:11:23 -0600, Wes Peters wrote: Daniel C. Sobral wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. Well, now, that message of mine quoted above was intended to lighten up the mood. I'spose so was Wes' message. -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. Har! 8-). I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Har again! 8-). Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. UNIX doesn't do this _at all_ well for hosted OS's. The NetWare for UNIX product, which I worked on the attributed FS (with resource forks and multiple namespaces) and the lock coherency model on is one example. We had to add hooks into the fcntl() call in the FS to support this. Something that might be interesting to look at in this regard is the latest SAMBA code. The latest SAMBA code has an OS OPLOCK (Opportunity Locking) API that it will utilize, if it is present (currently only on IRIX), which deals with the lock coherency issues. Note that since SAMBA externalizes via SMB an interface that has to implement NetBIOS calls, and those, in turn, externalize via the DOS INT 2A/2C mechanisms into the file I/O INTs, that SAMBA has to support mandatory locking. In effect, it is an API which externalizes much of the same types of operations that implement LEASE operations used by the current FreeBSD NFS server implementation. I don't know if this would be quite sufficient (it's been a while since I did a lease and order of operation audit on the VFS code, and written up patches to support Jordan's class project of making the NFS server locking work), but it should be a healthy start down the right road, I think. Might even fix a couple of NFS bugs as a side benefit... Anyway, Sean's also right about needing mandatory file locks for binary emulation of other platforms (some of which Sean coded for). Are file locks sufficient for Vinum, Greg? The current lock structures in FreeBSD are hung off the backing inodes (of which specfs has none available that are not themselves abstract), and the spec_advlock() function returns either EOPNOTSUPP or EINVAL, based on the arguments its given. IMO, unless the lock list is hung off the vnode (I guess you could hang the mandatory locks there, and leave the advisory ones alone, but that's kind of a waste, especially if the locking code can be reused), you aren't going to be able to do range locks on your stripes. Am I not understanding how Vinum's RAID 5 is implemented? Are you using files for the stripes, or are you laying out the disk as a true RAID 5 controller would lay it out? I guess this all moderately re-begs the struct fileops question, yet again, too... 8-(. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Not to jump down your throat, or anything, but you seem to be perpetuating some incorrct assumptions about both effect and proposed implementation details, and they must be stomped. 8-). And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. It should only take one, as long as the arguments made are not bogus. IMHO Greg made some very silly arguments (or at least used some very stupid examples) for mandatory locking and never answered my points regarding them. (The arguments of some of the ones opposing mandatory locking have been equally silly.) I must have missed this... I *do* agree that mandatory locking *can* be useful, but the usefulness is not nearly as broad as some people seem to be implying, and advisory locking is not as useless as some claim. Advisory locking lacks coherency for a NetWare, SMB, AppleTalk, or other file server running under FreeBSD as a hosted OS. It also has the problem that the hosted OS semantics, if they include mandatory locking, are not enforced against other processes, e.g. between an SMB server and an AppleTalk server running on the same machine, or beteen an SMB server and a UNIX program both needing access to the same database. The most significant advantage I see with mandatory locking over advisory locking is guaranteeing atomicity for things done by programs that do use locking. This only protects the data when programs that access the same data without locking don't need locking, which generally means that they either don't need to modify the data or that there can't be multiple instances of those other programs *and* the modifications made are themselves atomic (can't be read-modify-write, or even multiple writes if consistency is required). This is a somewhat limited set of cases. If anyone can come up with a counter-example, please present it. See above. Also, I believe your example is flawed. If a file is opened by a process that requires mandatory locking, no process that does not also open the file with mandatory locking turned on can access the file. Neither can a program that requires mandatory locking semantics open the file if it is open by a process not using those same semantics. This means that your example applications that access the same data without locking can't exist. Mandatory locking for things like database files is necessary, unless the underlying FS supports records (in which case, like FILES-11, it most likely supports record locking anyway, and may only decide not to support them if it seperately implements a transaction facility). You have to have mandatory locking to implement transactions... like updating the parity bits on a RAID 5 stripe. This is why so many _real_ UNIX databases like to squat on their own raw disk partition. Locking entire files, in addition to ranges, would seem to me to be of further benefit, as it would allow properly locking programs to fully protect against any single non-locking program which, like Greg's cat example, would presumably be run interactively and thus would require explicit stupidity to create additional races. This is already possible, using O_EXCL. Likewise, it doesn't apply to device files, and can not be applied (via fcntl(2)) to any files whose vnodes indirect through other than the vfsops version of struct fileops. Greg's cat example, I believe, was intended to illustrate the example of something which opens and closes the file between operations. These operations are, themselves, atomic, so in effect, the mandatory-lock-using program is dealing with a snapshot of the file, as far as it is concerned, with there being absolutely no danger of a process not obeying the access protocol running amok during the operation, merely because a range lock was not asserted. Locking entire files is also the only way to ensure that non-locking programs can even see the file in a consistent state. For SVR4 semantics, you can set the suid/sgid permission bits on a non-executable file. The act of opening the file for O_RDONLY sets a read lock on the entire file, which allows multiple readers, and the act of opening the file O_RDWR sets a write lock on the file, which allows a single writer. Again, there are no issues with badly behaved processes. There is no such thing as a badly behaved process. For mandatory range locks, you would have to implement a seperate open mode (open for mandatory range locking), and enforce against writing or reading, which requires modifying the system calls, any range which was not locked for writing or reading by the process making the request. Obviously, one mode could be the setgid bit being set on the file, and the other the setuid bit, and/or you could define additional open(2) flags and/or new chflags(1) arguments. The reason SVR4 uses the suid/sgid bits is that SVR4 has no chflags(1), an underlying FS type may not support
Re: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? The remote lockd tells your local lockd about it. If the remote system dies entirely, your statd tells you about it. Mandatory locking via SGID/SUID bits works over NFS, so long as locking works over NFS, and so long as both the client and the server implement the same feature signalling mechanism. Yet another reason to not go the chflags/open parameter route, since neither can be communicated well over NFS. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
[Cc's trimmed] On Wed, Aug 25, 1999 at 12:15:24AM -0600, Wes Peters wrote: How 'bout anyone who can kill the process holding the lock? + file owner ( + root ). Which processes can't root kill? Zombies? :) Kill their parents. Eventually, you will get init to reap the process status. Not that this matters, of course, since a zombie has already engaged in resource tracking cleanup via _exit(). 8-p. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
The thing about well-intentioned but incorrect locking code is that it will appear to work fine, until it trips over the one code path where it forgets to lock some file that it should have locked. And even then, the code will work just fine, until multiple processes are accessing that file at the same time. Which is why you perform unit testing, itegration testing, and full branch path validation, if you are a professional software engineer engaging in due dilligence. Unfortunately, most professional software engineers are only permitted by their management to engage in due dilligence when the softwar being worked on is part of a life-support system of some kind (e.g. the software that runs on the microcontroller in your cars anti-lock braking system). Most software malfunctions are the result of either software engineer unprofessionalism or management unprofessionalism, not because software is magic and not because software is this amorphorous thing and not because the software is too complex for one mind to grasp all of it. Anyone who tells you any different is either lying or a hardware engineer. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote: All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? NSK is a prorietary operating system (NonStop Kernel, previously known as Guardian, previously known as TOS), not UNIX. There is no NFS, and there is no distinction between network access and local access: all goes over the message system. When a file is closed, its locks are released. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Thu, Aug 26, 1999 at 09:09:33AM +0930, Greg Lehey wrote: On Wednesday, 25 August 1999 at 6:05:11 -0400, Thomas David Rivers wrote: All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? This sounds interesting... But - aren't there NFS issues? I mean, in stateless access to a file - how do you know if the process holding the file is killed if it's remote? NSK is a prorietary operating system (NonStop Kernel, previously known as Guardian, previously known as TOS), not UNIX. There is no NFS, and there is no distinction between network access and local access: all goes over the message system. When a file is closed, its locks are released. How does this message system handle releasing stale locks when components talking to the message system die unexpectedly? Is there some sort of aging, override or timeout mechanism to purge stale locks? -- Christian Kuhtz, Sr. Network ArchitectBellSouth Corporation c...@adsu.bellsouth.com -wk, c...@gnu.org -hmAdvanced Data Services Affiliation given for identification, not representation. Atlanta, GA, U.S. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
[EMAIL PROTECTED] (Chuck Robey) writes: On 23 Aug 1999, Ville-Pertti Keinonen wrote: And even without otherwise incorrect behavior, if you have a program that doesn't use any locking and another one that uses mandatory locking to prevent races with the non-locking program, the mere existence of the locking program does not prevent multiple non-locking programs from generating similar conditions. That's very odd, I thought the idea behind mandatory locking was to completely eliminate the possibility that a program could do what you're saying; all programs would *mandatorily* be forced to do locking to access the resource. I don't know what the textbook definition for mandatory locking is, but was assuming (particularly considering the proposal to use a fcntl interface) that by mandatory locking, Greg was referring to a "harder" lock than current advisory locking, one that had to be instantiated explicitly but would not only lock out other attempts to lock, but all other attempts to access the file. The further messages in this the thread seems to indicate that different individuals have different definitions for mandatory locking... I'd still assume that marking a file to be accessible by only one process at a time is *not* what is being discussed. Particularly since it is not even clear what this would mean for forked processes, dup, sending file descriptors over local sockets etc. Note that my arguments earlier don't apply in a case where you might want to e.g. ensure consistency for non-locking programs with read-only access, with the only program with privileges to modify the data making the data inaccessible during updates. This is a scenario where it would, IMHO, actually be quite useful to have mandatory locking. In any case, if shared (open) access is allowed, such a feature can introduce semantic changes to read/write system calls - normally, read/write can never return EAGAIN or block for unlimited amounts of time on regular, local files. EAGAIN is not that much of a problem, as it requires explicitly setting O_NONBLOCK, but blocking can introduce new deadlocks. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Ville-Pertti Keinonen scribbled this message on Aug 24:
[EMAIL PROTECTED] (Greg Lehey) writes:
an agreement of some kind. But what if I want to merge the contents
of another mail folder:
cat oldmail /var/mail/grog
That works, but it's playing with fire: if sendmail is delivering a
message at the same time, it won't see me, and my cat doesn't get a
lock beforehand, so both an incoming message and part of my mail
folder could end up getting written to the same location. With
mandatory locking, it would work, transparently.
Certainly not with range-locking rather than file-locking. cat is
certainly not guaranteed to be atomic, and while you shouldn't end up
writing things in the same location, what might happen unless you are
preventing multiple openers is:
cat writes part of oldmail to /var/mail/grog
sendmail locks /var/mail/grog
(cat may try to write more to /var/mail/grog but blocks)
sendmail delivers new mail
sendmail unlocks /var/mail/grog
cat writes the rest of oldmail to /var/mail/grog
You'll still probably end up with a broken mailbox.
what you do is this:
lockf -k $mailfile cat ${mailtmp} $mailfile
then you don't have to worry.. that's what lockf is for...
--
John-Mark Gurney Voice: +1 541 684 8449
Cu Networking P.O. Box 5693, 97405
"The soul contains in itself the event that shall presently befall it.
The event is only the actualizing of its thought." -- Ralph Waldo Emerson
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
[EMAIL PROTECTED] (John-Mark Gurney) writes:
Ville-Pertti Keinonen scribbled this message on Aug 24:
cat writes part of oldmail to /var/mail/grog
sendmail locks /var/mail/grog
(cat may try to write more to /var/mail/grog but blocks)
sendmail delivers new mail
sendmail unlocks /var/mail/grog
cat writes the rest of oldmail to /var/mail/grog
You'll still probably end up with a broken mailbox.
what you do is this:
lockf -k $mailfile cat ${mailtmp} $mailfile
Which doesn't support Greg's arguments for mandatory locking, as
you're now doing locking in both programs.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Tim Vanderhoek wrote: On Mon, Aug 23, 1999 at 10:12:38PM +0200, Mark Murray wrote: In process-space, this is the kernel. In file-space, this should be root. Processes that require mandatory locking must revoke superuser before attempting locks. I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... How 'bout "anyone who can kill the process holding the lock?" -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 08:35:25AM -0600, Wes Peters wrote: You've got this so wrong, perhaps you should just go find a System V man page and read about mandatory locking before embarassing yourself any- more. First of all, when was it decided that we were all talking about System V? I think that's part of the problem here; some people are arguing whether locking should exist while others are arguing what the semantics should be. I have seen (at least) four ideas mentioned: SysV Root doesn't obey locking Root can override locking Only root can lock So don't insult me for giving a "what if?" example. If it's not a problem under certain semantics, that's good. Let's see if that's the case. Locking will only block if another process is holding an overlapping lock. opening won't block due to mandatory locking. The only operations that can block are read and write, and then only if the read or write operation will overlap a locked range of bytes. Ok, so that means the program doesn't have to be so poorly written. It can read part of the file, then go to write it just after User 1 locks that part of the file, so its write will block until User 1 is finished, but will still stomp over User 1's data. No need to close and reopen the file. Yes, it's WRONG code. Correct code would aquire a lock before reading. So how does mandatory locking help? In this situation, I don't think it does. And this situation is analogous to "I want to modify this file without taking it offline, but I need to guarantee that no other processes are using it." How do you know someone hasn't, say, opened it in an editor, made some changes, and is about to save? Especially when that modification occurs spontaneously, perhaps as part of some recovery process unknown to the user editing the file. I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a "useless" feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can "only" cause a denial of service.. or maybe I can make a new race condition in a periodic script. By the way, I like the idea of mandatory locking, and I "grew up" on an OS that had it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Garance A Drosihn wrote: There's a difference between a program which has a locking-oversight or race-condition, and a program you write to deliberately destroy data. Yep. Intention. But that's all. The thing about well-intentioned but incorrect locking code is that it will appear to work fine, until it trips over the one code path where it forgets to lock some file that it should have locked. And even then, the code will "work" just fine, until multiple processes are accessing that file at the same time. A well-intentioned but incorrect mandatory locking can result in data inconsistency because it is not using locking correctly, and thus introducing race conditions, and can also result in dead-locks. It's really quite simple. If the algorithm is correct, both advisory and mandatory locking will work. If the algorithm is not correct, neither will. What mandatory locking provides is a way to lock out anyone who doesn't use locking. The only two situations I can think of for this are incorrect algorithms and intentional behavior. In the first case, you still gain nothing, because mandatory locking is not protecting you from incorrect algorithms, only from those with an error as huge as not using locking at all. In the second case, mandatory locking is useless, as I previously demonstrated. Now, if you know any other situation where mandatory locking results in a different behavior from advisory locking, please feel free to enlighten me. I think it is appropriate for an operating system to provide an option such that *it* (the system) will enforce the locking, and not have to trust that all code-paths in all programs will do the right thing WRT advisory locking. Actually, I have nothing against having it as an option, as long as root can override it. It opens a pandora box for denial of service attacks, but as long as I, the system manager, can completely disable it (by inaction, preferably), I don't mind. (I also think the implementation needs to be thought out carefully, to make sure it doesn't become a way for a malicious user to implement denial-of-service attacks...) Wanna make a bet that if we introduce this feature today we'll still be closing security holes by 2005? -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
At 11:27 PM -0400 8/23/99, Christopher Masto wrote: On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: What has that code you show above got to do with mandatory locking? You completely missed the explicit locking calls that you have to make, to get and release the locks. If you don't make the call, and you have madatory locking, then your process will sleep until someone else releases the lock; Exactly. You said that mandatory locking means that user A's correct use of locking means that user B doesn't have to be careful. That's not the case, since A can step in between B's read and write. A's mandatory lock doesn't help. I don't see the use for it. Then don't use it. There is one thing about this discussion that you seem to be missing. We are not talking about mandatory locking for *all* files. Only for files where mandatory locking is explicitly requested. If you don't have a use for mandatory locking, then do not go out of your way to turn it on for the files you are using. We are just saying that if some developer does want to use mandatory locking, it would be nice if FreeBSD provided that as an option. And as an option which doesn't require writing special daemons and other tricks just to make sure that programs have to lock the file. Not sure if you were at RPI in the days of MTS, but mandatory locking certainly does come in handy in some cases. --- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
+[ Christopher Masto ]- | | that no other processes are using it." How do you know someone | hasn't, say, opened it in an editor, made some changes, and is about | to save? File Permissions, it's a pretty fundamental UNIX philosophy. How do you know someone hasn't opened your kernel in an editor, made some changes and is about to save? It's not about stopping random users doing random actions to files. It's about (almost) transparently enabling processes to cooperate using a shared resource. Think of it as a mutex on a part/whole file. Mandatory locking enables a process to ensure that its transaction is safe from interference. Interference that can come from a correctly running program writing at the wrong time (but not using the locks -- maybe you don't have source for it either). What happens if root-owned process X has gone off the deep-end and is randomly writing crap into every file on your filesystem? Well you're hosed anyway. -- Totally Holistic Enterprises Internet| P:+61 7 3870 0066 | Andrew The Internet (Aust) Pty Ltd | F:+61 7 3870 4477 | Milton ACN: 082 081 472 | M:+61 416 022 411 |72 Col .Sig PO Box 837 Indooroopilly QLD 4068|[EMAIL PROTECTED]|Specialist To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
At 11:17 AM -0400 8/24/99, Christopher Masto wrote: I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a "useless" feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can "only" cause a denial of service.. or maybe I can make a new race condition in a periodic script. I am also concerned about the implementation, and about the opening of denial-of-service attacks. How about we start talking about some possible implementations, and see which ones will work the best? Your position seems to be "it is possible that a bad implementation of this might cause some security issues -- and therefore let's not do it at all, even if it might be useful". My position is that "It is useful, and demonstrated as useful on many OS's, so let us sit down and figure out a good implementation". --- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 11:33:33AM -0400, Chuck Robey wrote: Yes, it's WRONG code. Correct code would aquire a lock before reading. My understanding of "mandatory locking" means that a program that completely ignores any locking at all, if it tries to grab the locked resource, will sleep until the resource is unlocked. This means that the program is mandatorily forced to obey other program's lock requests. It does not force that program to do it's own locks, but it forces it to respect others. Is that the kind of mandatory locking we're talking about? The rogue program, which doesn't do locking, can have it''s data corrupted, but it cannot corrupt the data for a correctly written program's transaction. I'm going to keep out of this thread from now on, but I want to answer this first. The "correctly written program" may aquire the lock and perform its transaction between the time the "rogue program" has read data and the time it is going to write that data back. In this case, mandatory locking only means that the rogue stalls in the middle, but it's still working with incorrect data. I don't think there's any way to update a file properly other than to aquire a lock before reading and to hold it until after writing. If the process with the mandatory locking is the only process that ever writes to the file, it may be OK. That seems a somewhat narrow use for the thing. I'm going to defer any futher comment until I see an implementation. It seems, as I said before, that it's not even clear to those participating in this discussion exactly what the definition of "mandatory locking" is intended to be. I never got to use MTS, but Garance seems to be saying that certain files could be flagged (perhaps this was the default) such that you can't access them at all if you don't aquire a lock. Others are implying that a "non-compliant" program only blocks if there is a lock on the file. As for "if you don't like it, don't enable it".. that's only an option if nothing appears that depends on it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Locking in Vinum (was: Mandatory locking?)
On Tuesday, 24 August 1999 at 10:59:34 +1000, Andrew Reilly wrote: Hi Greg, hackers list, I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. I thought that access was mediated by a driver or server process. Sure, that's how they do it. It's a driver, not a server, and the top half runs in process context. Isn't this process in a position to ensure the integrity of the transaction without any help from locking mechanisms? No. But what we're talking about here has nothing to do with Vinum's locking, which is internal. If some major maintenance utility needs to run on the raw device, during which time the state is inconsistent as far as user processes are concerned, isn't it sufficient to remove their read priveliges? What if the volumes are open? What if you don't want to upset the processes, just keep them away a little while you do your thing? While testing Vinum, I found a number of race conditions. Most lasted about 50 ms. I have to lock to prevent corruption, but nobody would notice the delay. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, 24 Aug 1999, Christopher Masto wrote: Ok, so that means the program doesn't have to be so poorly written. It can read part of the file, then go to write it just after User 1 locks that part of the file, so its write will block until User 1 is finished, but will still stomp over User 1's data. No need to close and reopen the file. Yes, it's WRONG code. Correct code would aquire a lock before reading. My understanding of "mandatory locking" means that a program that completely ignores any locking at all, if it tries to grab the locked resource, will sleep until the resource is unlocked. This means that the program is mandatorily forced to obey other program's lock requests. It does not force that program to do it's own locks, but it forces it to respect others. Is that the kind of mandatory locking we're talking about? The rogue program, which doesn't do locking, can have it''s data corrupted, but it cannot corrupt the data for a correctly written program's transaction. So how does mandatory locking help? In this situation, I don't think it does. And this situation is analogous to "I want to modify this file without taking it offline, but I need to guarantee that no other processes are using it." How do you know someone hasn't, say, opened it in an editor, made some changes, and is about to save? Especially when that modification occurs spontaneously, perhaps as part of some recovery process unknown to the user editing the file. I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a "useless" feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can "only" cause a denial of service.. or maybe I can make a new race condition in a periodic script. By the way, I like the idea of mandatory locking, and I "grew up" on an OS that had it. -- Christopher Masto Senior Network Monkey NetMonger Communications [EMAIL PROTECTED][EMAIL PROTECTED]http://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---+--- Chuck Robey| Interests include any kind of voice or data [EMAIL PROTECTED] | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 05:51:54PM -0400, Tim Vanderhoek wrote:
How 'bout "anyone who can kill the process holding the lock?"
On further reflection, I'd go even further: anyone who can set the
lock can break the lock. Presumably if they know enough to explicitly
break the lock, then they know enough to know what they're doing.
This is more-or-less like a chmod("-w")
--
This is my .signature which gets appended to the end of my messages.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. This could also be accomplished with a volume access lock at the CAM level. I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. For example, if I have a QUOTAFS that accesses the file "/.quota", and then lies during VOP_READDIR and other name lookup operations in order to hide the "/.quota" file from prying eyes. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Use of a mandatory lock mechanism has a significantly higher granularity than a logical volume lock (which is what would have to be use, instead of a physical volume, unless we can be guaranteed that the parity bits plus the bits over which the parity is being calculated do not span more than a single physical volume. It seems to me that this is a proper application of mandatory locks. There also seems to be a general misconception about mandatory locking implementation in SVR4 (or in general). The point of mandatory locks is to allow you to _prevent_ access to files where the locking applies, except when such access is encapsulated with a lock. This means that the "race scenarios", where a badly behaved process is able to thwart the locking, don't exist. For the other "deadlock" scenarios, mandatory locks are no worse than a chflags'ed ld.so file that you can't replace unless you chflag it back, and certainly no worse than the FreeBSD behaviour that prevented updating a running executable, in the past (EBUSY). At worst, you can always kill the process that has the lock, and allow resource tracking to clean up after it. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
On Wednesday, 25 August 1999 at 1:52:38 +, Terry Lambert wrote: I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. This could also be accomplished with a volume access lock at the CAM level. It could be, but it would introduce unnecessary waits. I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. Interesting terminology. I think you've lost most people already. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Correct. I lock a stripe at a time. It seems to me that this is a proper application of mandatory locks. Sure. It also has nothing to do with the userland file locking I was talking about in the previous thread. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. Vinum isn't proposing fcntl-based locks. I was looking at them for a specific application; Vinum has always had its own locking mechanism (see sys/dev/vinum/vinumlock.c). This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Can you point me to the patches again? Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. Interesting terminology. I think you've lost most people already. I hope not. It's not that hard a concept. You can hide filesystem objects from an upper layer, or you can "fold" directories. It's the idea of multiple views: hiding: folding: view 1: /foo/fee/.quota /xxx/data/yyy/data -- data fork /foo/fee/bob/xxx/rsrc/yyy/rsrc view 2: /foo/fee/bob/xxx/yyy-- data fork //rsrc//xxx/yyy -- resource fork I.e.: trivial. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Correct. I lock a stripe at a time. Again, I think that, even with a user space daemon, you really don't want to use an fcntl mechanism to deal with this. I think you really want an ioctl() against a device file, instead. It seems to me that this is a proper application of mandatory locks. Sure. It also has nothing to do with the userland file locking I was talking about in the previous thread. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. Vinum isn't proposing fcntl-based locks. I was looking at them for a specific application; Vinum has always had its own locking mechanism (see sys/dev/vinum/vinumlock.c). OK, it wasn't clear that you had jumped boats. This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Can you point me to the patches again? These particular ones are probably outdated by some more recent ones by Mike Hancock, but they should be in the mondo patch set that was intended to allow NFS server locking. It's in my home directory on freebsd.org. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Tuesday, 24 August 1999 at 22:41:15 -0400, Albert D. Cahalan wrote: It is clear to me that BSD won't do this. SysV and Linux have this feature. Linux runs everywhere that FreeBSD does and has better features too... so why run BSD at all? I assume you're talking about mandatory locking. Yes, you're right, FreeBSD has no merit, so why bother even reading the mailing lists? Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file "xyz" open? kind of query). btw, is there any way to get this info now in FBSD? Also, file need not be explicitly locked. Opening a file puts proper lock. A file which is open can't be deleted also ;) This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. All the files need not have this option. there must be a way to specify that "this file needs mandatory locking" to the kernel. I don't know what all changes are needed in the kernel.. but I'm thinking about dup() and fork().. What is the fate of the lock if the process does a fork()? I hope this makes sense.. -Biju To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 8:31:23 +0530, Biju Susmer wrote: All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write This isn't locking, it's access exclusion. It's also not correct for NSK. NSK's OPEN procedure allows the following possibilities: - shared access. Any process can open the file unless another process has it opened with a different access. - exclusive access. Only this process can open the file. If it's already open, the open call will fail. - protected access. Only this process can have the file open for write. If it's already open for writing, the open call will fail. NSK uses the procedures LOCKFILE and LOCKREC for locking. Like most operating systems, locks are in the kernel and mandatory. * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file "xyz" open? kind of query). btw, is there any way to get this info now in FBSD? fstat or lsof. Also, file need not be explicitly locked. Opening a file puts proper lock. I'm not sure what you're saying here. Opening a file doesn't lock it. If the file is audited, you must issue a BEGINTRANSACTION call before you can access it, and TMF will place locks where necessary. A file which is open can't be deleted also ;) That's because of NSK's simplistic file system. This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. Again, I'm not sure how much of this you understand. I was thinking of doing something like NSK's LOCKREC function. All the files need not have this option. there must be a way to specify that "this file needs mandatory locking" to the kernel. That's a point for discussion. I don't know what all changes are needed in the kernel.. but I'm thinking about dup() and fork().. What is the fate of the lock if the process does a fork()? I can think of a number of scenarios. Certainly another point for discussion. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Mandatory locking?
This isn't locking, it's access exclusion. It's also not correct for NSK. what is the difference between locking and access exclusion? i was thinking both are same (locking implies access exclusion). in other words, My idea of mandatory locking is same as exclusive access to the file. are we in sync? else tell me what exactly u mean by "Mandatory locking". NSK's OPEN procedure allows the following possibilities: - shared access. Any process can open the file unless another process has it opened with a different access. - exclusive access. Only this process can open the file. If it's already open, the open call will fail. - protected access. Only this process can have the file open for write. If it's already open for writing, the open call will fail. Fine. This is true. i didn't want to give too much details. What i want to tell is, there need not be a special call for lock. Combining lock/access exclusion with open will be better. Also, file need not be explicitly locked. Opening a file puts proper lock. I'm not sure what you're saying here. Opening a file doesn't lock it. same as above. put lock in open, not after getting fd. open should fail if file is locked. This will help programs unaware of locking to behave better. This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. Again, I'm not sure how much of this you understand. I was thinking of doing something like NSK's LOCKREC function. LOCKREC locks records only. The problem is: processes A B. B doesn't know about locking (like 'cat') 1. A opened file 2. It locked some part of the file 3. B opens and starts to write 4. B reached the region which is lockd and write blocks 5. A releases lock 6. B's write is unblocked. Here, how mandatory locking of a region help? isn't this same as advisory locking? If opening itself fails coz of lock, the programs unaware of mandatory locking will be forced to start from begining. -Biju To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
Sean Eric Fagan wrote: The fact that Greg thinks it's necessary and desirable (and he has considerably more OS experience than a lot of the people who have decided it's a stupid idea) should alone say a lot for the idea. I was waiting for someone else to bring up that point, because I might be accused of partiality. The fact that you and Terry both seem to agree with Greg makes it a very rare triumvirate. So, who's going to do it? ;^) -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
"Daniel C. Sobral" wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. ;^) -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
On Tuesday, 24 August 1999 at 22:28:10 -0700, Sean Eric Fagan wrote: In article [EMAIL PROTECTED] you write: Correct. I lock a stripe at a time. What people need to realize is that Greg is doing this locking in user mode. As such, he has two real options: 1. Implement a vinum-specific ioctl that locks a region of a file at the device level, or 2. Implement standard mandatory region locking, which damned near every OS in existence has, and which OSes which have existed for decades longer than unix has existed have always had. Now, because this _is_ (currently) a vinum-specific requirement, doing (1) is not all that unpalatable. However, it's at the wrong level (device driver, instead of file), and it is merely putting off the inevitable. In fact, this confusion with Vinum is more historical than anything. I started thinking "what tools are available for this as yet not clearly defined task that will run in user mode and require locking?". The obvious first question was "do we support read (i.e. mandatory) locking?". Or does nobody wish to have working Linux and Solaris/x86 compatibility after all? That reason, if nothing else, is reason to design it, so it gets done correctly. Exactly, and that was why I started this thread. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 0:11:23 -0600, Wes Peters wrote: "Daniel C. Sobral" wrote: Christopher Masto wrote: I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) And how many programmers with nearly (or more than) two decades of UNIX experience it takes to convince someone it really is useful. I must say, I'm really amazed at some of the opinions that have been voiced in this thread. Of course, that's all they are, and they show the origins of their owners. Any system with multiple concurrent accesses requires locking. Only UNIX uses advisory locking. It almost does the job, so nobody has tried to fix things. But that doesn't make it right. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
chu...@picnic.mat.net (Chuck Robey) writes: On 23 Aug 1999, Ville-Pertti Keinonen wrote: And even without otherwise incorrect behavior, if you have a program that doesn't use any locking and another one that uses mandatory locking to prevent races with the non-locking program, the mere existence of the locking program does not prevent multiple non-locking programs from generating similar conditions. That's very odd, I thought the idea behind mandatory locking was to completely eliminate the possibility that a program could do what you're saying; all programs would *mandatorily* be forced to do locking to access the resource. I don't know what the textbook definition for mandatory locking is, but was assuming (particularly considering the proposal to use a fcntl interface) that by mandatory locking, Greg was referring to a harder lock than current advisory locking, one that had to be instantiated explicitly but would not only lock out other attempts to lock, but all other attempts to access the file. The further messages in this the thread seems to indicate that different individuals have different definitions for mandatory locking... I'd still assume that marking a file to be accessible by only one process at a time is *not* what is being discussed. Particularly since it is not even clear what this would mean for forked processes, dup, sending file descriptors over local sockets etc. Note that my arguments earlier don't apply in a case where you might want to e.g. ensure consistency for non-locking programs with read-only access, with the only program with privileges to modify the data making the data inaccessible during updates. This is a scenario where it would, IMHO, actually be quite useful to have mandatory locking. In any case, if shared (open) access is allowed, such a feature can introduce semantic changes to read/write system calls - normally, read/write can never return EAGAIN or block for unlimited amounts of time on regular, local files. EAGAIN is not that much of a problem, as it requires explicitly setting O_NONBLOCK, but blocking can introduce new deadlocks. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
g...@lemis.com (Greg Lehey) writes: an agreement of some kind. But what if I want to merge the contents of another mail folder: cat oldmail /var/mail/grog That works, but it's playing with fire: if sendmail is delivering a message at the same time, it won't see me, and my cat doesn't get a lock beforehand, so both an incoming message and part of my mail folder could end up getting written to the same location. With mandatory locking, it would work, transparently. Certainly not with range-locking rather than file-locking. cat is certainly not guaranteed to be atomic, and while you shouldn't end up writing things in the same location, what might happen unless you are preventing multiple openers is: cat writes part of oldmail to /var/mail/grog sendmail locks /var/mail/grog (cat may try to write more to /var/mail/grog but blocks) sendmail delivers new mail sendmail unlocks /var/mail/grog cat writes the rest of oldmail to /var/mail/grog You'll still probably end up with a broken mailbox. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Ville-Pertti Keinonen scribbled this message on Aug 24:
g...@lemis.com (Greg Lehey) writes:
an agreement of some kind. But what if I want to merge the contents
of another mail folder:
cat oldmail /var/mail/grog
That works, but it's playing with fire: if sendmail is delivering a
message at the same time, it won't see me, and my cat doesn't get a
lock beforehand, so both an incoming message and part of my mail
folder could end up getting written to the same location. With
mandatory locking, it would work, transparently.
Certainly not with range-locking rather than file-locking. cat is
certainly not guaranteed to be atomic, and while you shouldn't end up
writing things in the same location, what might happen unless you are
preventing multiple openers is:
cat writes part of oldmail to /var/mail/grog
sendmail locks /var/mail/grog
(cat may try to write more to /var/mail/grog but blocks)
sendmail delivers new mail
sendmail unlocks /var/mail/grog
cat writes the rest of oldmail to /var/mail/grog
You'll still probably end up with a broken mailbox.
what you do is this:
lockf -k $mailfile cat ${mailtmp} $mailfile
then you don't have to worry.. that's what lockf is for...
--
John-Mark Gurney Voice: +1 541 684 8449
Cu Networking P.O. Box 5693, 97405
The soul contains in itself the event that shall presently befall it.
The event is only the actualizing of its thought. -- Ralph Waldo Emerson
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
gurne...@efn.org (John-Mark Gurney) writes:
Ville-Pertti Keinonen scribbled this message on Aug 24:
cat writes part of oldmail to /var/mail/grog
sendmail locks /var/mail/grog
(cat may try to write more to /var/mail/grog but blocks)
sendmail delivers new mail
sendmail unlocks /var/mail/grog
cat writes the rest of oldmail to /var/mail/grog
You'll still probably end up with a broken mailbox.
what you do is this:
lockf -k $mailfile cat ${mailtmp} $mailfile
Which doesn't support Greg's arguments for mandatory locking, as
you're now doing locking in both programs.
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tuesday, 24 August 1999 at 12:38:38 +0300, Ville-Pertti Keinonen wrote:
gurne...@efn.org (John-Mark Gurney) writes:
Ville-Pertti Keinonen scribbled this message on Aug 24:
cat writes part of oldmail to /var/mail/grog
sendmail locks /var/mail/grog
(cat may try to write more to /var/mail/grog but blocks)
sendmail delivers new mail
sendmail unlocks /var/mail/grog
cat writes the rest of oldmail to /var/mail/grog
You'll still probably end up with a broken mailbox.
what you do is this:
lockf -k $mailfile cat ${mailtmp} $mailfile
Which doesn't support Greg's arguments for mandatory locking, as
you're now doing locking in both programs.
Well, it doesn't support the last argument I made, which was
untenable. It doesn't affect most of the others. And we're some
distance from the dreaded deadlocks.
Greg
--
See complete headers for address, home page and phone numbers
finger g...@lemis.com for PGP public key
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Christopher Masto wrote: How about a little timing difference? User 1 User 2 open file read file open file lock file (blocks?) close file lock returnsopen file (blocks) diddle file unlock file scribble over User 1's changes What I'm getting at is that if User 2 has to do something special anyway, it might as well be using advisory locking. You've got this so wrong, perhaps you should just go find a System V man page and read about mandatory locking before embarassing yourself any- more. Locking will only block if another process is holding an overlapping lock. opening won't block due to mandatory locking. The only operations that can block are read and write, and then only if the read or write operation will overlap a locked range of bytes. Your example above is a perfect example of a poorly written program doing something stupid. If you allow poorly written programs to do stupid things on top of YOUR critical data files, that is your problem. Not having mandatory locking is OUR problem. If you don't want it, just leave option LOCKING out of your kernel. -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Christopher Masto wrote: The thing about well-intentioned but incorrect locking code is that it will appear to work fine, until it trips over the one code path where it forgets to lock some file that it should have locked. And even then, the code will work just fine, until multiple processes are accessing that file at the same time. I think it is appropriate for an operating system to provide an option such that *it* (the system) will enforce the locking, and not have to trust that all code-paths in all programs will do the right thing WRT advisory locking. Dunno about that.. if you're using advisory locking, you know to say lock the file, then read the data, do your calculation, write it out, and unlock. This manditory locking sounds like an invitation for disaster. I don't need to pay attention to the details because the kernel will take care of it for me. Wrong paradigm. Look at it from the lockers standpoint: Even if other processes don't do locking, they won't be able to screw me because I've locked the critical part of the file. Actually, I don't really understand the paradigm. Two processes need to safely update a file, so one of them aquires a mandatory lock, and the other.. uh.. just blocks trying to open the file? No, the locks are not per-file. You can lock multiple arbitrary byte ranges. Any file trying to write (shared lock) and/or read (exclusive lock) will be blocked until the lock is released. -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Tim Vanderhoek wrote: On Mon, Aug 23, 1999 at 10:12:38PM +0200, Mark Murray wrote: In process-space, this is the kernel. In file-space, this should be root. Processes that require mandatory locking must revoke superuser before attempting locks. I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... How 'bout anyone who can kill the process holding the lock? -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 08:35:25AM -0600, Wes Peters wrote: You've got this so wrong, perhaps you should just go find a System V man page and read about mandatory locking before embarassing yourself any- more. First of all, when was it decided that we were all talking about System V? I think that's part of the problem here; some people are arguing whether locking should exist while others are arguing what the semantics should be. I have seen (at least) four ideas mentioned: SysV Root doesn't obey locking Root can override locking Only root can lock So don't insult me for giving a what if? example. If it's not a problem under certain semantics, that's good. Let's see if that's the case. Locking will only block if another process is holding an overlapping lock. opening won't block due to mandatory locking. The only operations that can block are read and write, and then only if the read or write operation will overlap a locked range of bytes. Ok, so that means the program doesn't have to be so poorly written. It can read part of the file, then go to write it just after User 1 locks that part of the file, so its write will block until User 1 is finished, but will still stomp over User 1's data. No need to close and reopen the file. Yes, it's WRONG code. Correct code would aquire a lock before reading. So how does mandatory locking help? In this situation, I don't think it does. And this situation is analogous to I want to modify this file without taking it offline, but I need to guarantee that no other processes are using it. How do you know someone hasn't, say, opened it in an editor, made some changes, and is about to save? Especially when that modification occurs spontaneously, perhaps as part of some recovery process unknown to the user editing the file. I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a useless feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can only cause a denial of service.. or maybe I can make a new race condition in a periodic script. By the way, I like the idea of mandatory locking, and I grew up on an OS that had it. -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Chuck Robey wrote: On Mon, 23 Aug 1999, Christopher Masto wrote: Bleah.. I can't count the number of times I've seen idiotic code like: open file read data close file open file for write write data close file Mandatory locking of the type above doesn't force such a thing to work. What has that code you show above got to do with mandatory locking? You completely missed the explicit locking calls that you have to make, to get and release the locks. If you don't make the call, and you have madatory locking, then your process will sleep until someone else releases the lock; if you only have advisory locking, and you use the miscreant code you show, then indeed things will go awry. You missed the point. Insert the appropriate lockingunlocking at each open/close operation. See? :-) -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Christopher Masto wrote: Exactly. You said that mandatory locking means that user A's correct use of locking means that user B doesn't have to be careful. That's not the case, since A can step in between B's read and write. A's mandatory lock doesn't help. I don't see the use for it. :-) The thing is SO obviously flawed, that I wonder how many marketoid drones it took to make sensible people think it is actually useful. :-) -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
Garance A Drosihn wrote: There's a difference between a program which has a locking-oversight or race-condition, and a program you write to deliberately destroy data. Yep. Intention. But that's all. The thing about well-intentioned but incorrect locking code is that it will appear to work fine, until it trips over the one code path where it forgets to lock some file that it should have locked. And even then, the code will work just fine, until multiple processes are accessing that file at the same time. A well-intentioned but incorrect mandatory locking can result in data inconsistency because it is not using locking correctly, and thus introducing race conditions, and can also result in dead-locks. It's really quite simple. If the algorithm is correct, both advisory and mandatory locking will work. If the algorithm is not correct, neither will. What mandatory locking provides is a way to lock out anyone who doesn't use locking. The only two situations I can think of for this are incorrect algorithms and intentional behavior. In the first case, you still gain nothing, because mandatory locking is not protecting you from incorrect algorithms, only from those with an error as huge as not using locking at all. In the second case, mandatory locking is useless, as I previously demonstrated. Now, if you know any other situation where mandatory locking results in a different behavior from advisory locking, please feel free to enlighten me. I think it is appropriate for an operating system to provide an option such that *it* (the system) will enforce the locking, and not have to trust that all code-paths in all programs will do the right thing WRT advisory locking. Actually, I have nothing against having it as an option, as long as root can override it. It opens a pandora box for denial of service attacks, but as long as I, the system manager, can completely disable it (by inaction, preferably), I don't mind. (I also think the implementation needs to be thought out carefully, to make sure it doesn't become a way for a malicious user to implement denial-of-service attacks...) Wanna make a bet that if we introduce this feature today we'll still be closing security holes by 2005? -- Daniel C. Sobral(8-DCS) d...@newsguy.com d...@freebsd.org - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
+[ Christopher Masto ]- | | that no other processes are using it. How do you know someone | hasn't, say, opened it in an editor, made some changes, and is about | to save? File Permissions, it's a pretty fundamental UNIX philosophy. How do you know someone hasn't opened your kernel in an editor, made some changes and is about to save? It's not about stopping random users doing random actions to files. It's about (almost) transparently enabling processes to cooperate using a shared resource. Think of it as a mutex on a part/whole file. Mandatory locking enables a process to ensure that its transaction is safe from interference. Interference that can come from a correctly running program writing at the wrong time (but not using the locks -- maybe you don't have source for it either). What happens if root-owned process X has gone off the deep-end and is randomly writing crap into every file on your filesystem? Well you're hosed anyway. -- Totally Holistic Enterprises Internet| P:+61 7 3870 0066 | Andrew The Internet (Aust) Pty Ltd | F:+61 7 3870 4477 | Milton ACN: 082 081 472 | M:+61 416 022 411 |72 Col .Sig PO Box 837 Indooroopilly QLD 4068|a...@theinternet.com.au|Specialist To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
At 11:27 PM -0400 8/23/99, Christopher Masto wrote: On Mon, Aug 23, 1999 at 11:16:21PM -0400, Chuck Robey wrote: What has that code you show above got to do with mandatory locking? You completely missed the explicit locking calls that you have to make, to get and release the locks. If you don't make the call, and you have madatory locking, then your process will sleep until someone else releases the lock; Exactly. You said that mandatory locking means that user A's correct use of locking means that user B doesn't have to be careful. That's not the case, since A can step in between B's read and write. A's mandatory lock doesn't help. I don't see the use for it. Then don't use it. There is one thing about this discussion that you seem to be missing. We are not talking about mandatory locking for *all* files. Only for files where mandatory locking is explicitly requested. If you don't have a use for mandatory locking, then do not go out of your way to turn it on for the files you are using. We are just saying that if some developer does want to use mandatory locking, it would be nice if FreeBSD provided that as an option. And as an option which doesn't require writing special daemons and other tricks just to make sure that programs have to lock the file. Not sure if you were at RPI in the days of MTS, but mandatory locking certainly does come in handy in some cases. --- Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu Senior Systems Programmer or dro...@rpi.edu Rensselaer Polytechnic Institute To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
At 11:17 AM -0400 8/24/99, Christopher Masto wrote: I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a useless feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can only cause a denial of service.. or maybe I can make a new race condition in a periodic script. I am also concerned about the implementation, and about the opening of denial-of-service attacks. How about we start talking about some possible implementations, and see which ones will work the best? Your position seems to be it is possible that a bad implementation of this might cause some security issues -- and therefore let's not do it at all, even if it might be useful. My position is that It is useful, and demonstrated as useful on many OS's, so let us sit down and figure out a good implementation. --- Garance Alistair Drosehn = g...@eclipse.acs.rpi.edu Senior Systems Programmer or dro...@rpi.edu Rensselaer Polytechnic Institute To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 11:33:33AM -0400, Chuck Robey wrote: Yes, it's WRONG code. Correct code would aquire a lock before reading. My understanding of mandatory locking means that a program that completely ignores any locking at all, if it tries to grab the locked resource, will sleep until the resource is unlocked. This means that the program is mandatorily forced to obey other program's lock requests. It does not force that program to do it's own locks, but it forces it to respect others. Is that the kind of mandatory locking we're talking about? The rogue program, which doesn't do locking, can have it''s data corrupted, but it cannot corrupt the data for a correctly written program's transaction. I'm going to keep out of this thread from now on, but I want to answer this first. The correctly written program may aquire the lock and perform its transaction between the time the rogue program has read data and the time it is going to write that data back. In this case, mandatory locking only means that the rogue stalls in the middle, but it's still working with incorrect data. I don't think there's any way to update a file properly other than to aquire a lock before reading and to hold it until after writing. If the process with the mandatory locking is the only process that ever writes to the file, it may be OK. That seems a somewhat narrow use for the thing. I'm going to defer any futher comment until I see an implementation. It seems, as I said before, that it's not even clear to those participating in this discussion exactly what the definition of mandatory locking is intended to be. I never got to use MTS, but Garance seems to be saying that certain files could be flagged (perhaps this was the default) such that you can't access them at all if you don't aquire a lock. Others are implying that a non-compliant program only blocks if there is a lock on the file. As for if you don't like it, don't enable it.. that's only an option if nothing appears that depends on it. -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 08:25:59AM -0600, Wes Peters wrote: I don't like restricting the breaking of mandatory locks to the superuser. It could be restricted to specific users (say file owner + root)... How 'bout anyone who can kill the process holding the lock? + file owner ( + root ). Otherwise I would be able to lock ~wes/FreeBSDmarkers and you wouldn't be able to do anything about it until either notifying me or notifying root about the process I accidentally left hanging. [Not that I'm likely to ever need more than an advisory lock on that particular file, but the principle... :-] Hm. ``chmod go-MayLock ~wes/Fre*'' The sticky bit could be used. -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Locking in Vinum (was: Mandatory locking?)
On Tuesday, 24 August 1999 at 10:59:34 +1000, Andrew Reilly wrote: Hi Greg, hackers list, I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. I thought that access was mediated by a driver or server process. Sure, that's how they do it. It's a driver, not a server, and the top half runs in process context. Isn't this process in a position to ensure the integrity of the transaction without any help from locking mechanisms? No. But what we're talking about here has nothing to do with Vinum's locking, which is internal. If some major maintenance utility needs to run on the raw device, during which time the state is inconsistent as far as user processes are concerned, isn't it sufficient to remove their read priveliges? What if the volumes are open? What if you don't want to upset the processes, just keep them away a little while you do your thing? While testing Vinum, I found a number of race conditions. Most lasted about 50 ms. I have to lock to prevent corruption, but nobody would notice the delay. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tue, 24 Aug 1999, Christopher Masto wrote: Ok, so that means the program doesn't have to be so poorly written. It can read part of the file, then go to write it just after User 1 locks that part of the file, so its write will block until User 1 is finished, but will still stomp over User 1's data. No need to close and reopen the file. Yes, it's WRONG code. Correct code would aquire a lock before reading. My understanding of mandatory locking means that a program that completely ignores any locking at all, if it tries to grab the locked resource, will sleep until the resource is unlocked. This means that the program is mandatorily forced to obey other program's lock requests. It does not force that program to do it's own locks, but it forces it to respect others. Is that the kind of mandatory locking we're talking about? The rogue program, which doesn't do locking, can have it''s data corrupted, but it cannot corrupt the data for a correctly written program's transaction. So how does mandatory locking help? In this situation, I don't think it does. And this situation is analogous to I want to modify this file without taking it offline, but I need to guarantee that no other processes are using it. How do you know someone hasn't, say, opened it in an editor, made some changes, and is about to save? Especially when that modification occurs spontaneously, perhaps as part of some recovery process unknown to the user editing the file. I'm sure there are situations where mandatory locking accomplishes something useful. Are they worth it? (I don't claim to know; if the problems I thought I pointed out don't really exist, good.) More seriously than just being a useless feature, I am concerned about the possibility of opening up security holes with mandatory locking. BSD Unix does not currently have it.. if I understand correctly, the kind of locking we're talking about means that if I can get another user to read a file I own, I can make them block indefinately. Maybe I can't do anything bad with that.. maybe I can only cause a denial of service.. or maybe I can make a new race condition in a periodic script. By the way, I like the idea of mandatory locking, and I grew up on an OS that had it. -- Christopher Masto Senior Network Monkey NetMonger Communications ch...@netmonger.neti...@netmonger.nethttp://www.netmonger.net Free yourself, free your machine, free the daemon -- http://www.freebsd.org/ ---+--- Chuck Robey| Interests include any kind of voice or data chu...@picnic.mat.net | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770| picnic.mat.net: FreeBSD/i386 (301) 220-2114 | jaunt.mat.net : FreeBSD/Alpha ---+--- To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tue, Aug 24, 1999 at 05:51:54PM -0400, Tim Vanderhoek wrote: How 'bout anyone who can kill the process holding the lock? On further reflection, I'd go even further: anyone who can set the lock can break the lock. Presumably if they know enough to explicitly break the lock, then they know enough to know what they're doing. This is more-or-less like a chmod(-w) -- This is my .signature which gets appended to the end of my messages. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. This could also be accomplished with a volume access lock at the CAM level. I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. For example, if I have a QUOTAFS that accesses the file /.quota, and then lies during VOP_READDIR and other name lookup operations in order to hide the /.quota file from prying eyes. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Use of a mandatory lock mechanism has a significantly higher granularity than a logical volume lock (which is what would have to be use, instead of a physical volume, unless we can be guaranteed that the parity bits plus the bits over which the parity is being calculated do not span more than a single physical volume. It seems to me that this is a proper application of mandatory locks. There also seems to be a general misconception about mandatory locking implementation in SVR4 (or in general). The point of mandatory locks is to allow you to _prevent_ access to files where the locking applies, except when such access is encapsulated with a lock. This means that the race scenarios, where a badly behaved process is able to thwart the locking, don't exist. For the other deadlock scenarios, mandatory locks are no worse than a chflags'ed ld.so file that you can't replace unless you chflag it back, and certainly no worse than the FreeBSD behaviour that prevented updating a running executable, in the past (EBUSY). At worst, you can always kill the process that has the lock, and allow resource tracking to clean up after it. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
On Wednesday, 25 August 1999 at 1:52:38 +, Terry Lambert wrote: I don't want to express an opinion about the need or otherwise for mandatory locking, but I would appreciate a teensy clarification of the problem domain: On Mon, Aug 23, 1999 at 05:43:45PM +0930, Greg Lehey wrote: To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Are you suggesting that random user processes have to do all of this every time that they access a vinum drive? Yes. This could also be accomplished with a volume access lock at the CAM level. It could be, but it would introduce unnecessary waits. I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. Interesting terminology. I think you've lost most people already. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Correct. I lock a stripe at a time. It seems to me that this is a proper application of mandatory locks. Sure. It also has nothing to do with the userland file locking I was talking about in the previous thread. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. Vinum isn't proposing fcntl-based locks. I was looking at them for a specific application; Vinum has always had its own locking mechanism (see sys/dev/vinum/vinumlock.c). This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Can you point me to the patches again? Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
I think what people are missing here is that Vinum, when doing software RAID, is implementing a type of namespace escape, only it isn't a standard namespace escape. Interesting terminology. I think you've lost most people already. I hope not. It's not that hard a concept. You can hide filesystem objects from an upper layer, or you can fold directories. It's the idea of multiple views: hiding: folding: view 1: /foo/fee/.quota /xxx/data/yyy/data -- data fork /foo/fee/bob/xxx/rsrc/yyy/rsrc view 2: /foo/fee/bob/xxx/yyy-- data fork //rsrc//xxx/yyy -- resource fork I.e.: trivial. Because this escapes the whole file, it is _not_ like the Vinum usage, which needs to escape parity bits on a block device. The Vinum usage needs to prevent access to the file range covered by the parity bits, rather than merely protecting the parity bits. Correct. I lock a stripe at a time. Again, I think that, even with a user space daemon, you really don't want to use an fcntl mechanism to deal with this. I think you really want an ioctl() against a device file, instead. It seems to me that this is a proper application of mandatory locks. Sure. It also has nothing to do with the userland file locking I was talking about in the previous thread. For the use proposed by Vinum, however, fcntl() based mandatory locks are probably not the proper tool. Vinum isn't proposing fcntl-based locks. I was looking at them for a specific application; Vinum has always had its own locking mechanism (see sys/dev/vinum/vinumlock.c). OK, it wasn't clear that you had jumped boats. This is because you can only apply locks to devices that have a VOP_ADVLOCK on their backing store, and which use the VFSOPS based fileops structures. For the same reason that Linux user lament the inability to place advisory range locks on special files in FreeBSD, so, too, would Vinum be unable to place mandatory locks through that same mechanism against special files in FreeBSD. To correct this to allow it to work would require hanging the locks off of the vnode, instead of hanging them off the backing object (I have been suggesting -- and providing patches for -- this for literally years). Can you point me to the patches again? These particular ones are probably outdated by some more recent ones by Mike Hancock, but they should be in the mondo patch set that was intended to allow NFS server locking. It's in my home directory on freebsd.org. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Tuesday, 24 August 1999 at 22:41:15 -0400, Albert D. Cahalan wrote: It is clear to me that BSD won't do this. SysV and Linux have this feature. Linux runs everywhere that FreeBSD does and has better features too... so why run BSD at all? I assume you're talking about mandatory locking. Yes, you're right, FreeBSD has no merit, so why bother even reading the mailing lists? Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
RE: Mandatory locking?
All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? Also, file need not be explicitly locked. Opening a file puts proper lock. A file which is open can't be deleted also ;) This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. All the files need not have this option. there must be a way to specify that this file needs mandatory locking to the kernel. I don't know what all changes are needed in the kernel.. but I'm thinking about dup() and fork().. What is the fate of the lock if the process does a fork()? I hope this makes sense.. -Biju To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Wednesday, 25 August 1999 at 8:31:23 +0530, Biju Susmer wrote: All the files under Tandem's NSK has mandatory locking. The file cannot be opened if another process has it opened. some thing like * if the file is opened for reading, any one can open it for reading but opening for writing gives error * if the file is open for writing, it can't be opened for read/write This isn't locking, it's access exclusion. It's also not correct for NSK. NSK's OPEN procedure allows the following possibilities: - shared access. Any process can open the file unless another process has it opened with a different access. - exclusive access. Only this process can open the file. If it's already open, the open call will fail. - protected access. Only this process can have the file open for write. If it's already open for writing, the open call will fail. NSK uses the procedures LOCKFILE and LOCKREC for locking. Like most operating systems, locks are in the kernel and mandatory. * if the process holding the file is killed, the lock is gone * it is possible to get the pid of the process(es) which has a given file open (like which process has file xyz open? kind of query). btw, is there any way to get this info now in FBSD? fstat or lsof. Also, file need not be explicitly locked. Opening a file puts proper lock. I'm not sure what you're saying here. Opening a file doesn't lock it. If the file is audited, you must issue a BEGINTRANSACTION call before you can access it, and TMF will place locks where necessary. A file which is open can't be deleted also ;) That's because of NSK's simplistic file system. This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. Again, I'm not sure how much of this you understand. I was thinking of doing something like NSK's LOCKREC function. All the files need not have this option. there must be a way to specify that this file needs mandatory locking to the kernel. That's a point for discussion. I don't know what all changes are needed in the kernel.. but I'm thinking about dup() and fork().. What is the fate of the lock if the process does a fork()? I can think of a number of scenarios. Certainly another point for discussion. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
RE: Mandatory locking?
This isn't locking, it's access exclusion. It's also not correct for NSK. what is the difference between locking and access exclusion? i was thinking both are same (locking implies access exclusion). in other words, My idea of mandatory locking is same as exclusive access to the file. are we in sync? else tell me what exactly u mean by Mandatory locking. NSK's OPEN procedure allows the following possibilities: - shared access. Any process can open the file unless another process has it opened with a different access. - exclusive access. Only this process can open the file. If it's already open, the open call will fail. - protected access. Only this process can have the file open for write. If it's already open for writing, the open call will fail. Fine. This is true. i didn't want to give too much details. What i want to tell is, there need not be a special call for lock. Combining lock/access exclusion with open will be better. Also, file need not be explicitly locked. Opening a file puts proper lock. I'm not sure what you're saying here. Opening a file doesn't lock it. same as above. put lock in open, not after getting fd. open should fail if file is locked. This will help programs unaware of locking to behave better. This kind of locking will solve most of the problems which are pointed out. Mandatory locking on a part of the file is in no way better than advisory locking. Again, I'm not sure how much of this you understand. I was thinking of doing something like NSK's LOCKREC function. LOCKREC locks records only. The problem is: processes A B. B doesn't know about locking (like 'cat') 1. A opened file 2. It locked some part of the file 3. B opens and starts to write 4. B reached the region which is lockd and write blocks 5. A releases lock 6. B's write is unblocked. Here, how mandatory locking of a region help? isn't this same as advisory locking? If opening itself fails coz of lock, the programs unaware of mandatory locking will be forced to start from begining. -Biju To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Locking in Vinum (was: Mandatory locking?)
In article 19990825113518.d83273.kithrup.freebsd.cvs-...@freebie.lemis.com you write: Correct. I lock a stripe at a time. What people need to realize is that Greg is doing this locking in user mode. As such, he has two real options: 1. Implement a vinum-specific ioctl that locks a region of a file at the device level, or 2. Implement standard mandatory region locking, which damned near every OS in existence has, and which OSes which have existed for decades longer than unix has existed have always had. Now, because this _is_ (currently) a vinum-specific requirement, doing (1) is not all that unpalatable. However, it's at the wrong level (device driver, instead of file), and it is merely putting off the inevitable. Or does nobody wish to have working Linux and Solaris/x86 compatibility after all? That reason, if nothing else, is reason to design it, so it gets done correctly. The fact that Greg thinks it's necessary and desirable (and he has considerably more OS experience than a lot of the people who have decided it's a stupid idea) should alone say a lot for the idea. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 8:47:34 +0200, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Greg Lehey writes: Why should it be made unavailable ? So that certain multiple accesses can be done atomically. You don't need that. You initialize a index to 0, and whenever the sector with that index is written, you increment it. At any one time you know that all parityblocks = your index are valid. Sure, that's pretty much what I do in the situation you're thinking about. But it won't work without locking. Take a look at vinumrevive.c and vinumrequest.c. My real question was more like "OK, I have a situation coming up in which I need to be able to lock out other processes. What tools are available for it". I don't know yet whether what I end up with can be solved by this method, but it's nice to know what tools you have. You certainly wouldn't want to lock out access to the entire device during this time. I'm a little surprised that there's any objection to the concept of mandatory locking. Too many of us have had wedged systems because of it I guess... Strange, I've probably used it more than anybody here, and I've never had a wedged system. Of course, you need to use it appropriately. 'rm' can be a lethal tool :-) Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
In message [EMAIL PROTECTED], Greg Lehey writes: Why should it be made unavailable ? So that certain multiple accesses can be done atomically. You don't need that. You initialize a index to 0, and whenever the sector with that index is written, you increment it. At any one time you know that all parityblocks = your index are valid. Sure, that's pretty much what I do in the situation you're thinking about. But it won't work without locking. Take a look at vinumrevive.c and vinumrequest.c. I still don't see the need for mandatory locking, or locking out user access in general... I'm a little surprised that there's any objection to the concept of mandatory locking. Too many of us have had wedged systems because of it I guess... Strange, I've probably used it more than anybody here, and I've never had a wedged system. Of course, you need to use it appropriately. 'rm' can be a lethal tool :-) Well, maybe you were more lucky, I've had my share of troubles, and I think the very concept stinks... -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On Monday, 23 August 1999 at 9:47:40 +0200, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Greg Lehey writes: Why should it be made unavailable ? So that certain multiple accesses can be done atomically. You don't need that. You initialize a index to 0, and whenever the sector with that index is written, you increment it. At any one time you know that all parityblocks = your index are valid. Sure, that's pretty much what I do in the situation you're thinking about. But it won't work without locking. Take a look at vinumrevive.c and vinumrequest.c. I still don't see the need for mandatory locking, or locking out user access in general... I'll try to explain, using a different example, since I'm not quite sure where my rebuild stuff is going yet. To write a block to a RAID-5 device, you need to: 1. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 5. Write the data block. 6. Write the parity block. Consider what happens if a second request which refers the same parity data comes in during this business: 1. Read the old data into a temporary buffer. 1a. Read the old data into a temporary buffer. 2. Read the old parity data corresponding to the data into a temporary buffer. 2a. Read the old parity data corresponding to the data into a temporary buffer. 3. XOR the two, storing the result in one of the temporary buffers. 3a. XOR the two, storing the result in one of the temporary buffers. 4. XOR the result with the data which is to be written. 4a. XOR the result with the data which is to be written. 5. Write the data block. 5a. Write the data block. 6. Write the parity block. 6a. Write the parity block. The parity data read at (2a) will be wrong. It won't know of the data block to be written at (6). If the volume becomes degraded, it may no longer know about the first transaction. For this, we don't need fcntl locking: it would be too slow, and it's all done in the kernel anyway. In userland, we'd use a different example: I make a number of financial transactions over the Internet. In each case, the system checks my account balance, transfers the money and deducts it from my account: 1. Check balance. 2. Perform transfer. 3. Write updated balance back. Again, if we have two concurrent transactions, we stand to gain money: the updated balance is likely not to know about the other transaction, and will thus "forget" one of the deductions. Now I suppose you're going to come and say that this is bad programming, and advisory locking would do the job if the software is written right. Correct. You could also use the same argument to say that memory protection isn't necessary, because a correctly written program doesn't overwrite other processes address space. It's the same thing: file protection belongs in the kernel. I'm a little surprised that there's any objection to the concept of mandatory locking. Too many of us have had wedged systems because of it I guess... Strange, I've probably used it more than anybody here, and I've never had a wedged system. Of course, you need to use it appropriately. 'rm' can be a lethal tool :-) Well, maybe you were more lucky, I've had my share of troubles, and I think the very concept stinks... You and Garrett. I've never heard this from *anybody* else. The general opinion I've heard until now was that the lack of mandatory locking was a significant weakness in BSD. Greg -- See complete headers for address, home page and phone numbers finger [EMAIL PROTECTED] for PGP public key To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On 23-Aug-99 Greg Lehey wrote: I'm a little surprised that there's any objection to the concept of mandatory locking. In transaction processing, locking is not optional, and if any process at all can access a file or set of files without locking, you can't guarantee the database integrity. Other OSs have used mandatory locking for decades, and System V has it too. So far I haven't seen any arguments, let alone valid ones, against having it in FreeBSD. I think its a good idea, and hey if people object it can always be an option like - option NO_MANDATORY_LOCKING Not quite - developers have to deal with the mess that it would cause - Matt for example says: :Ugh. Yuch. No, nothing to do with permission bits, not for something :this convoluted! Are you saying that he should just enable an option and forget about it when he tweaks something horrible in NFS that only a handfull of others understand ? :-I -- Brian [EMAIL PROTECTED][EMAIL PROTECTED] http://www.Awfulhak.org [EMAIL PROTECTED] Don't _EVER_ lose your sense of humour ! [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
[EMAIL PROTECTED] (Greg Lehey) writes: Again, if we have two concurrent transactions, we stand to gain money: the updated balance is likely not to know about the other transaction, and will thus "forget" one of the deductions. Now I suppose you're going to come and say that this is bad programming, and advisory locking would do the job if the software is written right. Correct. You could also use the same argument to say that memory protection isn't necessary, because a correctly written program doesn't overwrite other processes address space. It's the The difference is that if a program has privileges to screw up whatever you are protecting, it can do so even if you do have mandatory locking, simply by functioning incorrectly when it does gain access to the data. And even without otherwise incorrect behavior, if you have a program that doesn't use any locking and another one that uses mandatory locking to prevent races with the non-locking program, the mere existence of the locking program does not prevent multiple non-locking programs from generating similar conditions. (I'm not opposed to mandatory locking in principle, but I don't find your reasoning very convincing.) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
On 23 Aug 1999, Ville-Pertti Keinonen wrote: [EMAIL PROTECTED] (Greg Lehey) writes: Again, if we have two concurrent transactions, we stand to gain money: the updated balance is likely not to know about the other transaction, and will thus "forget" one of the deductions. Now I suppose you're going to come and say that this is bad programming, and advisory locking would do the job if the software is written right. Correct. You could also use the same argument to say that memory protection isn't necessary, because a correctly written program doesn't overwrite other processes address space. It's the The difference is that if a program has privileges to screw up whatever you are protecting, it can do so even if you do have mandatory locking, simply by functioning incorrectly when it does gain access to the data. And even without otherwise incorrect behavior, if you have a program that doesn't use any locking and another one that uses mandatory locking to prevent races with the non-locking program, the mere existence of the locking program does not prevent multiple non-locking programs from generating similar conditions. That's very odd, I thought the idea behind mandatory locking was to completely eliminate the possibility that a program could do what you're saying; all programs would *mandatorily* be forced to do locking to access the resource. It's the advisory locking that allows the scenario you paint. I think mandatory locking should exist, but only be available to root. If a program needs this, it must run with root privs, so that ordinary users cannot wedge the machine, but (as usual) root can shoot himself in the foot (traditional Unix methodology). (I'm not opposed to mandatory locking in principle, but I don't find your reasoning very convincing.) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message +--- Chuck Robey | Interests include any kind of voice or data [EMAIL PROTECTED] | communications topic, C programming, and Unix. 213 Lakeside Drive Apt T-1 | Greenbelt, MD 20770 | I run picnic and jaunt, both FreeBSD-current. (301) 220-2114 | +--- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Greg Lehey wrote: all done in the kernel anyway. In userland, we'd use a different example: I make a number of financial transactions over the Internet. In each case, the system checks my account balance, transfers the money and deducts it from my account: 1. Check balance. 2. Perform transfer. 3. Write updated balance back. Again, if we have two concurrent transactions, we stand to gain money: the updated balance is likely not to know about the other transaction, and will thus "forget" one of the deductions. Now I suppose you're going to come and say that this is bad programming, and advisory locking would do the job if the software is written right. Correct. You could also use the same argument to say that memory protection isn't necessary, because a correctly written program doesn't overwrite other processes address space. It's the same thing: file protection belongs in the kernel. Well, I'd say advisory lock does the job if the software is written right, and if the software is not written right, mandatory locking won't help. Let's give an example. You right a program using mandatory locking making access to a file. I write an "incorrect" program accessing that file. I garantee you that the file is going to be screwed up, because I intend to write random output to it as soon as I get access to it. After all, if I'm incorrect, I'm allowed to do anything. -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
Daniel O'Connor wrote: On 23-Aug-99 Greg Lehey wrote: I'm a little surprised that there's any objection to the concept of mandatory locking. In transaction processing, locking is not optional, and if any process at all can access a file or set of files without locking, you can't guarantee the database integrity. Other OSs have used mandatory locking for decades, and System V has it too. So far I haven't seen any arguments, let alone valid ones, against having it in FreeBSD. I think its a good idea, and hey if people object it can always be an option like - option NO_MANDATORY_LOCKING Phew, that was tough. When introducing security holes, the default should be the hole not being present. Ie, reverse that option. -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] - Come on. - Where are we going? - To get what you came for. - What's that? - Me. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Mandatory locking?
When I did a remote geographic disk based mirroring product a few years ago, I just had an ioctl that said that this disk was special for a while. Then the open routine would fail. This flag was cleared in the close routine (and by the companion ioctl). I did allow users to open the device w/o read and write to get status on the device, but that was it until the rebuilding process was complete. Granted, this was the control device for the mirroring driver... But that worked fairly well. I only needed to do this in recovery situations for a few hundred disk I/Os, so the average user would likely have never noticed. Oh, this was on Solaris, but the concepts are exactly the same for FreeBSD. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
