On Tue, Dec 1, 2015 at 9:07 AM David Cheney <[email protected]> wrote:
> http://0pointer.de/blog/projects/locking.html > > In short, opening the same file twice and asserting a lock on it will > succeed. > Thanks. The article is a bit exasperated. Yes, there are problems to be aware of, but it doesn't make them unusable in all cases. - Juju agents don't get installed onto NFS file systems, so doesn't matter for the agents. - We're in full control of the files we're locking, we're not locking some file like /etc/passwd where some other random bit of code in the process is going to open/close it and release the lock by accident. - We don't need byte-range locking. So only the original uncertainty remains: do we care about clients running their home directory on an NFS share, where the NFS *server* is too old to support flock? Maybe a named mutex on Windows and a domain socket on *NIX is the way to go. I'm not dead set on flock; I just want something that is simple, robust and portable. Cheers, Andrew On Tue, Dec 1, 2015 at 12:04 PM, Andrew Wilkins > <[email protected]> wrote: > > On Tue, Dec 1, 2015 at 8:57 AM David Cheney <[email protected]> > > wrote: > >> > >> fcntl won't work in threaded go applications, it barely works in non > >> threaded applications. > > > > > > This is news to me. I've used it plenty in the past, in multi-threaded > > programs. Please point me at relevant literature that explains where it > > doesn't work. > > > >> > >> I'm not interested in "doesn't work on windows" arguments. Yes, we > >> have to support windows, but that doesn't mean we have to be dragged > >> down to it's lowest common denominator. > > > > > > Agreed, if we're actually crippling anything. > > > >> > >> I think it's fine to develop a lock type that does the best available > >> for each platform using conditional compilation. > > > > > > Again, agreed, but only if there's something to be gained by doing this. > I'm > > still not convinced there is. > > > >> On Tue, Dec 1, 2015 at 11:47 AM, Andrew Wilkins > >> <[email protected]> wrote: > >> > On Tue, Dec 1, 2015 at 8:03 AM David Cheney < > [email protected]> > >> > wrote: > >> >> > >> >> On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins > >> >> <[email protected]> wrote: > >> >> > On Tue, Dec 1, 2015 at 7:57 AM David Cheney > >> >> > <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> Doesn't look like there is windows support, and it uses fcntl > >> >> >> (flock) > >> >> >> under the hood, which is what we have now. > >> >> > > >> >> > > >> >> > flock isn't the problematic thing Tim is talking about. > utils/fslock > >> >> > attempts to create a directory in a known location, and if it > >> >> > succeeds, > >> >> > it > >> >> > "holds the lock". Unlocking means removing the directory. > >> >> > >> >> The problem is if the process dies/exits/goes mental while holding > the > >> >> lock we get into this existential gridlock where we're not sure if > the > >> >> process _is_ alive, so we shouldn't remove the lock, or the process > is > >> >> dead, so we should remove the lock. > >> >> > >> >> abstract unix domain sockets do not have this problem on windows; > kill > >> >> the process, file is removed from the abstract namespace. > >> > > >> > > >> > POSIX advisory file locks (flock) do not have this problem either. > See: > >> > man(2) fcntl, section "Advisory record locking". When the file > >> > descriptor is > >> > closed, the lock is released; file descriptors are closed when the > >> > process > >> > dies. > >> > > >> > We could build a mutex on top of a unix domain socket, but then we'll > >> > have > >> > something completely separate for Windows. Shared named mutex? I'm not > >> > convinced the overall solution would be any more robust, and I'm > pretty > >> > sure > >> > it's going to be more complicated. Happy to be proven wrong though. > >> > > >> >> > > >> >> > We would have to contribute Windows support, but it's not hard, > I've > >> >> > done it > >> >> > before. > >> >> > > >> >> >> > >> >> >> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall > >> >> >> <[email protected]> wrote: > >> >> >> > How about github.com/camlistore/lock ? > >> >> >> > > >> >> >> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey > >> >> >> > <[email protected]> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> Hi folks, > >> >> >> >> > >> >> >> >> The fslock was a mistake that I added to the codebase some time > >> >> >> >> back. > >> >> >> >> It > >> >> >> >> provided an overly simplistic solution to a more complex > problem. > >> >> >> >> > >> >> >> >> Really the filesystem shouldn't be used as a locking mechanism. > >> >> >> >> > >> >> >> >> Most of the code that exists for the fslock now is working > around > >> >> >> >> its > >> >> >> >> deficiencies. Instead we should be looking for a better > >> >> >> >> replacement. > >> >> >> >> > >> >> >> >> Some "features" that were added to fslock were added to work > >> >> >> >> around > >> >> >> >> the > >> >> >> >> issue that the lock did not die with the process that created > it, > >> >> >> >> so > >> >> >> >> some mechanism was needed to determine whether the lock should > be > >> >> >> >> broken > >> >> >> >> or not. > >> >> >> >> > >> >> >> >> What we really need is a good OS agnostic abstraction that > >> >> >> >> provides > >> >> >> >> the > >> >> >> >> ability to create a "named" lock, acquire the lock, release the > >> >> >> >> lock, > >> >> >> >> and make sure that the lock dies when the process dies, so > >> >> >> >> another > >> >> >> >> process that is waiting can acquire the lock. This way no > >> >> >> >> "BreakLock" > >> >> >> >> functionality is required, nor do we need to try and do think > >> >> >> >> like > >> >> >> >> remember which process owns the lock. > >> >> >> >> > >> >> >> >> So... > >> >> >> >> > >> >> >> >> We have three current operating systems we need to support: > >> >> >> >> > >> >> >> >> Linux - Ubuntu and CentOS > >> >> >> >> MacOS - client only - bit the CLI uses a lock for the local > cache > >> >> >> >> Windows > >> >> >> >> > >> >> >> >> For Linux, and possibly MacOS, flock is a possibility, but can > we > >> >> >> >> do > >> >> >> >> better? Is there something that is better suited? > >> >> >> >> > >> >> >> >> For Windows, while you can create global semaphores or mutex > >> >> >> >> instances, > >> >> >> >> I'm not sure of entities that die with the process. Can people > >> >> >> >> recommend > >> >> >> >> solutions? > >> >> >> >> > >> >> >> >> Cheers, > >> >> >> >> Tim > >> >> >> >> > >> >> >> >> -- > >> >> >> >> Juju-dev mailing list > >> >> >> >> [email protected] > >> >> >> >> Modify settings or unsubscribe at: > >> >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > Juju-dev mailing list > >> >> >> > [email protected] > >> >> >> > Modify settings or unsubscribe at: > >> >> >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev > >> >> >> > > >> >> >> > >> >> >> -- > >> >> >> Juju-dev mailing list > >> >> >> [email protected] > >> >> >> Modify settings or unsubscribe at: > >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev >
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
