On Wed, Nov 16, 2016 at 11:26 PM John Meinel <j...@arbash-meinel.com> wrote:
> So we just ran into an issue when you are running multiple units on the
> same machine and one of them is particularly busy.
>
> The specific case is when deploying Openstack and colocating things like
> "monitoring" charms with the "keystone" charm. Keystone itself has *lots* of
> things that relate to it, so it wants to fire something like 50
> relation-joined+changed hooks.
>
> The symptom is that unit-keystone ends up acquiring and re-acquiring the
> uniter hook lock for approximately 50 minutes and starves out all other
> units from coming up, because they can't run any of their hooks.
>
> From what I can tell, on Linux we are using
> net.Listen("abstract-unix-socket") and then polling at a 250ms interval to
> see if we can grab that socket.
>
> However, that means that every process that *doesn't* have the lock has
> an average time of 125ms to wake up and notice that the lock isn't held.
> However, a process that had the lock but has more hooks to fire is just
> going to release the lock, do a bit of logic, and then be ready to acquire
> the lock again, most likely much faster than 125ms.
>
> We *could* introduce some sort of sleep there, to give some other
> processes a chance. And/or use a range of times, instead of a fixed 250ms.
> (If sometimes you sleep for 50ms, etc).
>
> However, if we were using something like 'flock' then it has a blocking
> mode, where it can give you the lock as soon as someone else releases it.
>
> AIUI the only reason we liked abstract-unix-sockets was to not have a file
> on disk, but we had a whole directory on disk, and flock seems like it
> still gives us better sharing primitives than net.Listen.
>
+1 to blocking file lock. We could probably leave Windows alone, and just
do that on *nix.
Thoughts?
> John
> =:->
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev
