Re: Writing workers
This is one of those things that should probably end up on the Wiki. Thanks for writing it up. John =:- On Thu, May 28, 2015 at 5:34 PM, William Reade william.re...@canonical.com wrote: Hi all I've noticed that there's a lot of confusion over how to write a useful worker. Here follow some guidelines that you should be *very* certain of yourself before breaking (and probably talk to me about anyway). If there's any uncertainty about these, I'm more than happy to expand. * If you really just want to run a dumb function on its own goroutine, use worker.NewSimpleWorker. * If you just want to do something every period, use worker.NewPeriodicWorker. * If you want to react to watcher events, you should probably use worker.NewNotifyWorker or worker.NewStringsWorker. * If your worker has any methods outside the Worker interface, DO NOT use any of the above callback-style workers. Those methods, that need to communicate with the main goroutine, *need* to know that goroutine's state, so that they don't just hang forever. * To restate the previous point: basically *never* do a naked channel send/receive. If you're building a structure that makes you think you need them, you're most likely building the wrong structure. * If you're writing a custom worker, and not using a tomb.Tomb, you are almost certainly doing it wrong. Read the blog post [0] or, hell, just read the code [1] -- it's less than 200 lines and it's about 50% comments. * If you're letting tomb.ErrDying leak out of your workers to any clients, you are definitely doing it wrong -- you risk stopping another worker with that same error, which will quite rightly panic (because that tomb is *not* yet dying). * If it's possible for your worker to call .tomb.Done() more than once, or less than once, you are *definitely* doing it very very wrong indeed. * If you're using .tomb.Dead(), you are very probably doing it wrong -- the only reason (that I'm aware of) to select on that .Dead() rather than on .Dying() is to leak inappropriate information to your clients. They don't care if you're dying or dead; they care only that the component is no longer functioning reliably and cannot fulfil their requests. Full stop. Whatever started the component needs to know why it failed, but that parent is usually not the same entity as the client that's calling methods. * If you're using worker/singular, you are quite likely to be doing it wrong, because you've written a worker that breaks when distributed. Things like provisioner and firewaller only work that way because we weren't smart enough to write them better; but you should generally be writing workers that collaborate correctly with themselves, and eschewing the temptation to depend on the funky layer-breaking of singular. * If you're passing a *state.State into your worker, you are almost certainly doing it wrong. The layers go worker-apiserver-state, and any attempt to skip past the apiserver layer should be viewed with *extreme* suspicion. * Don't try to make a worker into a singleton (this isn't particularly related to workers, really, singleton is enough of an antipattern on its own [2] [3] [4]). Singletons are basically the same as global variables, except even worse, and if you try to make them responsible for goroutines they become more horrible still. Did I miss anything major? Probably. If so, please remind me. Cheers William [0] http://blog.labix.org/2011/10/09/death-of-goroutines-under-control [1] launchpad.net/tomb (apparently... we really ought to be using v2, though) [2] https://sites.google.com/site/steveyegge2/singleton-considered-stupid [3] http://jalf.dk/blog/2010/03/singletons-solving-problems-you-didnt-know-you-never-had-since-1995/ [4] http://programmers.stackexchange.com/questions/40373/so-singletons-are-bad-then-what/ -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Writing workers
https://github.com/juju/juju/wiki/Guidelines-for-writing-workers On Mon, Jun 1, 2015 at 3:32 PM, Nate Finch nate.fi...@canonical.com wrote: Totally belongs on the wiki. On Mon, Jun 1, 2015 at 8:45 AM, John Meinel j...@arbash-meinel.com wrote: This is one of those things that should probably end up on the Wiki. Thanks for writing it up. John =:- On Thu, May 28, 2015 at 5:34 PM, William Reade william.re...@canonical.com wrote: Hi all I've noticed that there's a lot of confusion over how to write a useful worker. Here follow some guidelines that you should be *very* certain of yourself before breaking (and probably talk to me about anyway). If there's any uncertainty about these, I'm more than happy to expand. * If you really just want to run a dumb function on its own goroutine, use worker.NewSimpleWorker. * If you just want to do something every period, use worker.NewPeriodicWorker. * If you want to react to watcher events, you should probably use worker.NewNotifyWorker or worker.NewStringsWorker. * If your worker has any methods outside the Worker interface, DO NOT use any of the above callback-style workers. Those methods, that need to communicate with the main goroutine, *need* to know that goroutine's state, so that they don't just hang forever. * To restate the previous point: basically *never* do a naked channel send/receive. If you're building a structure that makes you think you need them, you're most likely building the wrong structure. * If you're writing a custom worker, and not using a tomb.Tomb, you are almost certainly doing it wrong. Read the blog post [0] or, hell, just read the code [1] -- it's less than 200 lines and it's about 50% comments. * If you're letting tomb.ErrDying leak out of your workers to any clients, you are definitely doing it wrong -- you risk stopping another worker with that same error, which will quite rightly panic (because that tomb is *not* yet dying). * If it's possible for your worker to call .tomb.Done() more than once, or less than once, you are *definitely* doing it very very wrong indeed. * If you're using .tomb.Dead(), you are very probably doing it wrong -- the only reason (that I'm aware of) to select on that .Dead() rather than on .Dying() is to leak inappropriate information to your clients. They don't care if you're dying or dead; they care only that the component is no longer functioning reliably and cannot fulfil their requests. Full stop. Whatever started the component needs to know why it failed, but that parent is usually not the same entity as the client that's calling methods. * If you're using worker/singular, you are quite likely to be doing it wrong, because you've written a worker that breaks when distributed. Things like provisioner and firewaller only work that way because we weren't smart enough to write them better; but you should generally be writing workers that collaborate correctly with themselves, and eschewing the temptation to depend on the funky layer-breaking of singular. * If you're passing a *state.State into your worker, you are almost certainly doing it wrong. The layers go worker-apiserver-state, and any attempt to skip past the apiserver layer should be viewed with *extreme* suspicion. * Don't try to make a worker into a singleton (this isn't particularly related to workers, really, singleton is enough of an antipattern on its own [2] [3] [4]). Singletons are basically the same as global variables, except even worse, and if you try to make them responsible for goroutines they become more horrible still. Did I miss anything major? Probably. If so, please remind me. Cheers William [0] http://blog.labix.org/2011/10/09/death-of-goroutines-under-control [1] launchpad.net/tomb (apparently... we really ought to be using v2, though) [2] https://sites.google.com/site/steveyegge2/singleton-considered-stupid [3] http://jalf.dk/blog/2010/03/singletons-solving-problems-you-didnt-know-you-never-had-since-1995/ [4] http://programmers.stackexchange.com/questions/40373/so-singletons-are-bad-then-what/ -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Writing workers
Totally belongs on the wiki. On Mon, Jun 1, 2015 at 8:45 AM, John Meinel j...@arbash-meinel.com wrote: This is one of those things that should probably end up on the Wiki. Thanks for writing it up. John =:- On Thu, May 28, 2015 at 5:34 PM, William Reade william.re...@canonical.com wrote: Hi all I've noticed that there's a lot of confusion over how to write a useful worker. Here follow some guidelines that you should be *very* certain of yourself before breaking (and probably talk to me about anyway). If there's any uncertainty about these, I'm more than happy to expand. * If you really just want to run a dumb function on its own goroutine, use worker.NewSimpleWorker. * If you just want to do something every period, use worker.NewPeriodicWorker. * If you want to react to watcher events, you should probably use worker.NewNotifyWorker or worker.NewStringsWorker. * If your worker has any methods outside the Worker interface, DO NOT use any of the above callback-style workers. Those methods, that need to communicate with the main goroutine, *need* to know that goroutine's state, so that they don't just hang forever. * To restate the previous point: basically *never* do a naked channel send/receive. If you're building a structure that makes you think you need them, you're most likely building the wrong structure. * If you're writing a custom worker, and not using a tomb.Tomb, you are almost certainly doing it wrong. Read the blog post [0] or, hell, just read the code [1] -- it's less than 200 lines and it's about 50% comments. * If you're letting tomb.ErrDying leak out of your workers to any clients, you are definitely doing it wrong -- you risk stopping another worker with that same error, which will quite rightly panic (because that tomb is *not* yet dying). * If it's possible for your worker to call .tomb.Done() more than once, or less than once, you are *definitely* doing it very very wrong indeed. * If you're using .tomb.Dead(), you are very probably doing it wrong -- the only reason (that I'm aware of) to select on that .Dead() rather than on .Dying() is to leak inappropriate information to your clients. They don't care if you're dying or dead; they care only that the component is no longer functioning reliably and cannot fulfil their requests. Full stop. Whatever started the component needs to know why it failed, but that parent is usually not the same entity as the client that's calling methods. * If you're using worker/singular, you are quite likely to be doing it wrong, because you've written a worker that breaks when distributed. Things like provisioner and firewaller only work that way because we weren't smart enough to write them better; but you should generally be writing workers that collaborate correctly with themselves, and eschewing the temptation to depend on the funky layer-breaking of singular. * If you're passing a *state.State into your worker, you are almost certainly doing it wrong. The layers go worker-apiserver-state, and any attempt to skip past the apiserver layer should be viewed with *extreme* suspicion. * Don't try to make a worker into a singleton (this isn't particularly related to workers, really, singleton is enough of an antipattern on its own [2] [3] [4]). Singletons are basically the same as global variables, except even worse, and if you try to make them responsible for goroutines they become more horrible still. Did I miss anything major? Probably. If so, please remind me. Cheers William [0] http://blog.labix.org/2011/10/09/death-of-goroutines-under-control [1] launchpad.net/tomb (apparently... we really ought to be using v2, though) [2] https://sites.google.com/site/steveyegge2/singleton-considered-stupid [3] http://jalf.dk/blog/2010/03/singletons-solving-problems-you-didnt-know-you-never-had-since-1995/ [4] http://programmers.stackexchange.com/questions/40373/so-singletons-are-bad-then-what/ -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Writing workers
Hi all I've noticed that there's a lot of confusion over how to write a useful worker. Here follow some guidelines that you should be *very* certain of yourself before breaking (and probably talk to me about anyway). If there's any uncertainty about these, I'm more than happy to expand. * If you really just want to run a dumb function on its own goroutine, use worker.NewSimpleWorker. * If you just want to do something every period, use worker.NewPeriodicWorker. * If you want to react to watcher events, you should probably use worker.NewNotifyWorker or worker.NewStringsWorker. * If your worker has any methods outside the Worker interface, DO NOT use any of the above callback-style workers. Those methods, that need to communicate with the main goroutine, *need* to know that goroutine's state, so that they don't just hang forever. * To restate the previous point: basically *never* do a naked channel send/receive. If you're building a structure that makes you think you need them, you're most likely building the wrong structure. * If you're writing a custom worker, and not using a tomb.Tomb, you are almost certainly doing it wrong. Read the blog post [0] or, hell, just read the code [1] -- it's less than 200 lines and it's about 50% comments. * If you're letting tomb.ErrDying leak out of your workers to any clients, you are definitely doing it wrong -- you risk stopping another worker with that same error, which will quite rightly panic (because that tomb is *not* yet dying). * If it's possible for your worker to call .tomb.Done() more than once, or less than once, you are *definitely* doing it very very wrong indeed. * If you're using .tomb.Dead(), you are very probably doing it wrong -- the only reason (that I'm aware of) to select on that .Dead() rather than on .Dying() is to leak inappropriate information to your clients. They don't care if you're dying or dead; they care only that the component is no longer functioning reliably and cannot fulfil their requests. Full stop. Whatever started the component needs to know why it failed, but that parent is usually not the same entity as the client that's calling methods. * If you're using worker/singular, you are quite likely to be doing it wrong, because you've written a worker that breaks when distributed. Things like provisioner and firewaller only work that way because we weren't smart enough to write them better; but you should generally be writing workers that collaborate correctly with themselves, and eschewing the temptation to depend on the funky layer-breaking of singular. * If you're passing a *state.State into your worker, you are almost certainly doing it wrong. The layers go worker-apiserver-state, and any attempt to skip past the apiserver layer should be viewed with *extreme* suspicion. * Don't try to make a worker into a singleton (this isn't particularly related to workers, really, singleton is enough of an antipattern on its own [2] [3] [4]). Singletons are basically the same as global variables, except even worse, and if you try to make them responsible for goroutines they become more horrible still. Did I miss anything major? Probably. If so, please remind me. Cheers William [0] http://blog.labix.org/2011/10/09/death-of-goroutines-under-control [1] launchpad.net/tomb (apparently... we really ought to be using v2, though) [2] https://sites.google.com/site/steveyegge2/singleton-considered-stupid [3] http://jalf.dk/blog/2010/03/singletons-solving-problems-you-didnt-know-you-never-had-since-1995/ [4] http://programmers.stackexchange.com/questions/40373/so-singletons-are-bad-then-what/ -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
