On Sun, Jun 6, 2010 at 3:12 PM, Luke Kanies <[email protected]> wrote:
> On Jun 2, 2010, at 6:54 PM, Trevor Vaughan wrote: > > -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> I've been following this thread with interest and I think that Donavan >> is hitting upon something that I've also been wanting. >> >> However, the way I was looking at it was as a set of atomic, optionally >> blocking, semaphores in a set of parallel threads. >> >> If you look at each puppet client as an individual thread and realize >> that you need things to happen cross-client that depend on the state of >> one or more clients, then you've deconstructed this into a classic >> parallel programming application (with all the constituent nonsense). >> >> I was toying with the idea of a, for lack of a better term, registry >> where you could have your modules place *any* type of data and other >> systems would retrieve that data/data sets when necessary. >> >> I would *not* build this into the puppetmaster for scaling reasons but >> would instead make it a separate data service perhaps optionally backed >> by a distributed database. >> >> This scenario works both for the OpenLDAP situation below as well as the >> situation where you can't start a service/don't want to apply part of a >> manifest, until something has happened on a different system. >> >> OpenLDAP Example: >> >> Node A - >> - Request Lock with data broker >> - Obtain lock >> Node B - >> - Request Lock with data broker >> - Sleep or skip on returned block >> Node A - >> - Obtain RID list >> - RID +1 >> - Register new RID with data broker >> - Release Lock >> Node B - >> - Get notified that lock is available if didn't skip >> - Continue with processing .... >> >> It's a complex situation, but I'm not entirely sure how else to do it >> without a kludgy web "service" or the like. If you could, perhaps, use >> some of the recent NoSQL abstractions then this may be a reasonably fast >> operation. >> >> Of course, not all data would need to be locked, if you're just reading >> data, then there's no need to lock at all. But, in my opinion, this is >> fundamentally cross-system parallel programming at its best and I think >> that the existing techniques for dealing with the problem would be best >> suited to the task. >> >> I do see a growing trend in this thread and others that, no matter what >> you choose, someone is going to need something else. Such is the nature >> of the vast array of data that we have to pull from. >> >> For example, Person A with 500 servers will be happy with some GUI >> wrapped around YAML. However, Person B with 5000 servers will find the >> same solution to be tedious and slow and will want a full-on database. >> >> Hope this helps and doesn't just make the whole thing more complicated. >> > > This is a lot like what I tried to do with my remote-resource stuff: > > http://github.com/reductivelabs/puppet-external-resource > > I didn't get much response to it when I wrote it a bit ago, but you could > entirely use it to create locks to do staged turn-ups and turn-downs. > > Interesting. That looks like it could be done with the tries/try_sleep patch for exec I just put through too. -- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
