[Puppet-dev] Re: Dealing with transitional states in Puppet
On Friday, December 19, 2014 3:14:23 PM UTC-6, Reid Vandewiele wrote: This thread is introducing a simple workaround for an observed limitation in Puppet's ability to automate inelegant but real configuration requirements. The desired outcome is to get feedback on the suitability of the workaround or its approach, how well it fits with Puppet's paradigm, and whether or not people would find it acceptable to implement this kind of pattern in their own environments. tl;dr: What do people think of https://forge.puppetlabs.com/puppetlabs/transition ? The Problem: Puppet has always been tightly focused on a single, final target state [...], but there are some situations where it isn't quite enough. For example, if a running service locks a file (Windows often does this), that file cannot be changed unless the service is stopped. Procedurally, to edit the file one would be expected to stop the service, make the change to the file, and then start the service back up. I agree that this is a longstanding issue, with no particularly good solution up to now: write a full-blown type and/or provider is not a realistic recommendation for most people and most requirements. I'd go so far as to say that it is an *inappropriate* solution in some cases. What if we had the ability to model transitional states as part of the catalog? [...] Does this pattern or capability make sense in the general context of Puppet? Is this a decent interim solution for something better currently under development? What do people think of this? I went looking for holes to poke in this approach, and didn't find any. I like that it builds on Puppet's core concepts of resources and state, and that it appears to be general enough to be adapted to a wide variety of situations. Having read the docs but not examined the code, I am curious about management of the Transition type's own state. How does it look ahead to determine whether it is already in sync? I guess it invokes 'in_sync?' on each of the 'prior_to' resources, or something like that? Also, I presume a Transition resource fails if it is not initially in sync and it cannot apply the transitional state it describes. Is that correct? And does the transitional state copy/inherit anything from the target final state given for the named resource, or does it have only the (explicit) attributes specified in the Transition resource? I'm also curious about the nature of some of the documented limitations. With regard to the transitioned resource being of native type, don't defined type instances have a representation in the catalog? Classes do. For that matter, are classes excluded from being the transitioned resource? The more I think about it, the more I foresee potential for difficulties if defined type instances or classes were eligible for transitioning. On the other hand, I'd be open to the argument that it's ok to offer additional capability to manifest writers in exchange for opening (more) possibilities for shooting themselves in the foot. With regard to 'noop' parameters, is it your thinking that the transition should not be performed if all the 'prior_to' resources have 'noop' enabled? What about the transitioned resource? I'd be inclined to say that the Transition resource *shouldn't* look at 'prior_to' resources' 'noop' parameters. If 'noop' is being applied on a per-resource basis, then the responsibility should be on the manifest developer to apply 'noop' to the Transition resource where needed. On the other hand, I think you should consider whether the transitional state should automatically be marked with the same 'noop' value as the final state of the transitioned resource, at least by default. I haven't reached a conclusion on that myself, but it seems more likely to be appropriate than basing the Transition's noop on the 'prior_to' resources'. I also have to ask: will this work with Puppet 4? All my questions notwithstanding, I think this is a really cool and useful idea. Thanks! John -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/e8eba24f-4424-4eff-b533-ce99f93892a8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet-dev] Re: Dealing with transitional states in Puppet
On Mon, Dec 22, 2014 at 9:11 AM, John Bollinger john.bollin...@stjude.org wrote: [...] I went looking for holes to poke in this approach, and didn't find any. I like that it builds on Puppet's core concepts of resources and state, and that it appears to be general enough to be adapted to a wide variety of situations. Having read the docs but not examined the code, I am curious about management of the Transition type's own state. How does it look ahead to determine whether it is already in sync? I guess it invokes 'in_sync?' on each of the 'prior_to' resources, or something like that? Also, I presume a Transition resource fails if it is not initially in sync and it cannot apply the transitional state it describes. Is that correct? Effectively, yes. For each prior_to resource the provider will retrieve it from the catalog, call insync?() against each of its properties, and if any are found that are not in sync that triggers the transition. For specifics see https://github.com/puppetlabs/puppetlabs-transition/blob/master/lib/puppet/provider/transition/ruby.rb#L49-L66 (about 16 lines). And does the transitional state copy/inherit anything from the target final state given for the named resource, or does it have only the (explicit) attributes specified in the Transition resource? Yes. Right now all attribute defaults in the transition resource are inherited from the target final state resource. Now that you mention it I can imagine that this might not always be the desired behavior, so it could make sense to introduce a parameter that affected whether or not the transitional state had this implicit relationship to the target state. The current behavior was mostly chosen for convenience. I'd be curious to think through a specific scenario where the difference was important. One doesn't spring to mind immediately for me. Obviously it's possible already to be explicit about many/all attributes in the transition declaration. I'm also curious about the nature of some of the documented limitations. With regard to the transitioned resource being of native type, don't defined type instances have a representation in the catalog? Classes do. For that matter, are classes excluded from being the transitioned resource? The more I think about it, the more I foresee potential for difficulties if defined type instances or classes were eligible for transitioning. On the other hand, I'd be open to the argument that it's ok to offer additional capability to manifest writers in exchange for opening (more) possibilities for shooting themselves in the foot. Allowing defined types or classes to be eligible for transitioning certainly makes things more complicated. The problem is that defined types and classes could have logic in Puppet code that dictates which resources to create and add to the graph. If statements, case statements, variables - the provider doesn't have access to any of that Puppet code. If a property or parameter on a defined type or a class is set to a different value in Puppet code, it's possible that an entirely different set of resources could be created. This is the root of the native type only limitation. At the time transition operates, none of the Puppet code exists anymore, only the raw catalog itself. To the best of my knowledge, the graph-level representation of a class is a pair of whits with relationships to each resource from the class. I've been imagining defined types to be the same. This is likely enough to identify which resources came from a particular class or defined type, but I suspect it isn't enough to confidently change a defined type or class parameter. I would appreciate a check though on that from someone more familiar with how the catalog is put together and what information is available. With regard to 'noop' parameters, is it your thinking that the transition should not be performed if all the 'prior_to' resources have 'noop' enabled? What about the transitioned resource? I'd be inclined to say that the Transition resource *shouldn't* look at 'prior_to' resources' 'noop' parameters. If 'noop' is being applied on a per-resource basis, then the responsibility should be on the manifest developer to apply 'noop' to the Transition resource where needed. On the other hand, I think you should consider whether the transitional state should automatically be marked with the same 'noop' value as the final state of the transitioned resource, at least by default. I haven't reached a conclusion on that myself, but it seems more likely to be appropriate than basing the Transition's noop on the 'prior_to' resources'. I think marking the transitional state with the target resource's noop value as default would be the way to go. This is not a design limitation, just work that has yet to be done. I also have to ask: will this work with Puppet 4? It should absolutely work with Puppet 4! :-) If it doesn't I think it'll be because of
[Puppet-dev] Anyone scripting around certificate authority? (puppet-dev version)
[ sorry for the double-post, I sent this to puppet-users as well, but am posting separately here to keep the threading separate.. Damn reply-to munging ] Hiya, one of the cool things in the new Puppet Server is a re-implementation of Puppet's certificate authority code. The implementation up to last week's 1.0.0 release is pretty strictly backwards-compatible with the Ruby implementation, using the same filesystem layout, same HTTP endpoints, etc., but early next year we need to start making some changes and I wanted to solicit some feedback to see what y'all are using. So, some questions: - Are you using scripts which run and parse output from `puppet cert`, `puppet certificate`, `puppet ca`, `puppet certificate_request` and/or `puppet certificate_revocation_list`? If so, what do the scripts do with the commands, and what output do they expect? (As an aside one of the problems we're aiming to fix is the multiplicity of confusingly overlapping functionality available in these subcommands) - Are you using the HTTP API around certificates in your own tooling/automation? These are endpoints like `/certificate/ca`, `/certificate/some host name`, `/environment/certificate_revocation_list/ca` , `/environment/certificate_request/`, `/environment/certificate_status` Same question -- what do you use the endpoints to accomplish, and are there particularly important pieces of data in the output for your use-cases? - Are you using any programs which load the Puppet Ruby code as a library in order to make use of the certificate-related classes/methods directly? Is that because there was something you couldn't do through the command-line or REST APIs? I would be pretty surprised if anyone was doing this but you're going to have to make the deepest changes so it's important for me to understand what you're relying on. - Are you making use of stuff that lives in the CA filesystem in your own tooling, that does NOT go through any of the Puppet APIs? If so, STOP DOING THAT! Just kidding, sorta. But it would be very interesting to know whether you're using things like the `serial` or `inventory.txt` files in your scripts or workflows. Feel free to follow-up here or on https://tickets.puppetlabs.com/browse/SERVER-270 Eric Sorenson - eric.soren...@puppetlabs.com - freenode #puppet: eric0 puppet platform // coffee // techno // bicycles -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/50D1D662-A11B-4CA6-8A63-0E7240C561B1%40puppetlabs.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet-dev] Anyone scripting around certificate authority? (puppet-dev version)
I can reply to all these questions I think. But will be a few days before I can write up all the details. Christmas celebrations and stuff :) On Mon, 22 Dec 2014 21:01 Eric Sorenson eric.soren...@puppetlabs.com wrote: [ sorry for the double-post, I sent this to puppet-users as well, but am posting separately here to keep the threading separate.. Damn reply-to munging ] Hiya, one of the cool things in the new Puppet Server is a re-implementation of Puppet's certificate authority code. The implementation up to last week's 1.0.0 release is pretty strictly backwards-compatible with the Ruby implementation, using the same filesystem layout, same HTTP endpoints, etc., but early next year we need to start making some changes and I wanted to solicit some feedback to see what y'all are using. So, some questions: - Are you using scripts which run and parse output from `puppet cert`, `puppet certificate`, `puppet ca`, `puppet certificate_request` and/or `puppet certificate_revocation_list`? If so, what do the scripts do with the commands, and what output do they expect? (As an aside one of the problems we're aiming to fix is the multiplicity of confusingly overlapping functionality available in these subcommands) - Are you using the HTTP API around certificates in your own tooling/automation? These are endpoints like `/certificate/ca`, `/certificate/some host name`, `/environment/certificate_revocation_list/ca` , `/environment/certificate_request/`, `/environment/certificate_status` Same question -- what do you use the endpoints to accomplish, and are there particularly important pieces of data in the output for your use-cases? - Are you using any programs which load the Puppet Ruby code as a library in order to make use of the certificate-related classes/methods directly? Is that because there was something you couldn't do through the command-line or REST APIs? I would be pretty surprised if anyone was doing this but you're going to have to make the deepest changes so it's important for me to understand what you're relying on. - Are you making use of stuff that lives in the CA filesystem in your own tooling, that does NOT go through any of the Puppet APIs? If so, STOP DOING THAT! Just kidding, sorta. But it would be very interesting to know whether you're using things like the `serial` or `inventory.txt` files in your scripts or workflows. Feel free to follow-up here or on https://tickets.puppetlabs.com/browse/SERVER-270 Eric Sorenson - eric.soren...@puppetlabs.com - freenode #puppet: eric0 puppet platform // coffee // techno // bicycles -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/50D1D662-A11B-4CA6-8A63-0E7240C561B1%40puppetlabs.com https://groups.google.com/d/msgid/puppet-dev/50D1D662-A11B-4CA6-8A63-0E7240C561B1%40puppetlabs.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLeUByLR2peA_oCVvZaY%3DeY0Sng6H3AxCLT%3DU37TKJShiA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet-dev] Re: Dealing with transitional states in Puppet
On 12/22/2014 07:55 PM, Reid Vandewiele wrote: And does the transitional state copy/inherit anything from the target final state given for the named resource, or does it have only the (explicit) attributes specified in the Transition resource? Yes. Right now all attribute defaults in the transition resource are inherited from the target final state resource. Now that you mention it I can imagine that this might not always be the desired behavior, so it could make sense to introduce a parameter that affected whether or not the transitional state had this implicit relationship to the target state. The current behavior was mostly chosen for convenience. I'd be curious to think through a specific scenario where the difference was important. One doesn't spring to mind immediately for me. Obviously it's possible already to be explicit about many/all attributes in the transition declaration. I would prefer for the transition to operate only on the properties that are specified. Using your original example, it would be very confusing for the user to observe the transition manage the enable property of Service[myapp]. I have not looked at your code, so I don't know about the specifics and how feasible this behavior is, but this would be my ideal design. Speaking of properties - transitions for classes and defines don't seem to make sense to me, because they don't have properties, only parameters. Sure, the compiler could try and figure out what property values are influenced by a given parameter, but that seems very complex to me, both in terms of implementation and user experience. Generally, I will join the chorus - this feature shows amazing promise! Thanks, Felix -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/5498C462.3010603%40Alumni.TU-Berlin.de. For more options, visit https://groups.google.com/d/optout.
