Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Kurt Wall updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Kurt Wall QA Status: Reviewed Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Kenn Hussey updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Kenn Hussey Component/s: Server Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Kenn Hussey commented on an issue Re: Create endpoint for enumerating environments kan rangsan is there a reason why you reopened this? Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Kenn Hussey updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Kenn Hussey Epic Status: Done Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Kenn Hussey updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Kenn Hussey Fix Version/s: 3.6.0 Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andy Parker updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Andy Parker Fix Version/s: 3.6.0 Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Andrew Parker Fix Version/s: 3.5.0 Fix Version/s: 3.6.0 Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker commented on an issue Re: Create endpoint for enumerating environments Eric Sorenson, can you be a bit more specific about the kind of configurability you are saying is needed? Do you mean that every environment needs to be able to have its own, completely unique, set of module directories? Could you give an example of how you envision they might be configured? Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title garrett honeycutt commented on an issue Re: Create endpoint for enumerating environments Every environment should have the ability to have its own modulepath, manifestdir, and hiera configuration. Andrew Parker the module directories do not have to be unique, though they could be. It is very common to have a part of the path be shared and part of it be unique. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Eric Sorenson commented on an issue Re: Create endpoint for enumerating environments Got a +1 from Zachary Smith about a configurable environment-specific module path ; a customer would like a way to build a very specific (and probably long) modulepath for the environments to incorporate modules at particular versions (= subdirectories) for the environments. The model is from Spotify's spikor code. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker commented on an issue Re: Create endpoint for enumerating environments Sean Millichamp thanks for the comments. For 1) I wasn't planning on it, but it should be an easy extension. Joshua Partlow and I were working through that aspect right now. I think all it would take is to have an environmentspath instead of the environmentsdir. For 2) that wasn't in the plans. However, I was thinking that every environment should have some way of having a configuration file. That file would be able to control the environment more fine grained and something like this, where it can specify what the module directories are would fit into it. Another way of handling this would be to incorporate Puppetfiles into the environment. I think that is the right direction, but it is a much larger change. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede)
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Sean Millichamp commented on an issue Re: Create endpoint for enumerating environments So, I know there has been lots of discussion on this, but we have a dynamic environments use case that is perhaps a little more sophisticated than some I've seen discussed out in the wild and I thought I'd share. In both cases the problem can (currently) be solved with a bunch of symlink work in the various check-out and management scripts. Two things I'd love to see are: 1) Specifying a multiple environment base search path (like you can do today with modulepath). We have two distinct types of environment: fully managed via git checkouts (via an internal librarian-puppet/r10k type script) and adhoc (created on a developer whim to quickly mock something up and then be destroyed). In order to accomodate Puppet's dynamic environment implementation today we make them appear at one place via a bunch of symlinks into a common directory but that seems gross and needless and, when developers manually remove their adhoc environment without the toolset we've provided to do so it leaves broken symlinks. Again, we could have a script that looks for and removes them, but an environment base search path seems so much cleaner. 2) The ability not only to specify a module search path external to the environment (e.g.: /opt/puppet/share/puppet/modules) but also a module search path internal to the environment. I mention this because it wasn't clear to me if all of the discussed approaches would support that. This is, of course, possible today but it wasn't clear to me if/how a directory-based environment design would support that. We have two major categories of modules. Our managed modules are designed, reviewed, tested, versioned by the Puppet team. Each gets its own Git repo and has reasonably high standards. The others can be more or less created by our Puppet users (the OS engineers), committed, and become live with no Puppet team involvement - the use case is primarily for deploying things that are one-off and particular for a given customer that we don't intend to reuse OR which have to go from nothing to deployed rapidly. This gives a modulepath of: modulepath=/path/to/environmentbase/$environment/coremodules:/opt/puppet/share/puppet/modules:/path/to/environmentbase/$environment/modules The nice thing with this configuration is that we get to determine what the precedence order is and ensure that the core and PE-provided modules never get accidentally overridden with one of the one-off modules. The symlink approach makes it work, but removes the ability to ensure that at the Puppet layer. As an aside, what I'd really like to be able to do is manipulate the module search path at run time early in the DSL. I know that would likely impact things like pluginsync and so it is non-trivial or near impossible to do (though I'd be okay with losing plugin sync for any path dynamically included). I think I recall seeing an ARM about that which seemed promising. Thanks!
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Eric Sorenson commented on an issue Re: Create endpoint for enumerating environments Michelle Johansen +1 to Andrew Parker's assessment of the issue breakdown – the capability is necessary for PE to enumerate the environments but the icing on the cake in 1119 and 1149 are not hard requirements. Luke Kanies the goal is to reflect the (git+hooks+dynamic environments) pattern in puppet itself; in this pattern the source of truth is the version control system, which puppet can view but does not control; so attempting to be authoritative for would be erroneous in se. I could easily see building and shipping a service which does manage the code branch-to-environment mappings and is responsible for pushing changes to all the masters in an environment and ensuring they're in sync; this endpoint would still be useful as a reality check to ensure the masters reflect the SOT accurately. Until then, this design lets us integrate with whatever home-tooled setup users/PS have built. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede)
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Luke Kanies commented on an issue Re: Create endpoint for enumerating environments Ok, thanks Aaron Armstrong was discussing the building of an environment service that the node classifier could talk to as a source of truth, and it sounds like this won't quite be that (because it would need to talk to each master). I just wanted to clarify. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Luke Kanies commented on an issue Re: Create endpoint for enumerating environments Will this be the single source of truth for environments, or meant more as a means of knowing what a given master sees? It seems difficult to make it a single source of truth, given that any master is likely to have a different list. Could/would we promote this to a separate service, like the node classifier? Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Michelle Johansen commented on an issue Re: Create endpoint for enumerating environments Eric Sorenson or Andrew Parker Can one of you please verify that this was pushed past 3.5.0? Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker commented on an issue Re: Create endpoint for enumerating environments An issue that we encountered while working through this was what to do with one more source of environments. There are currently 3 (yes, 3, not 2) sources for environments: Dynamic via the $environment interpolation Static from environment stanzas in puppet.conf Default from not doing anything beyond the master's default modulepath and manifest settings This last one hadn't been considered before. The plan for it is to deprecate it as well and move everything to being explicit in the environmentsdir. This means that there is no need anymore for the manifest setting and it can be deprecated (PUP-1433) and a basic install of puppet will have a /etc/puppet/environments/production/manifests/site.pp for the production environment. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node Classifier and Continuous Delivery work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Eric Sorenson commented on an issue Re: Create endpoint for enumerating environments One additional requirement that needs consideration: We need to support multi-element modulepaths with this scheme, for users who currently use this pattern: # search environment-specific modules first, then a common modulepath modulepath = /etc/puppet/environments/$environment/modules:/etc/puppet/environments/common/modules Some refinements/complexities on this pattern (not all of which need to port over directly but we need to understand their use cases) are https://projects.puppetlabs.com/issues/11900 (another layer of interpolation in the modulepath elements) PUP-1324 directory globbing in the final modulepath element librarian-puppet / r10k workflows (have not explored this personally but Adrien Thebo says modules are always installed into the modules/ directory, and modules in the git repo itself are in a separate directory Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the Node
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Eric Sorenson updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Eric Sorenson Muchofthe ENC/CD NodeClassifierandContinuousDelivery workispredicatedonimprovedsupportforenvironments,andonesignificantgapistheabsenceofawaytoenumeratetheenvironmentsknowntoapuppetmaster. Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Andrew Parker Issue Type: Story Epic Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker updated an issue Puppet / PUP-536 Create endpoint for enumerating environments Change By: Andrew Parker Epic Name: EnumeratingEnvironments Add Comment This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/groups/opt_out.
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title garrett honeycutt commented on an issue Re: Create endpoint for enumerating environments Awesome work! Suggest that the directory structure look like this environment_name ├── hiera │ └── hiera.yaml ├── manifests └── modules Difference is that some users have multiple manifests, so we should have a manifests directory. Also, they might keep manifests in a VCS and want to clone that repo to a directory. Besides modules and manifests, environments need a way to specify where to find their Hiera configuration (See HI-46). This should also be a directory for the VCS reason above. It might contain hiera data (YAML, JSON, etc) or just the hiera.yaml that points to other backends or storage locations on the filesystem. Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the ENC/CD work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede)
Jira (PUP-536) Create endpoint for enumerating environments
Title: Message Title Andrew Parker commented on an issue Re: Create endpoint for enumerating environments garrett honeycutt, you are absolutely right. I think it should leave out the hiera directory for now since we don't yet support a hiera configuration per environment . There wouldn't be anything stopping users from putting it there, but it just wouldn't mean anything to puppet. I had just been discussing the need for the manifests directory with Eric Sorenson on IRC. We need it because of the work on PUP-865, which is going to be the replacement for import (PUP-866). Add Comment Puppet / PUP-536 Create endpoint for enumerating environments Much of the ENC/CD work is predicated on improved support for environments, and one significant gap is the absence of a way to enumerate the environments known to a puppet master. This message was sent by Atlassian JIRA (v6.1.4#6159-sha1:44eaede) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this
