I'm wondering if it will be possible to disable this behaviour on resource collections in the terminus? For us puppet environments are mapped to git branches, and actual environments (like testing and prod) have different Puppet CAs and PuppetDBs (it is really best to not mix them anyway). So it is all right if we record the environment (git branch), but we want to collect across all of them. And actually this might be useful during a migration period for others as well, until all nodes have sent in their new catalogs with environment info.
Also, 2.x agents sends their environment as a fact, so for them you could migrate the data. On 28 March 2014 18:46, Ken Barber <k...@puppetlabs.com> wrote: > Hey all, > > TL;DR: We're adding support to environments to PuppetDB but have a > small migration hassle we wanted some community opinion on. If you're > interested in PuppetDB and environments read on. > > So we're looking at adding first class support to PuppetDB for > environments. In the past we would happily accept facts/reports & > catalogs from multiple different environments, but that information > was never stored with the data. > > As a consequence people trying to use the same PuppetDB instance for > environments would find they are collecting globally across all > environments thus creating a high chance of resource collision. For > many people this just wouldn't work, but for some they probably found > horrible ways of working around this (if you are one of these people, > I'd probably want to hear from you). > > Not to mention we also have no way to represent this data in the PE > console either, basically PuppetDB just has no environment awareness. > > So we're fixing this now (at least the PDB part). Our current Epic: > https://tickets.puppetlabs.com/browse/PDB-47 and relevant child > tickets covers this work if you want to follow along at home. > > The issue we've hit however is how to cleanly migrate from a world of > no environments, to one where environments are going to be > constraining collections. Let me break it down: > > * Currently all submissions to PDB have no environment knowledge, and > we can't store it anyway. > * During an upgrade to this new feature we have to set the environment > of all items in our database to 'something' internally and externally, > however without prior knowledge everything will be labelled the same. > * Once environment awareness is added to the terminus we only want to > collect on environments the agent transaction was run with, this > complies with the semantics of collections going back to whenever > storeconfigs & environments were added to Puppet. > > One strategy for upgrades is to set the environment to 'production' > for existing data. This will stay like this until another catalog is > submitted with the true environment for that node. > > Now the problem with that solution is related to people who might be > using exported resources (and somehow avoiding collection collisions): > > * In a single environment setup, where the name of the environment is > not 'production'. > * In a multi-environment setup where environment names are quite > different (but only 1 is production). > > For both of these cases, if we just default the environment to > 'production' there will be a short time where collections will return > nothing for environments not called 'production' - until the next > catalog submission occurs. This could be detrimental, and we'd like to > understand how many users this might impact. Please let me know if > this is you. > > An alternate solution is to migrate existing data to have the > environment set to something completely different (like 'nil' or > something else that can't possibly collide with a real name). With > this solution, we can put in a collection query that not only collects > for the current environment, but also for 'nil' to pick up the older > global resources ... until such time as all catalogs have submitted > thus putting the catalogs in the correct environment. > > This concept of 'nil' is almost transient (although we be longer lived > for old reports obviously) its a temporary marker to say 'we don't > know the environment'. So in most cases, once all catalogs have been > submitted by all nodes the concept disappears. No new data should be > added to this internal special environment. > > Anyway - I'm looking for some feedback for these two alternate > solutions (or a third more whacky solution - whatever :-). The first > one is obviously easiest and avoids leaving behind a transient > solution, the second we think solves this concern to some extent (at > least we think so). > > Any feedback or opinion on this would be much appreciated. > > ken. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CAE4bNT%3DwHZtttzvAuL7qYX_6Ao%2BJuPggY%2BosgwP1%2BB5%2B7QhOJA%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- Erik Dalén -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLd3ULjVxcbJW3OhHdD%2B_WKqnnjhBd0%2Bao7-Fafbv%3Dm7EQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
