On Sun, Jun 30, 2013 at 11:53 PM, Luke Kanies <[email protected]> wrote:
> On Jun 30, 2013, at 2:06 PM, Evelio VILA <[email protected]> wrote: > > >> We've found that many end up resorting to using inline templates to do >> many things that are not related to generating text, which is what a >> template is intended to be. Have you not found that to be the case for you? >> > > i dont remember using inline template as a last resort solution. it is > true though that its a pretty useful hack. every time i am confronted > with a need for a loop, i use create_resources. if the problem reveals to > be more complicated than that, then we turn to other types of > solutions(functions, providers, etc) > > i think that that feature is a drift on puppet 'fundamentals'. > > on the other hand, what we would really love to see: > > a serious increase on performance. simply adding more resources to the > master or installing more masters when the number of clients grow is > clearly not a scalable solution. imho there are lots of work to be done > here. (even if ruby has its limitations on this matter) > > > Hi, > > We're definitely doing this. You should have seen significant performance > differences in Puppet 3.2 and now PE 3.0, and we now have consistent, > repeatable performance tests baked into our CI. We're in the process of > building out a dedicated performance team to really focus on this, too. > > Yes, puppet 3 has show significant improvements of 2, but we can always get better. Much of the effort here needs to be on larger structural changes, since we are not seeing large gains from any micro optimizations. Some of these structure things was what Luke mentions a little later about changing the fundamental architecture of puppet to be queue based. Other changes will take place as nearly complete rewrites of sections to clarify the code and straighten out convoluted code paths, which is partly being done for maintenance and ongoing feature development reasons, but also almost always have performance improvements when performance is kept in mind while doing it. > We also recognize the pain of things like scaling, so while we're trying > to reduce the need to scale, we're also working on making that scaling > itself easier. This varies from things like making it easier to get a > master up and running to configuring the various masters to provide > different services and connecting them all with a message bus. > > So yeah, we've heard this complaint loud and clear, and we are working on > it. And I'm happy to say you're already seeing the benefits, so you don't > just have to trust us. > > a production ready 'environment' feature: we have found from our own > experience and also on irc that the risk of 'leak' between environments is > so high that makes it unusable on production. > > > This is fundamentally hard, and I wish we had a better answer here. The > closest we can get today is to use jruby and thus separate Java processes > for each environment, or segregate the environments to different masters. > As long as we stick to ruby, they are fundamentally stuck. > > Just to be clear, if we stick to MRI ruby, the solution to this is multiple processes (because we need a process boundary to get isolation). In Java it can call be in the same JVM process because with JRuby we can rely on multiple instance of JRuby that will stay isolated and can get more isolation from Java classloaders. > We're working on this, but maybe not as fast as we should be. > > more emphasis on security. not long ago we were migrating a master to a > PCI dss environment and i recall making a question on irc about this > subject. to my surprise, i learned that the master relies on a fact for > client auth and not on the dn of the certificate!(or maybe i just got it > all wrong). anyway, my point here is that puppet stills some work in this > area. > > > I seem to remember this conversation, but I unfortunately don't know the > details so I can't speak to specifics. > > We take security very seriously, so when this kind of thing shows up we > work hard to fix it quickly. Hopefully Andy will step in with info on what > the current status is on this. > > I would be very concerned if the security system of puppet were based on untrusted information from the agent. Unless we have a bug in this system, which we don't as far as I know, then the security is all based on the CN of the certificate subject. Now, you can write your manifests to rely on a fact, which cannot be trusted and that could be a problem. Short of signing all facts (which I'm not sure how to do in the face of a fact value changing) facts cannot be trusted. There are some that have been floating around at Puppet Labs around getting trusted data into the manifests via the certificate. So the short is: we should be using certificate information for security. This is untrusted data in manifests and so care must still be taken. > testing testing and more testing. we need tools to test the catalogs > locally. (a coworker works on a tool to addr this issue > http://lpuppet.banquise.net) people are forced to use all sorts of hacks > in the toolchain like apply the catalogs on a test server to check that > nothing broke , and a bunch of other similar solutions. > > > Sorry, I'm not sure what you mean. We've seen more and more testing tools > show up, like rspec-puppet, and while we know more would always be good, we > keep hearing from customers that there's no substitute for running in vivo. > One of our major goals right now is making continuous delivery more easily > attainable, which is all about being able to test changes on the way, but a > lot of the work we're see now is either integrating existing tools into a > CI system, or providing better visibility and control as the tools roll out. > > What kind of testing would you like to be able to do? > > we would really love to see puppet grow, but on the number of features ;) > > > I think in general that's been our focus. The work done to create this > new parser is a small amount of the work going into the overall system, and > the iteration work on the new parser is an even smaller fraction. > > The fact is that Puppet's whole parser was written by me, years ago, and > it was by about 10x the most complicated parser I'd ever written. Henrick, > who wrote this new parser, has been writing parsers for decades, and he's > written a much better and more maintainable parser than we have right now. > > While he was at it, he threw iteration into the mix, but the parser work > has its own merits. This should increase consistency, reduce bugs, > increase fix and feature velocity, and make it easier to provide great > tooling around the language. All of that is exactly what you want. > > It just happens to almost trivially enable features that were hard before, > and sometimes devs left to their own devices just add things our users are > asking for all the time. :) I think that's awesome, and I think in this > case, this feature isn't distracting from the other hard work being done. > > I agree that the core sustaining engineering work is critical, but we do > actually have users asking for new features all the time, and we can't > ignore them or we'll look up in three years and everyone will be using > SaltStack or Ansible. We're just going to have to do both. > > -- > Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | > +1-615-594-8199 > Join us at PuppetConf 2013, August 22-23 in San Francisco - > http://bit.ly/pupconf13 > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-dev. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- Andrew Parker [email protected] Freenode: zaphod42 Twitter: @aparker42 Software Developer *Join us at PuppetConf 2013, August 22-23 in San Francisco - * http://bit.ly/pupconf13* **Register now and take advantage of the Early Bird discount - save 25%!* -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev. For more options, visit https://groups.google.com/groups/opt_out.
