More information along these lines, highlighting ease of use and tools
for users to see their catalogs, will go along way towards soothing us
touchy sysadmins. The performance gains while nice don't have the appeal
of better troubleshooting. I'm happy to learn yet another stack, but I'd
like to be sure I'm getting some thing more than the status quo.
The information around tuning Passenger/Puppet explicitly provided by
Puppet Labs was mostly crap. It would be extremely useful for everyone
if there were 4-8 pages of serious and indepth docs specifically about
running puppet_server on the JVM. If that doesn't happen, you'll be
fighting the supposed poor performance of every un-tuned puppet_server
installation for years.
Ramin
On 9/27/14 2:23 PM, Ken Barber wrote:
(1) at my current shop, there's an immense hatred of everything JVM. That's
going to be a hard transition. Not to mention Puppet is the only place we
run Ruby, so it's nice and easy to let puppet do whatever it wants with Ruby.
Not so much for installing JVMs that may break production (improperly
configured and installed, I'll grant) applications.
And rightly so - its had a bad history, but I must argue that largely
my hatred of JVM in the past wasn't the JVM per se, it was the
applications written for it. But I would gladly blame the JVM most
times. Also - most of the hatred I see in the industry is a lack of
understanding around the JVM. For me, I'm an old Perl programmer and
certainly making the transition over the last ~17 years was one I
fought against, more because of my own stubbornness I guess. But once
I started to actually study and learn about the tooling for JVM and
accepts its place in the application stack instead of just hating it,
my attitude began to change.
For example, I could never have understood memory usage in PuppetDB if
it was written in Ruby - never is probably too strong - but its hard
in Ruby to do this ... I have tried and it kind of sucks. But hey,
with clojure/jvm, I can use YourKit which gives me an almost
ludicrously simple way of seeing the memory flow. Point in case, we
used to use the urlencoded way of doing POST submissions for commands,
but when I analyzed command submission in Yourkit (live service mind
you) I quickly realized we had 2 objects, the encoded one and the
unencoded. Just think about that for a second - 2 copies of a very
large catalog in memory ... very wasteful :-). So yeah, we stopped
encoding, it wasn't needed anyway and halved our memory consumption
for command submissions and removed that processing need completely -
again thanks to JVM tooling. This work took at best a day or two,
including the patch I believe.
Same again for queries, we switched to streaming for this same reason
... versus loading up the answer and serving it all in one go ... we
now open a cursor on the db, and as answers come back we stream it via
HTTP. The Java core libraries and Clojure in particular are actually
very very good at doing streaming ... and on our platform streaming
becomes critical to reducing memory usage.
For me, I would only see the Erlang runtime coming close to this as a
serious contender (and perhaps the .Net framework/CLR might have
something here, but this isn't my area of expertise), and while the
tooling there for Erlang is pretty awesome, its not as evolved as the
JVM stuff. Don't get me wrong, I love Erlang too :-).
(2) I've gotta say, I'll really miss dropping log statements directly in the
puppet source when something seems wonky (and not having to compile
something).
Our answer to this for Clojure is usually a combination of NREPL and
(log/spy <original item you want to see>) from the
clojure.tools.logging library or #spy/d statements from the spy scope
library. Works great, and can wrap just about any variable as a nice
piece of magic to drop "debug" statements.
The nice thing here that we didn't have in Ruby is that NREPL allows
changes to a running service. So no need to stop/start the service to
see your debug lines.
I do this quite often for PuppetDB while developing, that is I have a
running PuppetDB instance and the PDB source code open in Emacs (with
the cider plugin for nrepl support already bootstrapped of course) ...
I modify the code ... save it ... hit Ctrl-C Ctrl-K ... and I see the
debug lines start to appear in the log. Its a far more rapid workflow
(to be clear: Emacs is only my choice, I believe there is NREPL
support in vim, eclipse, intellij and various other editors as well).
Oh yeah, and this can be done on real running systems also it doesn't
just have to be a dev workflow, you just need to have the NREPL port
exposed in your PDB config.ini:
https://docs.puppetlabs.com/puppetdb/2.2/configure.html#repl-settings.
ken.
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/5427AD5B.3060203%40badapple.net.
For more options, visit https://groups.google.com/d/optout.
