My motivating use case was generating configs for aurproxy ( https://github.com/tellapart/aurproxy), which takes all of its configuration as commandline parameters. It would be possible to do most of it with pystachio, but I frequently feel like I'm fighting against pystachio rather than working with a helpful tool. With jsonnet I have a purely functional templating language where I can build concise and modular interfaces that yield pretty elaborate aurora jobs and load balancer configs. The output is guaranteed to be deterministic, as since it doesn't offer the temptation of adding terrible hacks (talking to the network, spawning subprocess, writing to disk, who knows) that Python allows. Ultimately I'm finding that it is easier to reason about what's happening without the syntactic ambiguity of which parts of a configuration get bound when the template is loaded vs at job runtime.
My justification ends up being pretty similar to the explanation from this example in Flabbergast's examples directory: https://github.com/flabbergast-config/flabbergast/tree/master/examples/complex I'm happy to share some of what I've been cobbling together with jsonnet if there is interest in such a thing. The --read-json option to the aurora cli tool has come in very handy :-) On Mon, Jan 11, 2016 at 6:02 AM Chris Bannister <[email protected]> wrote: > As part of building a tool to make it really easy to run Spark on Aurora > (with the goal being to do 'spark run <name> <jar>' which launches 2 aurora > jobs) I am creating the ExecutorData JSON in Go and using the Thrift > interfaces to call createJob, it works, and is very verbose. I think it > will work well as long as you dont want to do too much configuration. To > ensure I got the correct things I wrote an aurora config then looked at the > generated JobConfiguration. > > It would be nice if the ExecutorData was defined in Thrift or such, but I > can see why it is not. > > On Mon, 11 Jan 2016 at 13:34 Erb, Stephan <[email protected]> > wrote: > >> A couple of days ago there was a rather brief discussion in IRC regarding >> the generation of Aurora configuration files: >> >> > 00:16 <benley> random poll: anyone know who is something other than the >> normal Python DSL for building Aurora jobs? >> > 00:17 <benley> I know some people are using flabbergast, and rafik was >> at least experimenting with Nix to generate them >> > 00:17 <benley> and I've been tinkering with using Jsonnet to generate >> them, which ends up being quite similar to Flabbergast >> > 00:19 <wfarner> benley: i don't know of any other DSLs, but i have >> heard of a few distillations into essentially properties files >> > 00:21 <benley> that makes sense - heavily templating a job definition >> so it's generally applicable >> >> I find this topic quite interesting. Does anyone has some experiences to >> share with the mailinglist? What was your the main motivation using an >> additional configuration or templating mechanism? Did it pay off? >> >> Thanks and Best Regards, >> Stephan > >
