Wow, looks like fun :-)
Certainly Slider will benefit from more flexibility. For example, sharing
configuration between some of the components, but not all of them.
But for now, can someone shed some light on how to access the resolved
component configuration from the .py script?
Let's take the following example from appConfig.json:
{
"metadata": {
},
"global": {
},
"components": {
"BROKER0": {
"broker.id": "0"
},
"BROKER1": {
"broker.id": "1",
},
"slider-appmaster": {
"jvm.heapsize": "256M"
}
}
}
How do I pull out broker.id (it is not part of the config dictionary)?
On Sun, May 10, 2015 at 9:16 AM, Steve Loughran <[email protected]>
wrote:
>
> > On 10 May 2015, at 05:23, Thomas Weise <[email protected]> wrote:
> >
> > Also, the configuration docs describe property inheritance and
> resolution:
> >
> > http://slider.incubator.apache.org/docs/configuration/core.html
> >
> > I could not find this used (other than for the app master) in existing
> > packages? Is there an example that also shows how these values are
> accessed
> > from the script?
> >
>
> I'm not sure that by the time the .py client gets to see things, they get
> to see the raw data, just the "resolved" stuff.
>
> That's something to look at in future: there's no reason why not to expose
> it.
>
> The other thing we have always talked about is cross-referencing in the
> .json config files, both to other bits of the specification (including
> between appconf.json & resources.json), and with some late-binding env
> variables (so that you really can get the env.PATH at the far end).
>
> That gets complex very, very fast, introduces loops, leads to very hard to
> debug configurations. And, once you start adding more features, the leap
> from declarative to full turing-equivalence. We've left things dumbed down
> (for now).
>
> However, did you notice the Google Borg paper hinted at their config
> language, with 100+K lines of config. That's one serious deployment.
>
> here's the only public documentation of their GCL
> http://alexandria.tue.nl/extra1/afstversl/wsk-i/bokharouss2008.pdf
>
