Benson,
What you did is pretty similar in many ways to what I did although I don’t have
the json/yaml support.
I’m not sure the [] characters are appropriate for key tokens. Anyway my
encoding uses key1.<number>.key2.<number>….. instead of
key1[<number>].key2[<number>]…..
Since I’m interested in using DS spec features as much as possible, the object
model is annotations or interfaces rather than concrete classes. So if your DS
component has
@Activate
void activate(MyNestedConfigAnnotation config) {…}
DS will generate implementations of the config annotation backed by the CA
properties map.
I just realized that I haven’t yet made it easy to turn this on by committing
the meta-annotation to generate the felix specific flag to allow nested
annotations….. maybe this weekend.
While both you and I picked complex keys, Peter Kriens seems to be strongly in
favor of complex values encoded in json. I believe one of his reasons is that
such configurations are less likely to completely break existing metatype-aware
config viewers.
At work I have a proprietary system that relates metatype to these nested
configurations, but so far there is no hint of spec support. Do you have a
plan for using metatype?
While I’m sure OSGI would like your employer to join and help pay $$, I think
you personally can join as a non-voting person for free or a trivial amount of
$$. I’d encourage you to consider this to help out with the spec development.
thanks
david jencks
> On Dec 25, 2015, at 3:00 PM, Benson Margulies <[email protected]> wrote:
>
> David,
>
> Warning, this is going to be a long message.
>
> Background:
>
> I created an OSGi DS component which is initialized from a complex
> configuration. I defined this configuration in terms of a Java object
> model. I implemented it by having a single configuration admin
> property that pointed to a file of YAML, and then I use Jackson to
> read that file into the Java objects of the model. I've typed a subset
> of it below.
>
> It all is quite readable and maintainable as a YAML file. That's the
> driver here: I want a component with relatively complex configuration,
> and I want that configuration to be maintainable and readable in a
> natural way. I don't want the devops people to have to deal with the
> consequences of force-fitting the complex configuration into a simple
> key-value model, particularly because there are sequences of objects
> in the natural model. This all works.
>
> How this (new) code came to be:
>
> It occurred to us that it would be desirable to be able to manipulate
> this, to some extent, through ConfigurationAdmin -- to be able to drop
> in a new config and have the service reconfigure, to see and edit some
> details in the existing tools.
>
> After some discussion with other people who had done similar things on
> the Karaf list, I decided to adopt 'flatten tree-shaped configurations
> into a single CA configuration', because I was able to think my way
> around it. The implementation uses complex keys to avoid complex
> values; I didn't know that Felix had existing support; I invented my
> own (rather obvious) mapping between the Jackson generic tree
> structure and complex keys. (I suspected that something awful would
> happen if I put complex values into the Dictionary and fed that to
> ConfigurationAdmin.) Certainly the web console and the Karaf shell
> would be unable to help edit such things.
>
> To illustrate:
>
> Consider a configuration file in YAML like:
>
> factories:
> - id: factory1
> componentId: tokenization
> - id: factory2
> componentId: ner
> pipelines:
> - endpoint: ner
> stages:
> - factoryId: factory1
> languages: ['*']
> - factoryId: factory2
> languages: ['*']
>
> This corresponds to an overall Java class (Configuration), plus
> Factory and Stage.
>
> class Configuration {
> List<Factory> factories;
> List<Pipeline> pipelines;
> }; // etc, etc.
>
> The service component in that git repo reads a file like this as a
> generic tree structure that can represent Json, Yaml (or even XML),
> and then 'flattens' it using complex keys:
>
> factories[0].id = factory2
> factories[0].componentId = tokenization
> ...
> pipelines[0].endpoint = ner
> pipelines[0].stages[0].factoryId = factory1
> ...
>
> Given all of this, my device reads the file, makes the Dictionary with
> the complex 'flattened' keys, and pushed it into ConfigurationAdmin.
> The service that does the work now receives the Dictionary with those
> keys and their values. However, it does not have to work with them as
> such. It can call a method I've supplied to 'deflatten' -- to convert
> the key-value dictionary back into the tree structure of the Jackson
> 'JsonNode' classes. But that's not all: once it has that tree, it can
> ask Jackson to convert to the original nice object model. And now
> we've got cake and and eating it too; the code of my service works
> with the nice Java object model, the config in source control is
> readable YAML, and you can at least look around and make simple
> changes in the standard tools.
>
> The code I wrote today is not all that complex: one class maps back
> and forth between JsonNode trees and Dictionary objects, and another
> uses the NIO2 watcher to implement the 'watch the file system and poke
> ConfigurationAdmin'.
>
> to answer your first question: I don't know if I have access to R7
> docs. Neither I nor my employer has any particular relationship to the
> OSGi consortium.
>
> --benson
>
>
>
>
> On Fri, Dec 25, 2015 at 5:15 PM, David Jencks <[email protected]>
> wrote:
>> Hi Benson,
>>
>> Can you get to the OSGI RFCs under development for R7? There’s the
>> Configurer one abstracted from the enRoute configurer that David mentioned
>> and an object conversion service abstracting the config-by-annotations from
>> DS 1.3/R6.
>>
>> I don’t quite understand what you mean by “Object model” and “flattening”.
>> Does your project deal with converting nested configurations into e.g. DS
>> references somehow? Or does it flatten tree-shaped configurations into a
>> single CA configuration? I’m in favor of both of these ideas making it into
>> the specs in some form; felix DS already has some support for tree-shaped
>> configurations using complex keys rather than complex values.
>>
>> I’d appreciate some more description of what your code does….
>>
>> thanks
>> david jencks
>>> On Dec 25, 2015, at 1:56 PM, David Daniel <[email protected]>
>>> wrote:
>>>
>>> Thank you for the explanation
>>>
>>> On Friday, December 25, 2015, Benson Margulies <[email protected]>
>>> wrote:
>>>
>>>> On Fri, Dec 25, 2015 at 4:45 PM, David Daniel
>>>> <[email protected] <javascript:;>> wrote:
>>>>> What are the differences other than yaml with enroutes configuration
>>>>> provider.
>>>>>
>>>> https://github.com/osgi/osgi.enroute.bundles/tree/master/osgi.enroute.configurer.simple.provider
>>>>
>>>> It does not look to me as if that does a full object model; it looks
>>>> as if it just replaces property file syntax for a single-level map
>>>> with json syntax. I may not be doing it justice in a fast read. Also,
>>>> mine allows the receiving component to convert the Dictionary (back)
>>>> to a Jackson JsonNode, and thence to a real class model. So, you can
>>>> define a configuration in terms of full data model, but still push it
>>>> through config admin and allow the use of the webconsole or karaf
>>>> shell to examine or make minor changes on the fly. I am not educated
>>>> in enroute, so I would not be surprised to learn that I've reinvented
>>>> a wheel.
>>>>
>>>>
>>>>>
>>>>> On Fri, Dec 25, 2015 at 4:39 PM, Benson Margulies <[email protected]
>>>> <javascript:;>>
>>>>> wrote:
>>>>>
>>>>>> After some hints from folks on the Karaf list, I implemented a mutant
>>>>>> cousin of fileinstall. It only does config admin, and it's role in
>>>>>> life is to read yaml or json files, 'flatten' them, and push the
>>>>>> results into config admin. My plan is for the things that use it to
>>>>>> reconstruct the original object model by reversing the flattening, not
>>>>>> to actually use keys like "foo.bar[3].baz' -- it includes an API that
>>>>>> maps a CA Dictionary back to a JsonTree.
>>>>>>
>>>>>> For expediency, I made it depend on NIO2 (not to mention Jackson) and
>>>> DS.
>>>>>>
>>>>>> I doubt that folks will see this as generally applicable, but I
>>>>>> carefully set it up with AL so that I could contribute it if there is
>>>>>> a surprising (to me) groundswell of interest. I know that some
>>>>>> karafers read this list, so I'm not going to bother to crosspost.
>>>>>>
>>>>>> I haven't written any automated tests yet.
>>>>>>
>>>>>> https://github.com/benson-basis/yaml-configuration-admin
>>>>>>
>>>>
>>
