David,
Warning, this is going to be a long message.
Background:
I created an OSGi DS component which is initialized from a complex
configuration. I defined this configuration in terms of a Java object
model. I implemented it by having a single configuration admin
property that pointed to a file of YAML, and then I use Jackson to
read that file into the Java objects of the model. I've typed a subset
of it below.
It all is quite readable and maintainable as a YAML file. That's the
driver here: I want a component with relatively complex configuration,
and I want that configuration to be maintainable and readable in a
natural way. I don't want the devops people to have to deal with the
consequences of force-fitting the complex configuration into a simple
key-value model, particularly because there are sequences of objects
in the natural model. This all works.
How this (new) code came to be:
It occurred to us that it would be desirable to be able to manipulate
this, to some extent, through ConfigurationAdmin -- to be able to drop
in a new config and have the service reconfigure, to see and edit some
details in the existing tools.
After some discussion with other people who had done similar things on
the Karaf list, I decided to adopt 'flatten tree-shaped configurations
into a single CA configuration', because I was able to think my way
around it. The implementation uses complex keys to avoid complex
values; I didn't know that Felix had existing support; I invented my
own (rather obvious) mapping between the Jackson generic tree
structure and complex keys. (I suspected that something awful would
happen if I put complex values into the Dictionary and fed that to
ConfigurationAdmin.) Certainly the web console and the Karaf shell
would be unable to help edit such things.
To illustrate:
Consider a configuration file in YAML like:
factories:
- id: factory1
componentId: tokenization
- id: factory2
componentId: ner
pipelines:
- endpoint: ner
stages:
- factoryId: factory1
languages: ['*']
- factoryId: factory2
languages: ['*']
This corresponds to an overall Java class (Configuration), plus
Factory and Stage.
class Configuration {
List<Factory> factories;
List<Pipeline> pipelines;
}; // etc, etc.
The service component in that git repo reads a file like this as a
generic tree structure that can represent Json, Yaml (or even XML),
and then 'flattens' it using complex keys:
factories[0].id = factory2
factories[0].componentId = tokenization
...
pipelines[0].endpoint = ner
pipelines[0].stages[0].factoryId = factory1
...
Given all of this, my device reads the file, makes the Dictionary with
the complex 'flattened' keys, and pushed it into ConfigurationAdmin.
The service that does the work now receives the Dictionary with those
keys and their values. However, it does not have to work with them as
such. It can call a method I've supplied to 'deflatten' -- to convert
the key-value dictionary back into the tree structure of the Jackson
'JsonNode' classes. But that's not all: once it has that tree, it can
ask Jackson to convert to the original nice object model. And now
we've got cake and and eating it too; the code of my service works
with the nice Java object model, the config in source control is
readable YAML, and you can at least look around and make simple
changes in the standard tools.
The code I wrote today is not all that complex: one class maps back
and forth between JsonNode trees and Dictionary objects, and another
uses the NIO2 watcher to implement the 'watch the file system and poke
ConfigurationAdmin'.
to answer your first question: I don't know if I have access to R7
docs. Neither I nor my employer has any particular relationship to the
OSGi consortium.
--benson
On Fri, Dec 25, 2015 at 5:15 PM, David Jencks <[email protected]> wrote:
> Hi Benson,
>
> Can you get to the OSGI RFCs under development for R7? There’s the
> Configurer one abstracted from the enRoute configurer that David mentioned
> and an object conversion service abstracting the config-by-annotations from
> DS 1.3/R6.
>
> I don’t quite understand what you mean by “Object model” and “flattening”.
> Does your project deal with converting nested configurations into e.g. DS
> references somehow? Or does it flatten tree-shaped configurations into a
> single CA configuration? I’m in favor of both of these ideas making it into
> the specs in some form; felix DS already has some support for tree-shaped
> configurations using complex keys rather than complex values.
>
> I’d appreciate some more description of what your code does….
>
> thanks
> david jencks
>> On Dec 25, 2015, at 1:56 PM, David Daniel <[email protected]>
>> wrote:
>>
>> Thank you for the explanation
>>
>> On Friday, December 25, 2015, Benson Margulies <[email protected]>
>> wrote:
>>
>>> On Fri, Dec 25, 2015 at 4:45 PM, David Daniel
>>> <[email protected] <javascript:;>> wrote:
>>>> What are the differences other than yaml with enroutes configuration
>>>> provider.
>>>>
>>> https://github.com/osgi/osgi.enroute.bundles/tree/master/osgi.enroute.configurer.simple.provider
>>>
>>> It does not look to me as if that does a full object model; it looks
>>> as if it just replaces property file syntax for a single-level map
>>> with json syntax. I may not be doing it justice in a fast read. Also,
>>> mine allows the receiving component to convert the Dictionary (back)
>>> to a Jackson JsonNode, and thence to a real class model. So, you can
>>> define a configuration in terms of full data model, but still push it
>>> through config admin and allow the use of the webconsole or karaf
>>> shell to examine or make minor changes on the fly. I am not educated
>>> in enroute, so I would not be surprised to learn that I've reinvented
>>> a wheel.
>>>
>>>
>>>>
>>>> On Fri, Dec 25, 2015 at 4:39 PM, Benson Margulies <[email protected]
>>> <javascript:;>>
>>>> wrote:
>>>>
>>>>> After some hints from folks on the Karaf list, I implemented a mutant
>>>>> cousin of fileinstall. It only does config admin, and it's role in
>>>>> life is to read yaml or json files, 'flatten' them, and push the
>>>>> results into config admin. My plan is for the things that use it to
>>>>> reconstruct the original object model by reversing the flattening, not
>>>>> to actually use keys like "foo.bar[3].baz' -- it includes an API that
>>>>> maps a CA Dictionary back to a JsonTree.
>>>>>
>>>>> For expediency, I made it depend on NIO2 (not to mention Jackson) and
>>> DS.
>>>>>
>>>>> I doubt that folks will see this as generally applicable, but I
>>>>> carefully set it up with AL so that I could contribute it if there is
>>>>> a surprising (to me) groundswell of interest. I know that some
>>>>> karafers read this list, so I'm not going to bother to crosspost.
>>>>>
>>>>> I haven't written any automated tests yet.
>>>>>
>>>>> https://github.com/benson-basis/yaml-configuration-admin
>>>>>
>>>
>
