>From this conversation https://groups.google.com/forum/#!topic/bndtools-users/YMcHgcPLlpw Ferry linked to a good blog on switch from bnd metatypes to osgi ds ones. Link is here http://pelagic-open-source.blogspot.nl/2015/08/convert-to-osgi-r6.html
On Fri, Dec 25, 2015 at 8:11 PM, Benson Margulies <[email protected]> wrote: > On Fri, Dec 25, 2015 at 7:51 PM, David Jencks <[email protected]> > wrote: > > Benson, > > > > What you did is pretty similar in many ways to what I did although I > don’t have the json/yaml support. > > I'm hardly attached to 4 hours of hacking. If you're willing to take > Jackson deps, I'd be game to collaborate and add json/yaml(/xml) to > what you have. > > > > > I’m not sure the [] characters are appropriate for key tokens. Anyway > my encoding uses key1.<number>.key2.<number>….. instead of > key1[<number>].key2[<number>]….. > > I think I could switch to that. On the other hand, can you elaborate > on what's wrong with []? I picked them because it was a less tricky > parse (no "oh, a _number_, that must mean it's a sequence") but I > don't feel very strongly about it. > > > > > Since I’m interested in using DS spec features as much as possible, the > object model is annotations or interfaces rather than concrete classes. So > if your DS component has > > > > @Activate > > void activate(MyNestedConfigAnnotation config) {…} > > > > DS will generate implementations of the config annotation backed by the > CA properties map. > > I'd be interested to see a fully-worked-out example of this. I often > end up using a fair amount of Jackson @nnotation to tune the > representation. It seems possible that they could coexist; Jackson has > support deserializing to concrete classes as informed by annotations. > > > > > I just realized that I haven’t yet made it easy to turn this on by > committing the meta-annotation to generate the felix specific flag to allow > nested annotations….. maybe this weekend. > > > > While both you and I picked complex keys, Peter Kriens seems to be > strongly in favor of complex values encoded in json. I believe one of his > reasons is that such configurations are less likely to completely break > existing metatype-aware config viewers. > > I waffled back and forth between 'CA is fundamentally a key-value > store, and I need fancy values' and 'I don't want to have a key-value > store at all'. I landed where I did because I wanted to be able to > design a configuration file however it made sense to me -- even with > an sequence rather than a map at the top. But I'm not surprised to > learn that my 'elders' in this design space feel that more > compatibility is more important than this ultimate flexibility, and I > could live with that. As to the contents of the config when in memory, > I don't see anything too horrible about strings of json; it would make > my deflattening scheme a bit slower, but who cares? How big could one > of these sensibly get? > > > > > At work I have a proprietary system that relates metatype to these > nested configurations, but so far there is no hint of spec support. Do you > have a plan for using metatype? > > The first three hits on google for metatype weren't informative enough > to get me off the ground here, so I deferred thinking about it. When > reading via Jackson, I know the type of each value, so I could, I > suppose, generate metatype data on the fly; I'd need to find my way > to/through the documentation about how to do that instead of having > XML sitting in OSGI-INF. Assuming that there is such a way. > > > > > While I’m sure OSGI would like your employer to join and help pay $$, I > think you personally can join as a non-voting person for free or a trivial > amount of $$. I’d encourage you to consider this to help out with the spec > development. > > I'll have a look. To be honest, I've posted enough stupid stuff here > as I've been learning my way around that I don't think of myself much > of a candidate member of a spec team; I can certainly represent my use > case. > > > > > > thanks > > david jencks > > > >> On Dec 25, 2015, at 3:00 PM, Benson Margulies <[email protected]> > wrote: > >> > >> David, > >> > >> Warning, this is going to be a long message. > >> > >> Background: > >> > >> I created an OSGi DS component which is initialized from a complex > >> configuration. I defined this configuration in terms of a Java object > >> model. I implemented it by having a single configuration admin > >> property that pointed to a file of YAML, and then I use Jackson to > >> read that file into the Java objects of the model. I've typed a subset > >> of it below. > >> > >> It all is quite readable and maintainable as a YAML file. That's the > >> driver here: I want a component with relatively complex configuration, > >> and I want that configuration to be maintainable and readable in a > >> natural way. I don't want the devops people to have to deal with the > >> consequences of force-fitting the complex configuration into a simple > >> key-value model, particularly because there are sequences of objects > >> in the natural model. This all works. > >> > >> How this (new) code came to be: > >> > >> It occurred to us that it would be desirable to be able to manipulate > >> this, to some extent, through ConfigurationAdmin -- to be able to drop > >> in a new config and have the service reconfigure, to see and edit some > >> details in the existing tools. > >> > >> After some discussion with other people who had done similar things on > >> the Karaf list, I decided to adopt 'flatten tree-shaped configurations > >> into a single CA configuration', because I was able to think my way > >> around it. The implementation uses complex keys to avoid complex > >> values; I didn't know that Felix had existing support; I invented my > >> own (rather obvious) mapping between the Jackson generic tree > >> structure and complex keys. (I suspected that something awful would > >> happen if I put complex values into the Dictionary and fed that to > >> ConfigurationAdmin.) Certainly the web console and the Karaf shell > >> would be unable to help edit such things. > >> > >> To illustrate: > >> > >> Consider a configuration file in YAML like: > >> > >> factories: > >> - id: factory1 > >> componentId: tokenization > >> - id: factory2 > >> componentId: ner > >> pipelines: > >> - endpoint: ner > >> stages: > >> - factoryId: factory1 > >> languages: ['*'] > >> - factoryId: factory2 > >> languages: ['*'] > >> > >> This corresponds to an overall Java class (Configuration), plus > >> Factory and Stage. > >> > >> class Configuration { > >> List<Factory> factories; > >> List<Pipeline> pipelines; > >> }; // etc, etc. > >> > >> The service component in that git repo reads a file like this as a > >> generic tree structure that can represent Json, Yaml (or even XML), > >> and then 'flattens' it using complex keys: > >> > >> factories[0].id = factory2 > >> factories[0].componentId = tokenization > >> ... > >> pipelines[0].endpoint = ner > >> pipelines[0].stages[0].factoryId = factory1 > >> ... > >> > >> Given all of this, my device reads the file, makes the Dictionary with > >> the complex 'flattened' keys, and pushed it into ConfigurationAdmin. > >> The service that does the work now receives the Dictionary with those > >> keys and their values. However, it does not have to work with them as > >> such. It can call a method I've supplied to 'deflatten' -- to convert > >> the key-value dictionary back into the tree structure of the Jackson > >> 'JsonNode' classes. But that's not all: once it has that tree, it can > >> ask Jackson to convert to the original nice object model. And now > >> we've got cake and and eating it too; the code of my service works > >> with the nice Java object model, the config in source control is > >> readable YAML, and you can at least look around and make simple > >> changes in the standard tools. > >> > >> The code I wrote today is not all that complex: one class maps back > >> and forth between JsonNode trees and Dictionary objects, and another > >> uses the NIO2 watcher to implement the 'watch the file system and poke > >> ConfigurationAdmin'. > >> > >> to answer your first question: I don't know if I have access to R7 > >> docs. Neither I nor my employer has any particular relationship to the > >> OSGi consortium. > >> > >> --benson > >> > >> > >> > >> > >> On Fri, Dec 25, 2015 at 5:15 PM, David Jencks <[email protected]> > wrote: > >>> Hi Benson, > >>> > >>> Can you get to the OSGI RFCs under development for R7? There’s the > Configurer one abstracted from the enRoute configurer that David mentioned > and an object conversion service abstracting the config-by-annotations from > DS 1.3/R6. > >>> > >>> I don’t quite understand what you mean by “Object model” and > “flattening”. Does your project deal with converting nested configurations > into e.g. DS references somehow? Or does it flatten tree-shaped > configurations into a single CA configuration? I’m in favor of both of > these ideas making it into the specs in some form; felix DS already has > some support for tree-shaped configurations using complex keys rather than > complex values. > >>> > >>> I’d appreciate some more description of what your code does…. > >>> > >>> thanks > >>> david jencks > >>>> On Dec 25, 2015, at 1:56 PM, David Daniel < > [email protected]> wrote: > >>>> > >>>> Thank you for the explanation > >>>> > >>>> On Friday, December 25, 2015, Benson Margulies <[email protected] > > > >>>> wrote: > >>>> > >>>>> On Fri, Dec 25, 2015 at 4:45 PM, David Daniel > >>>>> <[email protected] <javascript:;>> wrote: > >>>>>> What are the differences other than yaml with enroutes configuration > >>>>>> provider. > >>>>>> > >>>>> > https://github.com/osgi/osgi.enroute.bundles/tree/master/osgi.enroute.configurer.simple.provider > >>>>> > >>>>> It does not look to me as if that does a full object model; it looks > >>>>> as if it just replaces property file syntax for a single-level map > >>>>> with json syntax. I may not be doing it justice in a fast read. > Also, > >>>>> mine allows the receiving component to convert the Dictionary (back) > >>>>> to a Jackson JsonNode, and thence to a real class model. So, you can > >>>>> define a configuration in terms of full data model, but still push it > >>>>> through config admin and allow the use of the webconsole or karaf > >>>>> shell to examine or make minor changes on the fly. I am not educated > >>>>> in enroute, so I would not be surprised to learn that I've reinvented > >>>>> a wheel. > >>>>> > >>>>> > >>>>>> > >>>>>> On Fri, Dec 25, 2015 at 4:39 PM, Benson Margulies < > [email protected] > >>>>> <javascript:;>> > >>>>>> wrote: > >>>>>> > >>>>>>> After some hints from folks on the Karaf list, I implemented a > mutant > >>>>>>> cousin of fileinstall. It only does config admin, and it's role in > >>>>>>> life is to read yaml or json files, 'flatten' them, and push the > >>>>>>> results into config admin. My plan is for the things that use it to > >>>>>>> reconstruct the original object model by reversing the flattening, > not > >>>>>>> to actually use keys like "foo.bar[3].baz' -- it includes an API > that > >>>>>>> maps a CA Dictionary back to a JsonTree. > >>>>>>> > >>>>>>> For expediency, I made it depend on NIO2 (not to mention Jackson) > and > >>>>> DS. > >>>>>>> > >>>>>>> I doubt that folks will see this as generally applicable, but I > >>>>>>> carefully set it up with AL so that I could contribute it if there > is > >>>>>>> a surprising (to me) groundswell of interest. I know that some > >>>>>>> karafers read this list, so I'm not going to bother to crosspost. > >>>>>>> > >>>>>>> I haven't written any automated tests yet. > >>>>>>> > >>>>>>> https://github.com/benson-basis/yaml-configuration-admin > >>>>>>> > >>>>> > >>> > > >
