Should link to rfcs as well. https://github.com/osgi/design/tree/master/rfcs Object conversion in particular https://github.com/osgi/design/tree/master/rfcs/rfc0215 I am not sure if there are more in depth ones that committers or osgi members have access to.
On Fri, Dec 25, 2015 at 8:30 PM, David Daniel <[email protected]> wrote: > From this conversation > https://groups.google.com/forum/#!topic/bndtools-users/YMcHgcPLlpw Ferry > linked to a good blog on switch from bnd metatypes to osgi ds ones. Link > is here > http://pelagic-open-source.blogspot.nl/2015/08/convert-to-osgi-r6.html > > On Fri, Dec 25, 2015 at 8:11 PM, Benson Margulies <[email protected]> > wrote: > >> On Fri, Dec 25, 2015 at 7:51 PM, David Jencks <[email protected]> >> wrote: >> > Benson, >> > >> > What you did is pretty similar in many ways to what I did although I >> don’t have the json/yaml support. >> >> I'm hardly attached to 4 hours of hacking. If you're willing to take >> Jackson deps, I'd be game to collaborate and add json/yaml(/xml) to >> what you have. >> >> > >> > I’m not sure the [] characters are appropriate for key tokens. Anyway >> my encoding uses key1.<number>.key2.<number>….. instead of >> key1[<number>].key2[<number>]….. >> >> I think I could switch to that. On the other hand, can you elaborate >> on what's wrong with []? I picked them because it was a less tricky >> parse (no "oh, a _number_, that must mean it's a sequence") but I >> don't feel very strongly about it. >> >> > >> > Since I’m interested in using DS spec features as much as possible, the >> object model is annotations or interfaces rather than concrete classes. So >> if your DS component has >> > >> > @Activate >> > void activate(MyNestedConfigAnnotation config) {…} >> > >> > DS will generate implementations of the config annotation backed by the >> CA properties map. >> >> I'd be interested to see a fully-worked-out example of this. I often >> end up using a fair amount of Jackson @nnotation to tune the >> representation. It seems possible that they could coexist; Jackson has >> support deserializing to concrete classes as informed by annotations. >> >> > >> > I just realized that I haven’t yet made it easy to turn this on by >> committing the meta-annotation to generate the felix specific flag to allow >> nested annotations….. maybe this weekend. >> > >> > While both you and I picked complex keys, Peter Kriens seems to be >> strongly in favor of complex values encoded in json. I believe one of his >> reasons is that such configurations are less likely to completely break >> existing metatype-aware config viewers. >> >> I waffled back and forth between 'CA is fundamentally a key-value >> store, and I need fancy values' and 'I don't want to have a key-value >> store at all'. I landed where I did because I wanted to be able to >> design a configuration file however it made sense to me -- even with >> an sequence rather than a map at the top. But I'm not surprised to >> learn that my 'elders' in this design space feel that more >> compatibility is more important than this ultimate flexibility, and I >> could live with that. As to the contents of the config when in memory, >> I don't see anything too horrible about strings of json; it would make >> my deflattening scheme a bit slower, but who cares? How big could one >> of these sensibly get? >> >> > >> > At work I have a proprietary system that relates metatype to these >> nested configurations, but so far there is no hint of spec support. Do you >> have a plan for using metatype? >> >> The first three hits on google for metatype weren't informative enough >> to get me off the ground here, so I deferred thinking about it. When >> reading via Jackson, I know the type of each value, so I could, I >> suppose, generate metatype data on the fly; I'd need to find my way >> to/through the documentation about how to do that instead of having >> XML sitting in OSGI-INF. Assuming that there is such a way. >> >> > >> > While I’m sure OSGI would like your employer to join and help pay $$, I >> think you personally can join as a non-voting person for free or a trivial >> amount of $$. I’d encourage you to consider this to help out with the spec >> development. >> >> I'll have a look. To be honest, I've posted enough stupid stuff here >> as I've been learning my way around that I don't think of myself much >> of a candidate member of a spec team; I can certainly represent my use >> case. >> >> >> > >> > thanks >> > david jencks >> > >> >> On Dec 25, 2015, at 3:00 PM, Benson Margulies <[email protected]> >> wrote: >> >> >> >> David, >> >> >> >> Warning, this is going to be a long message. >> >> >> >> Background: >> >> >> >> I created an OSGi DS component which is initialized from a complex >> >> configuration. I defined this configuration in terms of a Java object >> >> model. I implemented it by having a single configuration admin >> >> property that pointed to a file of YAML, and then I use Jackson to >> >> read that file into the Java objects of the model. I've typed a subset >> >> of it below. >> >> >> >> It all is quite readable and maintainable as a YAML file. That's the >> >> driver here: I want a component with relatively complex configuration, >> >> and I want that configuration to be maintainable and readable in a >> >> natural way. I don't want the devops people to have to deal with the >> >> consequences of force-fitting the complex configuration into a simple >> >> key-value model, particularly because there are sequences of objects >> >> in the natural model. This all works. >> >> >> >> How this (new) code came to be: >> >> >> >> It occurred to us that it would be desirable to be able to manipulate >> >> this, to some extent, through ConfigurationAdmin -- to be able to drop >> >> in a new config and have the service reconfigure, to see and edit some >> >> details in the existing tools. >> >> >> >> After some discussion with other people who had done similar things on >> >> the Karaf list, I decided to adopt 'flatten tree-shaped configurations >> >> into a single CA configuration', because I was able to think my way >> >> around it. The implementation uses complex keys to avoid complex >> >> values; I didn't know that Felix had existing support; I invented my >> >> own (rather obvious) mapping between the Jackson generic tree >> >> structure and complex keys. (I suspected that something awful would >> >> happen if I put complex values into the Dictionary and fed that to >> >> ConfigurationAdmin.) Certainly the web console and the Karaf shell >> >> would be unable to help edit such things. >> >> >> >> To illustrate: >> >> >> >> Consider a configuration file in YAML like: >> >> >> >> factories: >> >> - id: factory1 >> >> componentId: tokenization >> >> - id: factory2 >> >> componentId: ner >> >> pipelines: >> >> - endpoint: ner >> >> stages: >> >> - factoryId: factory1 >> >> languages: ['*'] >> >> - factoryId: factory2 >> >> languages: ['*'] >> >> >> >> This corresponds to an overall Java class (Configuration), plus >> >> Factory and Stage. >> >> >> >> class Configuration { >> >> List<Factory> factories; >> >> List<Pipeline> pipelines; >> >> }; // etc, etc. >> >> >> >> The service component in that git repo reads a file like this as a >> >> generic tree structure that can represent Json, Yaml (or even XML), >> >> and then 'flattens' it using complex keys: >> >> >> >> factories[0].id = factory2 >> >> factories[0].componentId = tokenization >> >> ... >> >> pipelines[0].endpoint = ner >> >> pipelines[0].stages[0].factoryId = factory1 >> >> ... >> >> >> >> Given all of this, my device reads the file, makes the Dictionary with >> >> the complex 'flattened' keys, and pushed it into ConfigurationAdmin. >> >> The service that does the work now receives the Dictionary with those >> >> keys and their values. However, it does not have to work with them as >> >> such. It can call a method I've supplied to 'deflatten' -- to convert >> >> the key-value dictionary back into the tree structure of the Jackson >> >> 'JsonNode' classes. But that's not all: once it has that tree, it can >> >> ask Jackson to convert to the original nice object model. And now >> >> we've got cake and and eating it too; the code of my service works >> >> with the nice Java object model, the config in source control is >> >> readable YAML, and you can at least look around and make simple >> >> changes in the standard tools. >> >> >> >> The code I wrote today is not all that complex: one class maps back >> >> and forth between JsonNode trees and Dictionary objects, and another >> >> uses the NIO2 watcher to implement the 'watch the file system and poke >> >> ConfigurationAdmin'. >> >> >> >> to answer your first question: I don't know if I have access to R7 >> >> docs. Neither I nor my employer has any particular relationship to the >> >> OSGi consortium. >> >> >> >> --benson >> >> >> >> >> >> >> >> >> >> On Fri, Dec 25, 2015 at 5:15 PM, David Jencks < >> [email protected]> wrote: >> >>> Hi Benson, >> >>> >> >>> Can you get to the OSGI RFCs under development for R7? There’s the >> Configurer one abstracted from the enRoute configurer that David mentioned >> and an object conversion service abstracting the config-by-annotations from >> DS 1.3/R6. >> >>> >> >>> I don’t quite understand what you mean by “Object model” and >> “flattening”. Does your project deal with converting nested configurations >> into e.g. DS references somehow? Or does it flatten tree-shaped >> configurations into a single CA configuration? I’m in favor of both of >> these ideas making it into the specs in some form; felix DS already has >> some support for tree-shaped configurations using complex keys rather than >> complex values. >> >>> >> >>> I’d appreciate some more description of what your code does…. >> >>> >> >>> thanks >> >>> david jencks >> >>>> On Dec 25, 2015, at 1:56 PM, David Daniel < >> [email protected]> wrote: >> >>>> >> >>>> Thank you for the explanation >> >>>> >> >>>> On Friday, December 25, 2015, Benson Margulies < >> [email protected]> >> >>>> wrote: >> >>>> >> >>>>> On Fri, Dec 25, 2015 at 4:45 PM, David Daniel >> >>>>> <[email protected] <javascript:;>> wrote: >> >>>>>> What are the differences other than yaml with enroutes >> configuration >> >>>>>> provider. >> >>>>>> >> >>>>> >> https://github.com/osgi/osgi.enroute.bundles/tree/master/osgi.enroute.configurer.simple.provider >> >>>>> >> >>>>> It does not look to me as if that does a full object model; it looks >> >>>>> as if it just replaces property file syntax for a single-level map >> >>>>> with json syntax. I may not be doing it justice in a fast read. >> Also, >> >>>>> mine allows the receiving component to convert the Dictionary (back) >> >>>>> to a Jackson JsonNode, and thence to a real class model. So, you can >> >>>>> define a configuration in terms of full data model, but still push >> it >> >>>>> through config admin and allow the use of the webconsole or karaf >> >>>>> shell to examine or make minor changes on the fly. I am not educated >> >>>>> in enroute, so I would not be surprised to learn that I've >> reinvented >> >>>>> a wheel. >> >>>>> >> >>>>> >> >>>>>> >> >>>>>> On Fri, Dec 25, 2015 at 4:39 PM, Benson Margulies < >> [email protected] >> >>>>> <javascript:;>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> After some hints from folks on the Karaf list, I implemented a >> mutant >> >>>>>>> cousin of fileinstall. It only does config admin, and it's role in >> >>>>>>> life is to read yaml or json files, 'flatten' them, and push the >> >>>>>>> results into config admin. My plan is for the things that use it >> to >> >>>>>>> reconstruct the original object model by reversing the >> flattening, not >> >>>>>>> to actually use keys like "foo.bar[3].baz' -- it includes an API >> that >> >>>>>>> maps a CA Dictionary back to a JsonTree. >> >>>>>>> >> >>>>>>> For expediency, I made it depend on NIO2 (not to mention Jackson) >> and >> >>>>> DS. >> >>>>>>> >> >>>>>>> I doubt that folks will see this as generally applicable, but I >> >>>>>>> carefully set it up with AL so that I could contribute it if >> there is >> >>>>>>> a surprising (to me) groundswell of interest. I know that some >> >>>>>>> karafers read this list, so I'm not going to bother to crosspost. >> >>>>>>> >> >>>>>>> I haven't written any automated tests yet. >> >>>>>>> >> >>>>>>> https://github.com/benson-basis/yaml-configuration-admin >> >>>>>>> >> >>>>> >> >>> >> > >> > >
