After a bit of experimentation yesterday it appears that the properties Maven plugin along with filtered resources with .cfg files will work admirably for this purpose. Multiple properties files can be load one after the other permitting overwriting and the properties substitutions in the .cfg files works like a champ.
I'll be working with it more today but this is going to solve that rather sticky problem for me. Brad On Fri, Jul 8, 2016 at 4:11 PM, Brad Johnson <[email protected]> wrote: > David, > > Multilocations would be marvelous but it doesn't appear that there is any > focus on blueprint right now. For my clients it wouldn't help anyway > because Fuse is Karaf 2.x and on OSGi R4. That's an entirely different > issue that I'm extremely unhappy about but can't change. Red Hat is in the > driver's seat there. If I had my way I'd prefer multiple small > installations of Karaf 4 with my own custom distributions. Such is life. > > The Configurator might work as well. I'm obviously hamstrung currently by > the platform. So when I look at solutions that require specification > changes or changes to focus on project priorities across OSGi where I don't > have any inlets, Aries and Camel there isn't much I can expect > realistically. > > You probably read Christian's comments about Camel and DS with the SCR. > My experience trying to use the two together was pretty dreadful. I'll > attempt it again in the future I'm sure as I always prototype stuff on a > regular basis. Being in CXF, JMS and Camel means that adopting DS is likely > just going to buy me more headaches because I don't see any way out of > Blueprint right now even if I decided to once again delve into Camel's Java > DSL. > > Modifying Maven builds or even creating a custom plugin is something I can > do. With filtered resources I can profiles I can create dev, test, prod > cfg files and compile those into a single bundle (probably with my > features) and then have goals that install the relevant cfgs depending on > the feature installed. > > Hacking together something like that doesn't make me happy and I would > prefer a more elegant solution. But having read the Camel mailing list > long enough I know I'm not the only one who wrestles with that problem in > Blueprint with the pinned PIDs. At least a compilation mechanism via Maven > would permit me to work with single PIDs but not duplicate data in multiple > files. It would also have a side benefit of making it more natural to > modify the applicable cfg in the configuration bundle and use that to > install the configurations in a controlled manner rather than modifying > already installed cfg files in the etc directory. > > Most of that I can probably do myself out of the box with standard Maven > plugins. The only one that will probably require custom code is something > to roll up cfg properties before writing out to the pid.cfg. > > > > > On Fri, Jul 8, 2016 at 9:52 AM, Brad Johnson <[email protected] > > wrote: > >> I don't think anyone missed your points. >> >> On Fri, Jul 8, 2016 at 9:45 AM, David Jencks <[email protected]> >> wrote: >> >>> Somehow my original points have, I think, gotten completely lost. >>> >>> 1. Multilocations should completely solve the original problem of >>> sharing a configuration across bundles. This requires an R5 + config >>> admin, which has been around for years. I don’t think reinventing this >>> wheel is plausible. >>> 2. Blueprint should work with multi locations out of the box with no >>> problems. If not, it’s almost certainly an easy to fix bug. Try it, lets >>> find out. >>> >>> >>> ———— >>> >>> 3. The additional idea of combining configurations with overrides is >>> very similar to the DS multiple pid support. Adding that to blueprint >>> might be a big job, and IMO ought to proceed as part of specifying >>> blueprint CM support (at all), but is presumably doable. >>> 4. Alternatively, the particular form of combining configuration data >>> shown, apparently based on matching patterns in input data file names, >>> could be done by a management agent that creates the configurations in the >>> first place, e.g. a modified fileinstall (I don’t know if this is what >>> karaf uses). There’s the Configurer work DavidB mentions that might also >>> be relevant. >>> >>> —————— >>> >>> 5. I think it’s odd that if blueprint is so great and so popular the >>> spec work on it hasn’t progressed. It seems to me like using CMP entity >>> beans in an ejb 3 application. Maybe everyone is happy developing the >>> single implementation without a spec, but it seems to me like the people >>> using blueprint aren’t aware of new features in other osgi specs they might >>> like to emulate, such as the multiple pid support. This just seems odd to >>> me. >>> >>> ————————— >>> While a voting, company, membership in OSGI is IIUC somewhat expensive, >>> you can become an OSGI supporter and participate (non-voting) in spec >>> development for free: https://www.osgi.org/join/membership-benefits/ >>> (see the bottom) >>> >>> thanks >>> david jencks >>> >>> On Jul 8, 2016, at 7:22 AM, Brad Johnson <[email protected]> >>> wrote: >>> >>> After getting into these discussions I started thinking that perhaps I'd >>> move all the configuration files into one bundle and install them from >>> features file that way. I could have dev/test/prod cfg files in that same >>> bundle. >>> >>> Using a combination of filtered resources and some other mechanics then >>> I can perhaps come up with a way so that >>> >>> com.my.batch.transaction.cfg >>> com.my.batch.accountupdates.cfg >>> >>> have a com.my.batch.parent.cfg and during Maven build it is read into >>> and made part of both the transaction and account updates.cfg files. Those >>> are then put into the configuration bundle. When that configuration bundle >>> is installed the PIDs/properties match the bundles looking for them but >>> contain the common information as already been flattened into them. >>> >>> That configuration bundle could also have test/dev/production cfg files >>> and the features file could permit installing test, dev or production the >>> only difference being which set of configuration files they installed >>> before installing the bundles themselves. >>> >>> >>> On Fri, Jul 8, 2016 at 9:12 AM, Brad Johnson < >>> [email protected]> wrote: >>> >>>> Here's a small example. The two batch processes do very different >>>> things yet share some common configuration information pertaining to >>>> directories, ftp and service endpoints. That configuration information >>>> obviously also varies between dev, test, and production. This is one small >>>> part of the project. The connector project is already a crosscutting >>>> concern that keeps its connection information separated from all the other >>>> business processes/services that use it. Some of the properties have queue >>>> names (SEDA or JMS) that are shared between bundles and use runtime >>>> substitution in Camel. But duplicating configuration information between >>>> the bundles and hand massaging it is error prone and makes refactoring a >>>> pain. >>>> >>>> <feature name="someproject" version="${project.version}"> >>>> >>>> <configfile finalname="/etc/com.litle.cfg" >>>> override="false">mvn:com.my.connectors/litle/${project.version}/cfg/configuration</configfile> >>>> <configfile finalname="/etc/com.my.batch.transactions.cfg" >>>> override="false">mvn: >>>> com.my/batch.transactions/${project.version}/cfg/configuration >>>> <http://com.my/batch.transactions/$%7Bproject.version%7D/cfg/configuration> >>>> </configfile> >>>> <configfile finalname="/etc/com.my.batch.accountupdates.cfg" >>>> override="false">mvn: >>>> com.my/batch.accountupdates/${project.version}/cfg/configuration >>>> <http://com.my/batch.accountupdates/$%7Bproject.version%7D/cfg/configuration> >>>> </configfile> >>>> <feature>camel-core</feature> >>>> <feature>camel-blueprint</feature> >>>> snip >>>> >>>> And some route like: >>>> >>>> <to >>>> uri="{{batch.local.outbox}}?fileExist=Append&fileName=${header.outFile}" >>>> /> >>>> >>>> On Fri, Jul 8, 2016 at 8:33 AM, Christian Schneider < >>>> [email protected]> wrote: >>>> >>>>> Can you give some examples of typical config properties that need to >>>>> be shared between these bundles. Maybe they lead to an idea for a >>>>> different >>>>> design. >>>>> >>>>> Christian >>>>> >>>>> On 08.07.2016 15:09, Brad Johnson wrote: >>>>> >>>>>> Christian, >>>>>> >>>>>> Thanks for the feedback. I'm actually looking for a bit more robust >>>>>> solution. Just the one small project I'm working on now has 12 business >>>>>> processes with a number of OSGi services that provide cross-cutting >>>>>> concerns. In this case they are service like credit card authorization, >>>>>> refunds, etc. CXF front ends them with REST/SOAP and I use >>>>>> recipientList >>>>>> to route them. But each of those business processes use OSGi services >>>>>> that >>>>>> might call out to banks, PayPal, analytics services, etc. Keeping these >>>>>> nicely segregated in their own bundles makes them very testable for the >>>>>> routing, data transformation to/from canonical form, etc. But >>>>>> configuration management is a bit of a pain because of the PID pinning. >>>>>> I >>>>>> use update strategy reload on the bundles so that configurations can be >>>>>> changed in different environments. >>>>>> >>>>>> Until now I've kept my configuration files in each bundle and pop >>>>>> them out using the features install mechanism. But I'm rethinking that >>>>>> based on these discussions. I don't know if the Maven properties plugin >>>>>> would do what I'm thinking about but it might be a platform to develop >>>>>> such >>>>>> a plugin. In this case I'd have a separate configuration bundle with my >>>>>> PID cfgs in them and N levels of parent cfg files. The properties would >>>>>> just be rolled up and written out into PID specific cfg files. If I >>>>>> define >>>>>> a port or directory in a parent then all the children would inherit or >>>>>> override that property. That would permit the use of profiles for dev, >>>>>> test and production as well. Modifying any of the cfg files and then >>>>>> putting that through Jenkins would result in a nice configuration bundle >>>>>> that once reinstalled would trigger a reload. In the end the port or >>>>>> directory or endpoint configuration information would be duplicated in >>>>>> each >>>>>> cfg but it wouldn't require hand massaging of each cfg to change. >>>>>> >>>>>> >>>>> -- >>>>> Christian Schneider >>>>> http://www.liquid-reality.de >>>>> >>>>> Open Source Architect >>>>> http://www.talend.com >>>>> >>>>> >>>> >>> >>> >> >
