Hi JB, sounds like a good solution, this way we add some more value to Cave. And everything is provided from the same technology, so no breach there especially concerning the git/gist/external repo thing.
+1 regards, Achim 2014-12-15 10:34 GMT+01:00 Jean-Baptiste Onofré <[email protected]>: > Hi guys, > > following Guillaume work on the profile, I have an extended proposal that > I want to discuss with you. > > The Karaf profiles have to be stored somewhere: on github/git for > instance, in order to be applied on a distribution. > For instance, what I tested is the karaf docker.io feature with profiles: > it works fine, the docker.io feature bootstraps a docker image with > Karaf, and I applied a profile on this Karaf instance. > > A storage is also needed for the docker.io feature: where the images are > stored (basically a docker.io github like). > > On a private infrastructure, companies don't want to use "external" > storage (like github): they want to use first their "local" storages. It's > also valid for hybrid infrastructure: their want to use their local storage > first (as master), and possible proxy external storages. > > Right now, we have Karaf Cave providing: > - Maven repositories management (stored in Cave or mirroring/proxying > external repositories) > - OBR server management > - Feature Enterprise Repositories management > Cave is especially interesting with Cellar where all cluster nodes share > Cave repositories. > > In order to have centralised storage (and possibly replicated storage), I > wonder if we can not extend Cave to provide: > - Karaf profiles repositories > - Docker.io images repositories > It could work like for Maven repository, meaning local storage or > mirror/proxy to another one. > > Like this, we could have Karaf and shell commands to bootstrap Karaf > docker.io images, or apply profiles on an instance. > > WDYT ? > > Regards > JB > > On 12/11/2014 09:04 AM, Guillaume Nodet wrote: > >> I've just committed to the 4.x branch two new features: >> * profiles >> * static distributions >> >> Profiles >> ====== >> >> A profile is a data structure that brings together configuration and >> provisioning, so it's related to features, but is quite different. A demo >> is available at [1]. >> A profile contains a list of configuration files which can be either >> properties based or in any other format. Properties based configurations >> are handled in a very specific way, as they can overlay each other (more >> below). >> One of this properties based configuration holds provisioning informations >> along with profiles specific attributes, such as a list of parent >> profiles, >> a list of bundles, a list of features repositories and a list of features. >> >> A profile can have one or more parents, so the hierarchy is an inverted >> tree. This allows to define generic profiles and specialise them in >> children. >> >> Overlay profile: at some point, there is a need to flatten this hierarchy >> into an "overlay" profile. This process is done by walking into the >> parents hierarchy (depth first) and computing the resulting configuration >> for each configuration file. Non properties configuration files simply >> overwrite each other, while properties based configuration complement each >> other (with the ability to clear a single value or all values). >> >> Effective profile: properties configurations can contain placeholders to >> resolve. The only one defined atm is a property placeholder pointing to a >> value in a different configuration. Those placeholders are resolved on >> the >> overlay profile to give the end result effective profile which will be >> actually used. An example of using this placeholder is shown in [2] which >> end up pointing to the configuration in its parent [3] >> >> Provisioning: each profile contains a list of feature repositories, >> features and bundles. Those information will be used in the effective >> profile to know which bundles and features are needed on a given instance. >> Profiles are not used at runtime for the moment and there are 2 main >> usage : creating child instances and generating distributions using the >> maven plugin. >> >> Profile manipulation : profiles are manipulated using the Profiles class >> (static helpers) and the ProfileBuilder (to build profile instances, which >> are immutable). Reading and writing profiles is done using the jdk7 in >> FileSystem / Path interface, which obviously provides support for the >> standard file system, but could be used with various file systems (inside >> a >> zip [4], readonly github remote repository [5], sftp [6], in memory [7]. >> Those are the ones I know about). A set of commands can be used to edit >> the profiles. >> >> Further considerations: possible improvements include an overlay mechanism >> for xml, integration with cellar, file systems using a real git repo, >> hazelcast, zookeeper, etc... Applying profiles at runtime could be done >> too. Better integration with kars could be provided too. >> >> >> Static distributions >> ============== >> >> One use case of profiles is to generate distributions. Changes on the >> profile need a rebuild on the distribution (this may or may not be a >> problem, depending on your environment, but if it can seems weird in an >> OSGi world, it's not so much in a cloud environment). This leads to the >> distribution being mostly static, i.e. the user should not manually >> install >> / uninstall stuff. >> A demo is available at [8] >> Based on this idea, I improved the maven plugin a bit to allow generating >> mostly static distribution : the profiles / features are all installed in >> etc/startup.properties. A simplistic read-only configuration admin is >> used >> that will pick up configuration in the etc/ folder directly. This lead to >> no runtime dependencies for the provisioning of karaf itself : >> fileinstall, >> the features service can all be removed. The bare framework can only >> contain 3 bundles : pax-logging + the static configadmin [9]. One >> improvement is that the distribution is generated using reference:file: >> urls in startup.properties, avoiding a copy of each jar into the cache. >> An >> additional improvement could be to pre-compute the wiring which can take >> some time (but this is not supported by felix). >> >> >> Thoughts welcomed ! >> >> >> [1] https://github.com/apache/karaf/tree/master/demos/profiles >> [2] >> https://github.com/apache/karaf/blob/master/demos/ >> profiles/registry/src/main/resources/karaf.profile/profile.cfg#L25 >> [3] >> https://github.com/apache/karaf/blob/master/demos/ >> profiles/registry/src/main/resources/default.profile/version.cfg >> [4] >> http://docs.oracle.com/javase/7/docs/technotes/guides/io/ >> fsp/zipfilesystemprovider.html >> [5] https://github.com/gnodet/githubfs >> [6] >> https://github.com/gnodet/mina-sshd/commit/f68468c686099f6f12a9093e949c09 >> e16f618704 >> [7] https://github.com/google/jimfs >> [8] https://github.com/apache/karaf/blob/master/demos/profiles/static/ >> [9] >> https://github.com/apache/karaf/blob/master/demos/ >> profiles/static/src/main/resources/features.xml >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Apache Member Apache Karaf <http://karaf.apache.org/> Committer & PMC OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer & Project Lead blog <http://notizblog.nierbeck.de/> Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS> Software Architect / Project Manager / Scrum Master
