Hi guys,
following Guillaume work on the profile, I have an extended proposal
that I want to discuss with you.
The Karaf profiles have to be stored somewhere: on github/git for
instance, in order to be applied on a distribution.
For instance, what I tested is the karaf docker.io feature with
profiles: it works fine, the docker.io feature bootstraps a docker image
with Karaf, and I applied a profile on this Karaf instance.
A storage is also needed for the docker.io feature: where the images are
stored (basically a docker.io github like).
On a private infrastructure, companies don't want to use "external"
storage (like github): they want to use first their "local" storages.
It's also valid for hybrid infrastructure: their want to use their local
storage first (as master), and possible proxy external storages.
Right now, we have Karaf Cave providing:
- Maven repositories management (stored in Cave or mirroring/proxying
external repositories)
- OBR server management
- Feature Enterprise Repositories management
Cave is especially interesting with Cellar where all cluster nodes share
Cave repositories.
In order to have centralised storage (and possibly replicated storage),
I wonder if we can not extend Cave to provide:
- Karaf profiles repositories
- Docker.io images repositories
It could work like for Maven repository, meaning local storage or
mirror/proxy to another one.
Like this, we could have Karaf and shell commands to bootstrap Karaf
docker.io images, or apply profiles on an instance.
WDYT ?
Regards
JB
On 12/11/2014 09:04 AM, Guillaume Nodet wrote:
I've just committed to the 4.x branch two new features:
* profiles
* static distributions
Profiles
======
A profile is a data structure that brings together configuration and
provisioning, so it's related to features, but is quite different. A demo
is available at [1].
A profile contains a list of configuration files which can be either
properties based or in any other format. Properties based configurations
are handled in a very specific way, as they can overlay each other (more
below).
One of this properties based configuration holds provisioning informations
along with profiles specific attributes, such as a list of parent profiles,
a list of bundles, a list of features repositories and a list of features.
A profile can have one or more parents, so the hierarchy is an inverted
tree. This allows to define generic profiles and specialise them in
children.
Overlay profile: at some point, there is a need to flatten this hierarchy
into an "overlay" profile. This process is done by walking into the
parents hierarchy (depth first) and computing the resulting configuration
for each configuration file. Non properties configuration files simply
overwrite each other, while properties based configuration complement each
other (with the ability to clear a single value or all values).
Effective profile: properties configurations can contain placeholders to
resolve. The only one defined atm is a property placeholder pointing to a
value in a different configuration. Those placeholders are resolved on the
overlay profile to give the end result effective profile which will be
actually used. An example of using this placeholder is shown in [2] which
end up pointing to the configuration in its parent [3]
Provisioning: each profile contains a list of feature repositories,
features and bundles. Those information will be used in the effective
profile to know which bundles and features are needed on a given instance.
Profiles are not used at runtime for the moment and there are 2 main
usage : creating child instances and generating distributions using the
maven plugin.
Profile manipulation : profiles are manipulated using the Profiles class
(static helpers) and the ProfileBuilder (to build profile instances, which
are immutable). Reading and writing profiles is done using the jdk7 in
FileSystem / Path interface, which obviously provides support for the
standard file system, but could be used with various file systems (inside a
zip [4], readonly github remote repository [5], sftp [6], in memory [7].
Those are the ones I know about). A set of commands can be used to edit
the profiles.
Further considerations: possible improvements include an overlay mechanism
for xml, integration with cellar, file systems using a real git repo,
hazelcast, zookeeper, etc... Applying profiles at runtime could be done
too. Better integration with kars could be provided too.
Static distributions
==============
One use case of profiles is to generate distributions. Changes on the
profile need a rebuild on the distribution (this may or may not be a
problem, depending on your environment, but if it can seems weird in an
OSGi world, it's not so much in a cloud environment). This leads to the
distribution being mostly static, i.e. the user should not manually install
/ uninstall stuff.
A demo is available at [8]
Based on this idea, I improved the maven plugin a bit to allow generating
mostly static distribution : the profiles / features are all installed in
etc/startup.properties. A simplistic read-only configuration admin is used
that will pick up configuration in the etc/ folder directly. This lead to
no runtime dependencies for the provisioning of karaf itself : fileinstall,
the features service can all be removed. The bare framework can only
contain 3 bundles : pax-logging + the static configadmin [9]. One
improvement is that the distribution is generated using reference:file:
urls in startup.properties, avoiding a copy of each jar into the cache. An
additional improvement could be to pre-compute the wiring which can take
some time (but this is not supported by felix).
Thoughts welcomed !
[1] https://github.com/apache/karaf/tree/master/demos/profiles
[2]
https://github.com/apache/karaf/blob/master/demos/profiles/registry/src/main/resources/karaf.profile/profile.cfg#L25
[3]
https://github.com/apache/karaf/blob/master/demos/profiles/registry/src/main/resources/default.profile/version.cfg
[4]
http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
[5] https://github.com/gnodet/githubfs
[6]
https://github.com/gnodet/mina-sshd/commit/f68468c686099f6f12a9093e949c09e16f618704
[7] https://github.com/google/jimfs
[8] https://github.com/apache/karaf/blob/master/demos/profiles/static/
[9]
https://github.com/apache/karaf/blob/master/demos/profiles/static/src/main/resources/features.xml
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
