Hi Mark
let me do a last try to clarifiy my ideas, I am open to discuss things (and do
add some ideas at the end, so we can start the discussions right away ;) ):
1) my idea was to define a minimalistic interface for properties first. It
should as simple as possible to implement. Just using Map<String,String> I
found not optimal, since a Map requires several methods that are not easy to
implement in all cases, such as size(). Also Spring provides such a similarl
abstraction(PropertySource). For this I created PropertyProvider.
2) Looking at Configuration there is much more functionality needed to have a
powerful config API, such as type support (getInt, getBoolean, getAdapted),
registering of listeners, extension points (with, query). From our bank
additional methods were added for evaluation of the key ranges present,
assuming hereby a.b.d: a is an area, containing an area b containing a key d.
So I defined the Configuration interface.
3) Now there are very common types of properties sources you want to read from.
I personally identified: classpath resources, files and URLs/URIs. So I added
some kind of factory class (PropertyProviders), which allows to easily
construct them descriptively: PropertyProvider proc =
PropertyProviders.fromPath("classpath:META-INF/cfg/config.ini");
4) In many companies you do not have a fixed configuration file name, you will
have locations, where you consume all files (even recursively). So adding
resource loader support similar to Spring was compelling:
PropertyProvider proc =
PropertyProviders.fromPath("classpath:META-INF/cfg/**/*.ini");
5) Looking at what we have in Credit Suisse and what other prople told they
have in their companies, simple priority based adding of maps is not enough.
There are cases, where I want to filter out values, prevent overriding of
certain values etc. Also different locations (classpath, files, remote) are
mixed and depending on the source different priorities may be assigned. Finally
configuration pairs themselves can be attributed (default entries, global
default entries, explicit entries, ..), which also lead to different
configuration priorities within the same source: or even config file, e.g:
<config>
<defaults>
<a>aValue</a>
<b>bValue</b>
</deault>
<pluginConfig plugin="abc">
<a>aValue2</a>
</pluginConfig>
<server ip="1.2.4.5">
<b>bValue2</b>
</server>
</config>
Since I saw definitively the most complex scenarios I wanted to support power
users doing that kind of modelling more easily. One mechanism to achive this is
allowing to aggregate and combine maps (=providers), so
I added the combination methods on PropertyProviders like aggregate, intersect,
filter, ...
Now this seems more complicated for you guys than it seems for me ;) I can
imagine different things that may be simplified:
- Remove PropertyProvider/ PropertyProviders from the API part and just add
corresponding functionality using a ConfigurationBuilder:
Configuration config =
ConfigurationBuilder.create("myConfig").addPaths("ab/B/B",
"../fhgg.xml").with(AggregationPolicy.OVERRIDE).addPaths("jhbjh/kjhkjh/").build();
Config aggregatedConfig = config.with(Aggregators.from(config);
- We still could put the PropertyProvider to the core implementation module, or
even add it as a separate module, for the ones that like a more leightweight
implementation SPI ;)
Configuration conf = Configuration.current(); will still be the accessor method
of choice. Though I am well aware of the fact that the abstractions (SPIs)
required to evaluate the right configuration during runtime is not yet
discussed. Also the current state in the repo in that area is far away from
finished (I currently try to cleanup a few things, especially artifacts that
are duplicates...).
Best,
Anatole
-----Original Message-----
From: Mark Struberg [mailto:[email protected]]
Sent: Donnerstag, 4. Dezember 2014 11:08
To: [email protected]
Subject: Re: UC: Combining PropertyProvider Instances
I think I need a bit time to understand this. I've tried to grasp it for 2 days
now and this might mean that it is too complicated if not even I get it. Why
are we doing this so complicated? Pulling in all the database logic into a tool
which main target is that it should be really easy to use?
What benefit does this add over hiding those details? e.g. in DeltaSpike I
would just add a DatabaseConfigSource and be done. The user would not even see
that it magically picks up the values from the db then.
LieGrue,
strub
> On Tuesday, 2 December 2014, 14:02, Anatole Tresch <[email protected]> wrote:
> > Hi all
> find following an additional part of use cases for comment. Note that this
> area IMO definitively may be improved in several ways. Always keep in mind
> that I try to setup the design guide similarly in parallel. It can be
> easily accessed from the GH mirror:
>
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
>
> Cheers,
> Anatole
>
> Combining Property Providers
>
> Looking at the structures of configuration system used by large companies
> we typically encounter some kind of configuration hierarchies that are
> combined in arbitrary ways. Users of the systems are typically not aware of
> the complexities in this area, since they simply know the possible
> locations, formats and the overriding policies. Framework providers on the
> other side must face the complexities and it would be very useful if Tamaya
> can support here by providing prebuilt functionality that helps
> implementing these aspects. All this leads to the feature set of combining
> property providers. Hereby the following strategies are useful:
>
> -
>
> aggregating providers, hereby later providers added
> -
>
> override any existing entries from earlier providers
> -
>
> combine conflicting entries from earlier providers, e.g. into a
> comma-separated structure.
> -
>
> may throw a ConfigExcepotion ig entries are conflicting
> -
>
> may only add entries not yet defined by former providers, preventing
> entries that are already present to be overwritte
> -
>
> any custom aggregation strategy, which may be a mix of above
> -
>
> intersecting providers
> -
>
> subtracting providers
> -
>
> filtering providers
>
> These common functionality is provided by the PropertyProviders
> singleton. Additionally to the base strategies above a MetaInfo instance
> can be passed optionally as well to define the meta information for the
> newly created provider instances. Let’s assume we have two property
> providers with the following data:
> Provider 1
>
> a=a
> b=b
> c=c
> g=g
> h=h
> i=i
>
> Provider 2
>
> a=A
> b=B
> c=C
> d=D
> e=E
> f=F
>
> Looking in detail you see that the entries a,b,c are present in both
> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
> provider 2.
> Example Combining PropertyProviders
>
> PropertyProvider provider1 = ...
> PropertyProvider provider2 = ...
> // aggregate, hereby values from provider 2 override values from provider 1
> PropertyProvider unionOverriding =
> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> provider2);*System*.out.println("unionOverriding: " +
> unionOverriding);
> // ignore duplicates, values present in provider 1 are not overriden
> by provider 2
> PropertyProvider unionIgnoringDuplicates =
> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> + unionIgnoringDuplicates);
> // this variant combines/maps duplicate values into a new value
> PropertyProvider unionCombined =
> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> provider2);*System*.out.println("unionCombined: " + unionCombined);
> // This variant throws an exception since there are key/value paris in
> both providers, but with different values*try*{
> PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> provider1, provider2);
> }*catch*(ConfigException e){
> // expected!
> }
>
> The example above produces the following outpout:
> Example Combining PropertyProviders
>
> AggregatedPropertyProvider{
> (name = dynamicAggregationTests)
> a = "[a][A]"
> b = "[b][B]"
> c = "[c][C]"
> d = "[D]"
> e = "[E]"
> f = "[F]"
> g = "[g]"
> h = "[h]"
> i = "[i]"
> }
> unionOverriding: AggregatedPropertyProvider{
> (name = <noname>)
> a = "A"
> b = "B"
> c = "C"
> d = "D"
> e = "E"
> f = "F"
> g = "g"
> h = "h"
> i = "i"
> }
> unionIgnoringDuplicates: AggregatedPropertyProvider{
> (name = <noname>)
> a = "a"
> b = "b"
> c = "c"
> d = "D"
> e = "E"
> f = "F"
> g = "g"
> h = "h"
> i = "i"
> }
> unionCombined: AggregatedPropertyProvider{
> (name = <noname>)
> a = "a,A"
> b = "b,B"
> c = "c,C"
> d = "D"
> e = "E"
> f = "F"
> g = "g"
> h = "h"
> i = "i"
> }
>
> No AggregationPolicy is also an interface that can be implemented:
> AggregationPolicy Interface
>
> @FunctionalInterface*public* *interface* *AggregationPolicy* {
> *String* aggregate(*String* key, *String* value1, *String* value2);
> }
>
> So we can also define our own aggregation strategy using a Lambda
> expression:
> Use a Custom AggregationPolicy
>
> PropertyProvider provider1 = ...;
> PropertyProvider provider2 = ...;
> PropertyProvider props = PropertyProviders.aggregate(
> (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
> "]",
> MetaInfo.of("dynamicAggregationTests"),
> props1, props2);*System*.out.println(props);
>
> Additionally we also pass here an instance of MetaInfo. The output of
> this code snippet is as follows:
> Listing of dynamic aggregation policy
>
> AggregatedPropertyProvider{
> (name = dynamicAggregationTests)
> a = "[a][A]"
> b = "[b][B]"
> c = "[c][C]"
> d = "[D]"
> e = "[E]"
> f = "[F]"
> g = "[g]"
> h = "[h]"
> i = "[i]"
> }
>
> Summarizing the PropertyProviders singleton allows to combine providers
> in various forms:
> Methods provided on PropertyProviders
>
> public final class PropertyProviders {
>
> private PropertyProviders() {}
>
> public static PropertyProvider fromArgs(String... args) {
> public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> args)
> {
> public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, String... paths) {
> public static PropertyProvider fromPaths(String... paths) {
> public static PropertyProvider fromPaths(List<String> paths) {
> public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, List<String> paths) {
> public static PropertyProvider fromPaths(MetaInfo metaInfo,
> List<String> paths) {
> public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> public static PropertyProvider fromUris(URI... uris) {
> public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, URI... uris) {
> public static PropertyProvider fromUris(List<URI> uris) {
> public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, List<URI> uris) {
> public static PropertyProvider fromUris(MetaInfo metaInfo, URI... uris)
> {
> public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> uris) {
> public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> public static PropertyProvider fromMap(Map<String, String> map) {
> public static PropertyProvider fromMap(MetaInfo metaInfo,
> Map<String, String> map) {
> public static PropertyProvider empty() {
> public static PropertyProvider emptyMutable() {
> public static PropertyProvider empty(MetaInfo metaInfo) {
> public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> public static PropertyProvider fromEnvironmentProperties() {
> public static PropertyProvider fromSystemProperties() {
> public static PropertyProvider freezed(PropertyProvider provider) {
> public static PropertyProvider aggregate(AggregationPolicy
> mapping, MetaInfo metaInfo, PropertyProvider... providers){
> public static PropertyProvider aggregate(PropertyProvider... providers)
> {
> public static PropertyProvider aggregate(List<PropertyProvider>
> providers) {
> public static PropertyProvider aggregate(AggregationPolicy
> mapping, PropertyProvider... propertyMaps) {
> public static PropertyProvider aggregate(AggregationPolicy
> mapping, List<PropertyProvider> providers) {
> public static PropertyProvider mutable(PropertyProvider provider) {
> public static PropertyProvider intersected(AggregationPolicy
> aggregationPolicy, PropertyProvider... providers) {
> public static PropertyProvider intersected(PropertyProvider... providers)
> {
> public static PropertyProvider subtracted(PropertyProvider target,
> PropertyProvider... providers) {
> public static PropertyProvider filtered(Predicate<String> filter,
> PropertyProvider provider) {
> public static PropertyProvider
> contextual(Supplier<PropertyProvider> mapSupplier,
> Supplier<String>
> isolationKeySupplier) {
> public static PropertyProvider delegating(PropertyProvider
> mainMap, Map<String, String> parentMap) {
> public static PropertyProvider replacing(PropertyProvider mainMap,
> Map<String, String> replacementMap) {
> }
>
