Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Christian Schneider]

On 22.11.2016 21:19, Balázs Zsoldos wrote:


The focus is not on webconsole here or on any tools. Webconsole uses 
the current API and this is true for everything else that creates 
configuration at the moment. The other thing I wanted to highlight 
here: It is much more comfortable to create configuration with a 
running OSGi container. We can visualize that we do directly. We see 
if we made something wrong, we can have buttons to wire two 
configurations together and avoid "CopyPasteException"s. We have the 
metatype info and we can enter configuration in a typesafe way with 
validation.


On the other hand, writing configurations in files by hand and than 
deploy them (e.g.: with fileinstall) feels a bit like programming with 
notepad. It can be done, but painful after we have felt the benefit of 
using an IDE.
In my opinion there are benefits to both ways. Changing configs online 
can be very comfortable with good tooling.
On the other hand there are also systems where you want no online config 
changes (like http://martinfowler.com/bliki/ImmutableServer.html). Btw. 
this pattern works very nicely with the style you describe where config 
is stored in a special source control.


In this case the deployment needs to procure complete configs. Editing 
these configs will always happen offline. We still should be able to 
provide good IDE support for these configs too. As meta type information 
is normally provided in a static way it should be possible to have IDE 
support without starting the actual OSGi framework of the user application.
The new functions do not break compatibility. They create a more 
flexible way of creating factory configurations as finally it is said 
that we can trust the programmers as much that they can create unique 
keys within the scope of an application :). However, there will be two 
kind of factory configurations from now on. The old and limited one 
and the new one that is applicable in more use-cases (one of them can 
be distributed, but the other one not). Are you sure we want to have 
this distinction? It would be better to have one kind of factory 
configuration with more flexibility than now.
I think the main issue here is compatibility. Do you think it would be 
possible to have the new factory config style only while being 
compatible to old code?


Christian

--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Balázs Zsoldos]

Hi Raymond,

thanks for the answers. See my notes below with blue.

Regards
*Balázs*


On Tue, Nov 22, 2016 at 5:37 PM, Raymond Auge 
wrote:

> I can only answer to a small subset of your email.
>
> But perhaps we should have an answer for most of this from Carsten
> Ziegeler who drove most of the changes in the RFC.
>
> But please see my comments inline:
>
> On Mon, Nov 21, 2016 at 5:39 PM, Balázs Zsoldos  > wrote:
>
>> Hi Raymond (and others),
>>
>> I meant chapter 5.1 of RFC-227
>> .
>> I will try to write down the flow I went through after I read the spec, so
>> you might be able to tell me where I was wrong.
>>
>> When I first saw this chapter, I got really happy. I felt that finally,
>> one does not need workarounds to achieve the use-cases that I wrote about
>> in my last mail. However, I soon realized that deploying those
>> configurations are not 100% possible. It is not possible as I cannot deploy
>> existing configurations.
>>
>> Imagine the following steps:
>>
>>- I create some factory configurations with webconsole user
>>interface. The new configuration will be created with the existing
>>createFactoryConfiguration(factoryPid)
>>
>> 
>>function
>>
>> This is only because the current version of webconsole doesn't support
> the new version of the spec! After this version is released why couldn't
> webconsole support this feature?
>

I am aware of that. My point here was that I will be able to deploy
configurations live that are created with the new functions, but not the
ones that are created with the existing function. It feels like there are
two kinds of factory configuration from now on where one of them has a
limitation.


>
>
>>
>>- I save the configurations into my source code controller. Well,
>>that is Felix ConfigAdmin specific step :). Luckily Felix ConfigAdmin
>>stores the configs on the file system in the way that the configurations
>>are comparable by any diff tool (sorted and array values on new line).
>>Therefore storing them in git or SVN works pretty well
>>- Now if I want to apply new configuration (that was uploaded to
>>source control) without server restart, I will not be able to with the new
>>functions in ConfigurationAdmin, as the factory configurations were 
>> created
>>with the old function (that has a different syntax concerning to the Pid)
>>
>> I believe this is again in terms of old configurations which could not be
> used for any target which wants to support the new configurations.
>
>
>>
>> Must note that diffing configs are not only useful if you want to check
>> differences on source codes, but also if you want to check differences
>> between two servers where one works and other does not.
>>
>> The question in short: If I have existing configuration, can I somehow
>> deploy it to a server with the new functions?
>>
>> I asked this question on the felix mailing
>>  list
>> and based on the answer, it will not be possible.
>>
>
> But why would you do that? Won't you have to change the code anyway to
> take advantage of these new configurations? That's the goal here I
> believe... someone could correct me.
>

The configuration of our projects are in the source control system, so one
can check out the sources, build it and start the app from the IDE. After
the new functions, we will have the chance to deploy configuration changes
without restarting the OSGi container (started by the IDE). But only the
factory configurations that are created with the new functions.


>
>
>>
>> So I started to think. I tried put myself into the shoes of the ones who
>> wrote the RFC. I come up with new requirements always based on my
>> experiences. There are always aspects that I miss as I do not meet them at
>> work. I tried to find out, what is the experience and motivation behind the
>> new functions. I visited a couple of methods how people on the world use
>> ConfigurationAdmin with external tools and plugins. I found that I would
>> have written a very similar RFC if I used Felix FileInstall to deploy
>> configurations. Why?
>>
>> If I use Felix FileInstall to create my configs
>>
>>- I will have all my configs in a ConfigAdmin independent format in
>>the source code control and I edit these files by hand
>>- I will never go to the webconsole and create factory configurations
>>via the UI, so I will not have configurations that I cannot deploy to 
>> other
>>servers
>>- I can always delete all configurations from the folder where the
>>configAdmin bundle stores them, as I can recreate them from the format 
>> that
>>used by FileInstall. I will not feel the 

Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Raymond Auge]

I can only answer to a small subset of your email.

But perhaps we should have an answer for most of this from Carsten Ziegeler
who drove most of the changes in the RFC.

But please see my comments inline:

On Mon, Nov 21, 2016 at 5:39 PM, Balázs Zsoldos 
wrote:

> Hi Raymond (and others),
>
> I meant chapter 5.1 of RFC-227
> .
> I will try to write down the flow I went through after I read the spec, so
> you might be able to tell me where I was wrong.
>
> When I first saw this chapter, I got really happy. I felt that finally,
> one does not need workarounds to achieve the use-cases that I wrote about
> in my last mail. However, I soon realized that deploying those
> configurations are not 100% possible. It is not possible as I cannot deploy
> existing configurations.
>
> Imagine the following steps:
>
>- I create some factory configurations with webconsole user interface.
>The new configuration will be created with the existing
>createFactoryConfiguration(factoryPid)
>
> 
>function
>
> This is only because the current version of webconsole doesn't support the
new version of the spec! After this version is released why couldn't
webconsole support this feature?


>
>- I save the configurations into my source code controller. Well, that
>is Felix ConfigAdmin specific step :). Luckily Felix ConfigAdmin stores the
>configs on the file system in the way that the configurations are
>comparable by any diff tool (sorted and array values on new line).
>Therefore storing them in git or SVN works pretty well
>- Now if I want to apply new configuration (that was uploaded to
>source control) without server restart, I will not be able to with the new
>functions in ConfigurationAdmin, as the factory configurations were created
>with the old function (that has a different syntax concerning to the Pid)
>
> I believe this is again in terms of old configurations which could not be
used for any target which wants to support the new configurations.


>
> Must note that diffing configs are not only useful if you want to check
> differences on source codes, but also if you want to check differences
> between two servers where one works and other does not.
>
> The question in short: If I have existing configuration, can I somehow
> deploy it to a server with the new functions?
>
> I asked this question on the felix mailing
>  list
> and based on the answer, it will not be possible.
>

But why would you do that? Won't you have to change the code anyway to take
advantage of these new configurations? That's the goal here I believe...
someone could correct me.


>
> So I started to think. I tried put myself into the shoes of the ones who
> wrote the RFC. I come up with new requirements always based on my
> experiences. There are always aspects that I miss as I do not meet them at
> work. I tried to find out, what is the experience and motivation behind the
> new functions. I visited a couple of methods how people on the world use
> ConfigurationAdmin with external tools and plugins. I found that I would
> have written a very similar RFC if I used Felix FileInstall to deploy
> configurations. Why?
>
> If I use Felix FileInstall to create my configs
>
>- I will have all my configs in a ConfigAdmin independent format in
>the source code control and I edit these files by hand
>- I will never go to the webconsole and create factory configurations
>via the UI, so I will not have configurations that I cannot deploy to other
>servers
>- I can always delete all configurations from the folder where the
>configAdmin bundle stores them, as I can recreate them from the format that
>used by FileInstall. I will not feel the issue if I cannot re-create
>configurations created by the old createFactoryConfiguration... function
>- FileInstall has files with the following name format: *factory-pid*.
>What if I just change the colon to some special character like hashmark?
>Why not dot? Because Felix ConfigAdmin implementation already uses that
>character.
>
> As I said, if I used Felix FileInstall in the last years, I would come to
> the same conclusion and wrote the same RFC. The writers might had a
> different motivation and experience, but in that case I would be really
> interested in it.
>

I don't believe this was the key motivation! At least partially it was more
about "human recognizable association" AND at least for me "foreign key
relationships"

Let me explain both things:

*"human recognizable association"*: I want to have a name for an instance
of a configuration in a factory which I can recognize with my eyes
  *e.g.* I have 4 

Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Balázs Zsoldos]

Hi Raymond (and others),

I meant chapter 5.1 of RFC-227
.
I will try to write down the flow I went through after I read the spec, so
you might be able to tell me where I was wrong.

When I first saw this chapter, I got really happy. I felt that finally, one
does not need workarounds to achieve the use-cases that I wrote about in my
last mail. However, I soon realized that deploying those configurations are
not 100% possible. It is not possible as I cannot deploy existing
configurations.

Imagine the following steps:

   - I create some factory configurations with webconsole user interface.
   The new configuration will be created with the existing
   createFactoryConfiguration(factoryPid)
   

   function
   - I save the configurations into my source code controller. Well, that
   is Felix ConfigAdmin specific step :). Luckily Felix ConfigAdmin stores the
   configs on the file system in the way that the configurations are
   comparable by any diff tool (sorted and array values on new line).
   Therefore storing them in git or SVN works pretty well
   - Now if I want to apply new configuration (that was uploaded to source
   control) without server restart, I will not be able to with the new
   functions in ConfigurationAdmin, as the factory configurations were created
   with the old function (that has a different syntax concerning to the Pid)


Must note that diffing configs are not only useful if you want to check
differences on source codes, but also if you want to check differences
between two servers where one works and other does not.

The question in short: If I have existing configuration, can I somehow
deploy it to a server with the new functions?

I asked this question on the felix mailing
 list
and based on the answer, it will not be possible.

So I started to think. I tried put myself into the shoes of the ones who
wrote the RFC. I come up with new requirements always based on my
experiences. There are always aspects that I miss as I do not meet them at
work. I tried to find out, what is the experience and motivation behind the
new functions. I visited a couple of methods how people on the world use
ConfigurationAdmin with external tools and plugins. I found that I would
have written a very similar RFC if I used Felix FileInstall to deploy
configurations. Why?

If I use Felix FileInstall to create my configs

   - I will have all my configs in a ConfigAdmin independent format in the
   source code control and I edit these files by hand
   - I will never go to the webconsole and create factory configurations
   via the UI, so I will not have configurations that I cannot deploy to other
   servers
   - I can always delete all configurations from the folder where the
   configAdmin bundle stores them, as I can recreate them from the format that
   used by FileInstall. I will not feel the issue if I cannot re-create
   configurations created by the old createFactoryConfiguration... function
   - FileInstall has files with the following name format: *factory-pid*.
   What if I just change the colon to some special character like hashmark?
   Why not dot? Because Felix ConfigAdmin implementation already uses that
   character.

As I said, if I used Felix FileInstall in the last years, I would come to
the same conclusion and wrote the same RFC. The writers might had a
different motivation and experience, but in that case I would be really
interested in it.

I feel that RFCs are only meaningful if experience and motivation is shared
behind the content. I am missing this at the moment. Only functional
requirements are written, that a spec should be able to do that. But it is
not written there, why, so it is hard to decide if something might be
missing. I would be very thankful if the people who write the RFCs would
sacrifice their time not only to write RFC but also some pages about
the use-cases they want to solve.

My motivation is the following (just to see it as a short example):

   - I like building a component graph on a running OSGi container. I see
   rapidly the result of changing some config. I can build a component tree
   from top to down and see how it looks like.
   - To build component graph live, I use webconsole and therefore the
   ConfigurationAdmin.createFactoryConfiguration function is used for factory
   configs.
   - I want to export the configurations that I created on the live living
   OSGi container and reuse it in unit tests, dev, test or production servers.
   At the moment, I simply copy the folder where Felix ConfigAdmin stores the
   configs to the source control
   - In the future, I would like to implement some Eclipse plugin
   extensions that live updates the running OSGi container with the 

Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Raymond Auge]

Furthermore, it's quite rare (I've never seen it) in my limited experience
than an OSGi spec is directly influenced by a particular implementation in
a way that "since this impl does it... so should the spec". I assure you
_at least_ BJ would lose his mind if we tried to do this.

Sincerely,
- Ray

On Mon, Nov 21, 2016 at 2:26 PM, Raymond Auge 
wrote:

> Would you mind referring to the section you are referring to specifically
> by number and paragraph number just so we can better orient ourselves? We
> don't read these every day so we often forget what's there.
>
> :)
>
> Sincerely,
> - Ray
>
> On Mon, Nov 21, 2016 at 2:21 PM, Balázs Zsoldos  > wrote:
>
>> Hi,
>>
>> I read about the new features of ConfigurationAdmin that will be applied
>> in the new OSGi release. After checking the new functions, I feel that I
>> might miss the point.
>>
>> What I feel that the new functions have no other purpose but satisfying
>> some of the requirements of Apache Felix FileInstall.
>>
>> It is possible to create factory config instances, but it is not possible
>> to do it with any kind of pid. That somehow does not make sense to me. Why
>> not all of them just some of them?
>>
>> Why is the special character the hashmark? Is it because Felix
>> ConfigAdmin uses a dot at the moment? What about other implementations of
>> the specifications? Or the new versions of specifications are written based
>> on the current implementation specific details? Even if almost everybody
>> uses the Felix implementation.
>>
>> I think that a universal solution would help much more projects, not only
>> Config Admin. By an universal solution, I mean: Possibility to deploy
>> configurations with pre-defined pids. This could be a help for FileInstall,
>> too.
>>
>> With a universal solution, use-cases could be solved without workaround
>> like:
>>
>>-  Deployment of configurations from source-control at development
>>time: Imagine that we use an IDE that supports starting OSGi containers,
>>keep the config with the sources and update configuration without
>>restarting the OSGi container
>>- Deployment of configuration to several computers in a clustered
>>environment
>>
>>
>> What do you think? Am I missing the point?
>>
>> *Balázs **Zsoldos*
>>
>>
>> ___
>> OSGi Developer Mail List
>> osgi-dev@mail.osgi.org
>> https://mail.osgi.org/mailman/listinfo/osgi-dev
>>
>
>
>
> --
> *Raymond Augé* 
>  (@rotty3000)
> Senior Software Architect *Liferay, Inc.* 
>  (@Liferay)
> Board Member & EEG Co-Chair, OSGi Alliance 
> (@OSGiAlliance)
>



-- 
*Raymond Augé* 
 (@rotty3000)
Senior Software Architect *Liferay, Inc.* 
 (@Liferay)
Board Member & EEG Co-Chair, OSGi Alliance  (@OSGiAlliance)
___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Universal purpose of Configuration Admin new functions

[Raymond Auge]

Would you mind referring to the section you are referring to specifically
by number and paragraph number just so we can better orient ourselves? We
don't read these every day so we often forget what's there.

:)

Sincerely,
- Ray

On Mon, Nov 21, 2016 at 2:21 PM, Balázs Zsoldos 
wrote:

> Hi,
>
> I read about the new features of ConfigurationAdmin that will be applied
> in the new OSGi release. After checking the new functions, I feel that I
> might miss the point.
>
> What I feel that the new functions have no other purpose but satisfying
> some of the requirements of Apache Felix FileInstall.
>
> It is possible to create factory config instances, but it is not possible
> to do it with any kind of pid. That somehow does not make sense to me. Why
> not all of them just some of them?
>
> Why is the special character the hashmark? Is it because Felix ConfigAdmin
> uses a dot at the moment? What about other implementations of the
> specifications? Or the new versions of specifications are written based on
> the current implementation specific details? Even if almost everybody uses
> the Felix implementation.
>
> I think that a universal solution would help much more projects, not only
> Config Admin. By an universal solution, I mean: Possibility to deploy
> configurations with pre-defined pids. This could be a help for FileInstall,
> too.
>
> With a universal solution, use-cases could be solved without workaround
> like:
>
>-  Deployment of configurations from source-control at development
>time: Imagine that we use an IDE that supports starting OSGi containers,
>keep the config with the sources and update configuration without
>restarting the OSGi container
>- Deployment of configuration to several computers in a clustered
>environment
>
>
> What do you think? Am I missing the point?
>
> *Balázs **Zsoldos*
>
>
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev
>



-- 
*Raymond Augé* 
 (@rotty3000)
Senior Software Architect *Liferay, Inc.* 
 (@Liferay)
Board Member & EEG Co-Chair, OSGi Alliance  (@OSGiAlliance)
___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev