Re: [osgi-dev] Double config

2018-08-26 Thread David Leangen via osgi-dev

Hi Neil,

Thanks for the tips.

> With regard to "testing bouncing", there are two sides to this question: the 
> component being bounced, and the other components that may be exposed to it 
> bouncing.
> 
> The component itself should never actually know that it is being, or has 
> been, bounced. SCR always creates a new instance for each activation…

Yeah, that was one of my learnings during this iteration. The simplicity of 
using a static component cannot be understated. By removing as much threading 
and dynamic components as possible, it makes the system **MUCH** more easy to 
reason, and at least in my case with no perceptible performance loss.


> The harder question is about components that deliberately expose themselves 
> to the bouncing of other components – in other words, they have dynamic 
> service references – and/or expose themselves to dynamic configuration 
> changes by having a modified method. Tim talked about components that require 
> heavyweight initialisation... to implement such a component you might write a 
> modify method to accept dynamic config changes and then internally implement 
> some kind of damping mechanism. This gets you into some slightly hairy 
> multithreading code.

Yeah… I’m trying to avoid that when possible.


> I don't have a good answer myself yet for testing this kind of code, it seems 
> like even the best available practices are still quite primitive and 
> probabilistic, e.g. running the same code thousands of times and hoping to 
> find an error. I would love to find something better.

I get the impression that the only thing to do is reason about the code. I am 
finding that there are two places where there is potential for things to go 
wrong: (1) concurrency due to “regular” multithreading and (2) when using 
promises. Concurrency is a little easier to reason, at least for me. I have 
tried to limit exposure to these risks as much as possible. I think I have this 
pretty much under control, but since I am relying on my ability to reason and 
don’t have a conclusive means of testing, I have no guarantee that I have not 
reasoned incorrectly.

As for promises, the bigger challenge is protecting against services that come 
and go. I am finding that dealing with these dynamics is more challenging than 
concurrency. However, by reducing multithreading (accepting synchronous 
processing as much as possible instead of worrying too much in advance about 
performance), I have greatly reduced these risks and, at least for now, the 
timing problems I was having. Although I understand that it is not good 
practice to linger too long in, for example, an activate method, by moving back 
“long” processes (in the order of a second or so) synchronously into the same 
thread, I have greatly simplified my system, at the expense of a startup time 
that is about 5-10 seconds slower. I can certainly live with that.

KISS


Thanks again.
=David


___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Double config

2018-08-26 Thread David Leangen via osgi-dev


>> Unless maybe there is a bug in SCR for some edge case (felix version 2.0.2).
> 
> This is indeed my suspicion based on what you have described. Have you tried 
> using a later SCR implementation, just in case it's a bug that's already been 
> fixed?

Thanks, Neil! I updated to 2.0.14, and the problem has gone away.

As an extra bonus, there is no more bouncing, either. I could never figure out 
why the system was bouncing, so I guess this explains it. :-)


Cheers,
=David

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev


Re: [osgi-dev] Double config

2018-08-26 Thread Neil Bartlett via osgi-dev
On Sun, Aug 26, 2018 at 1:35 AM David Leangen via osgi-dev <
osgi-dev@mail.osgi.org> wrote:

>
> Hi,
>
> Thank you again for all the great tips in this thread. I have taken the
> time to review concurrency, reread all the blogs, articles, and email
> messages in detail, and I have done a fairly thorough review of all my
> code. I have identified a lot of places where there was potential for
> concurrency errors, and I made a lot of simplifications (mostly by
> eliminating any clever use of concurrency or synchronization when it wasn’t
> really providing any tangible benefit, and only introducing risk). I also
> reviewed again the DS lifecycle, as well as Trackers. I am pretty confident
> that my understanding of the concurrent issues, and consequently my system,
> are now in good shape. I think I can handle the dynamics now in a robust
> way (though I still don’t know how to test for it to confirm).
>
> However, I am experiencing one problem on startup still that baffles me.
>
> Take this component:
>
>
> @Component(
> factory = ...,
> configurationPid = ...,
> configurationPolicy = ConfigurationPolicy.REQUIRE)
> public class D
> implements Descriptor
> {
> @Reference( scope = ReferenceScope.PROTOTYPE_REQUIRED )
> private WriterFactory.JsonWriterFactory factory;
>
> @Activate
> void activate( Configuration.Config config )
> {
>   ...
> }
>
> ...
> }
>
>
> There is really nothing more to the component that this. One static
> reference (which is satisfied), and a required config (which is available).
>
> The factory basically looks like this (called from a different component
> that instantiates this tracker):
>
>
> public class ComponentFactoryTracker
> extends ServiceTracker
> {
> public ConfinementAwareComponentFactoryTracker( BundleContext context,
> Filter filter )
> {
> super( context, filter, null );
> }
>
> @Override
> public ComponentInstance addingService(
> ServiceReference reference )
> {
> final ComponentFactory cf = context.getService( reference );
> final Dictionary properties = new Hashtable<>();
> properties.put( ... );
> final ComponentInstance instance = cf.newInstance( properties );
> return instance;
> }
>
> @Override
> public void removedService( ServiceReference
> reference, ComponentInstance instance )
> {
> context.ungetService( reference );
> instance.dispose();
> }
> }
>
> Fairly simple.
>
> The system starts up, and the D component (and others like it) also
> starts. Then bouncing happens. When things finally settle down after one
> bounce, D (and others like it) gets deactivated. This is where I start
> pulling my hair out. **Why** does it get deactivated???
>

I can't explain this either. The only suggestions I can make are:

1) Write a deactivate method, stick a breakpoint on it and look at the
stack. That might give you a clue as to why and how you got there.
2) Create a demonstrator project and raise a bug against the SCR
implementation.


>
>  —> The one and only static reference is available. Check.
>
>  —> The configuration is available. Check.
>
>  —> The  ComponentFactory services are enabled. Check.
>
> However, despite all the prerequisites being available, D (and others) are
> getting deactivated. I even created an immediate service that references D
> (and others) to ensure that somebody was calling it. Nothing. Nada. All
> that I can validate is that as D (and others) get deactivated, they are
> unbound from this test service.
>
> If I restart any bundle that causes D to refresh, then the state of the
> system comes up correctly. It is only upon initial startup that this
> problem occurs.
>
> If I try injecting some service that potentially changes startup order,
> sometimes it works, but I have not yet figured out the pattern. In any
> case, since the prerequisites for the service are available, I don’t see
> why on Earth that should matter, anyway. Unless maybe there is a bug in SCR
> for some edge case (felix version 2.0.2).
>

This is indeed my suspicion based on what you have described. Have you
tried using a later SCR implementation, just in case it's a bug that's
already been fixed?

Neil


>
>
> I am at a loss...
>
>
> Cheers,
> =David
>
>
>
> On Jul 14, 2018, at 4:05, David Leangen via osgi-dev <
> osgi-dev@mail.osgi.org> wrote:
>
>
> Hi Tim,
>
> What is a good way to test for robustness against this phenomenon?
>
>
> Again, I wish to go on record as saying that bouncing does not mean that
> anything is wrong, nor is it intrinsically bad.
>
>
> Thanks. Understood. I did not mean to imply anything about “goodness” or
> “badness”, but it is good to have that on record.
>
> So, my question was: knowing that this happens, is there a good way to
> test against it? My understanding is that currently there is not.
>
>
> Cheers,
> =David
>
> ___
> OSGi 

Re: [osgi-dev] Double config

2018-08-26 Thread Neil Bartlett via osgi-dev
With regard to "testing bouncing", there are two sides to this question:
the component being bounced, and the other components that may be exposed
to it bouncing.

The component itself should never actually know that it is being, or has
been, bounced. SCR always creates a new instance for each activation, so
unless you are storing state in static fields (hopefully you don't need me
to tell you that would be a bad idea!) your component only needs to deal
with a straightforward start/stop lifecycle. For example when it starts, it
may as well be the only time it has ever started. If you have specified
optional configuration or any optional service references then you need to
deal with the possibility that config/services may be null, and you should
write unit tests for all combinations of config/services being unbound and
unbound. Obviously you don't want too many optional references because the
number of required test cases rises exponentially.

The harder question is about components that deliberately expose themselves
to the bouncing of other components – in other words, they have dynamic
service references – and/or expose themselves to dynamic configuration
changes by having a modified method. Tim talked about components that
require heavyweight initialisation... to implement such a component you
might write a modify method to accept dynamic config changes and then
internally implement some kind of damping mechanism. This gets you into
some slightly hairy multithreading code. I don't have a good answer myself
yet for testing this kind of code, it seems like even the best available
practices are still quite primitive and probabilistic, e.g. running the
same code thousands of times and hoping to find an error. I would love to
find something better.

Hope this helps,
Neil



On Fri, Jul 13, 2018 at 8:05 PM David Leangen via osgi-dev <
osgi-dev@mail.osgi.org> wrote:

>
> Hi Tim,
>
> >> What is a good way to test for robustness against this phenomenon?
> >
> > Again, I wish to go on record as saying that bouncing does not mean that
> anything is wrong, nor is it intrinsically bad.
>
> Thanks. Understood. I did not mean to imply anything about “goodness” or
> “badness”, but it is good to have that on record.
>
> So, my question was: knowing that this happens, is there a good way to
> test against it? My understanding is that currently there is not.
>
>
> Cheers,
> =David
>
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev
___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Double config

2018-08-26 Thread Alain Picard via osgi-dev
David,

In my own journey I have had (and still does have) my share of moments
where I'm not sure what's happening. We are still on Equinox/PDE (can't
wait to switch to BND but that will have to wait a bit), and there I have
used the tracing to get insight into the service registry events. With
Equinox you can turn tracing under o.e.osgi. The keys are debug and
debug/services. This activates the tracing from Felix SCR, so that is not
limited to Equinox, but I'm not sure how you would activate it if not using
Equinox.

HTH,
Alain


On Sat, Aug 25, 2018 at 8:35 PM David Leangen via osgi-dev <
osgi-dev@mail.osgi.org> wrote:

>
> Hi,
>
> Thank you again for all the great tips in this thread. I have taken the
> time to review concurrency, reread all the blogs, articles, and email
> messages in detail, and I have done a fairly thorough review of all my
> code. I have identified a lot of places where there was potential for
> concurrency errors, and I made a lot of simplifications (mostly by
> eliminating any clever use of concurrency or synchronization when it wasn’t
> really providing any tangible benefit, and only introducing risk). I also
> reviewed again the DS lifecycle, as well as Trackers. I am pretty confident
> that my understanding of the concurrent issues, and consequently my system,
> are now in good shape. I think I can handle the dynamics now in a robust
> way (though I still don’t know how to test for it to confirm).
>
> However, I am experiencing one problem on startup still that baffles me.
>
> Take this component:
>
>
> @Component(
> factory = ...,
> configurationPid = ...,
> configurationPolicy = ConfigurationPolicy.REQUIRE)
> public class D
> implements Descriptor
> {
> @Reference( scope = ReferenceScope.PROTOTYPE_REQUIRED )
> private WriterFactory.JsonWriterFactory factory;
>
> @Activate
> void activate( Configuration.Config config )
> {
>   ...
> }
>
> ...
> }
>
>
> There is really nothing more to the component that this. One static
> reference (which is satisfied), and a required config (which is available).
>
> The factory basically looks like this (called from a different component
> that instantiates this tracker):
>
>
> public class ComponentFactoryTracker
> extends ServiceTracker
> {
> public ConfinementAwareComponentFactoryTracker( BundleContext context,
> Filter filter )
> {
> super( context, filter, null );
> }
>
> @Override
> public ComponentInstance addingService(
> ServiceReference reference )
> {
> final ComponentFactory cf = context.getService( reference );
> final Dictionary properties = new Hashtable<>();
> properties.put( ... );
> final ComponentInstance instance = cf.newInstance( properties );
> return instance;
> }
>
> @Override
> public void removedService( ServiceReference
> reference, ComponentInstance instance )
> {
> context.ungetService( reference );
> instance.dispose();
> }
> }
>
> Fairly simple.
>
> The system starts up, and the D component (and others like it) also
> starts. Then bouncing happens. When things finally settle down after one
> bounce, D (and others like it) gets deactivated. This is where I start
> pulling my hair out. **Why** does it get deactivated???
>
>  —> The one and only static reference is available. Check.
>
>  —> The configuration is available. Check.
>
>  —> The  ComponentFactory services are enabled. Check.
>
> However, despite all the prerequisites being available, D (and others) are
> getting deactivated. I even created an immediate service that references D
> (and others) to ensure that somebody was calling it. Nothing. Nada. All
> that I can validate is that as D (and others) get deactivated, they are
> unbound from this test service.
>
> If I restart any bundle that causes D to refresh, then the state of the
> system comes up correctly. It is only upon initial startup that this
> problem occurs.
>
> If I try injecting some service that potentially changes startup order,
> sometimes it works, but I have not yet figured out the pattern. In any
> case, since the prerequisites for the service are available, I don’t see
> why on Earth that should matter, anyway. Unless maybe there is a bug in SCR
> for some edge case (felix version 2.0.2).
>
>
> I am at a loss...
>
>
> Cheers,
> =David
>
>
>
> On Jul 14, 2018, at 4:05, David Leangen via osgi-dev <
> osgi-dev@mail.osgi.org> wrote:
>
>
> Hi Tim,
>
> What is a good way to test for robustness against this phenomenon?
>
>
> Again, I wish to go on record as saying that bouncing does not mean that
> anything is wrong, nor is it intrinsically bad.
>
>
> Thanks. Understood. I did not mean to imply anything about “goodness” or
> “badness”, but it is good to have that on record.
>
> So, my question was: knowing that this happens, is there a good way to
> test against it? My understanding is that currently there is not.
>
>
> 

Re: [osgi-dev] Double config

2018-08-25 Thread David Leangen via osgi-dev

Hi,

Thank you again for all the great tips in this thread. I have taken the time to 
review concurrency, reread all the blogs, articles, and email messages in 
detail, and I have done a fairly thorough review of all my code. I have 
identified a lot of places where there was potential for concurrency errors, 
and I made a lot of simplifications (mostly by eliminating any clever use of 
concurrency or synchronization when it wasn’t really providing any tangible 
benefit, and only introducing risk). I also reviewed again the DS lifecycle, as 
well as Trackers. I am pretty confident that my understanding of the concurrent 
issues, and consequently my system, are now in good shape. I think I can handle 
the dynamics now in a robust way (though I still don’t know how to test for it 
to confirm).

However, I am experiencing one problem on startup still that baffles me.

Take this component:


@Component(
factory = ...,
configurationPid = ...,
configurationPolicy = ConfigurationPolicy.REQUIRE)
public class D
implements Descriptor
{
@Reference( scope = ReferenceScope.PROTOTYPE_REQUIRED )
private WriterFactory.JsonWriterFactory factory;

@Activate
void activate( Configuration.Config config )
{
  ...
}

...
}


There is really nothing more to the component that this. One static reference 
(which is satisfied), and a required config (which is available).

The factory basically looks like this (called from a different component that 
instantiates this tracker):


public class ComponentFactoryTracker
extends ServiceTracker
{
public ConfinementAwareComponentFactoryTracker( BundleContext context, 
Filter filter )
{
super( context, filter, null );
}

@Override
public ComponentInstance addingService( ServiceReference 
reference )
{
final ComponentFactory cf = context.getService( reference );
final Dictionary properties = new Hashtable<>();
properties.put( ... );
final ComponentInstance instance = cf.newInstance( properties );
return instance;
}

@Override
public void removedService( ServiceReference reference, 
ComponentInstance instance )
{
context.ungetService( reference );
instance.dispose();
}
}

Fairly simple.

The system starts up, and the D component (and others like it) also starts. 
Then bouncing happens. When things finally settle down after one bounce, D (and 
others like it) gets deactivated. This is where I start pulling my hair out. 
**Why** does it get deactivated???

 —> The one and only static reference is available. Check.

 —> The configuration is available. Check.

 —> The  ComponentFactory services are enabled. Check.

However, despite all the prerequisites being available, D (and others) are 
getting deactivated. I even created an immediate service that references D (and 
others) to ensure that somebody was calling it. Nothing. Nada. All that I can 
validate is that as D (and others) get deactivated, they are unbound from this 
test service.

If I restart any bundle that causes D to refresh, then the state of the system 
comes up correctly. It is only upon initial startup that this problem occurs.

If I try injecting some service that potentially changes startup order, 
sometimes it works, but I have not yet figured out the pattern. In any case, 
since the prerequisites for the service are available, I don’t see why on Earth 
that should matter, anyway. Unless maybe there is a bug in SCR for some edge 
case (felix version 2.0.2).


I am at a loss...


Cheers,
=David



> On Jul 14, 2018, at 4:05, David Leangen via osgi-dev  
> wrote:
> 
> 
> Hi Tim,
> 
>>> What is a good way to test for robustness against this phenomenon?
>> 
>> Again, I wish to go on record as saying that bouncing does not mean that 
>> anything is wrong, nor is it intrinsically bad.
> 
> Thanks. Understood. I did not mean to imply anything about “goodness” or 
> “badness”, but it is good to have that on record.
> 
> So, my question was: knowing that this happens, is there a good way to test 
> against it? My understanding is that currently there is not.
> 
> 
> Cheers,
> =David
> 
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Double config

2018-07-13 Thread David Leangen via osgi-dev

Hi Tim,

>> What is a good way to test for robustness against this phenomenon?
> 
> Again, I wish to go on record as saying that bouncing does not mean that 
> anything is wrong, nor is it intrinsically bad.

Thanks. Understood. I did not mean to imply anything about “goodness” or 
“badness”, but it is good to have that on record.

So, my question was: knowing that this happens, is there a good way to test 
against it? My understanding is that currently there is not.


Cheers,
=David

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Double config

2018-07-13 Thread Tim Ward via osgi-dev
www.bosch.com <http://www.bosch.com/> 
>>>> Tel. +49 7153 666-1155 | dirk.fa...@de.bosch.com 
>>>> <mailto:dirk.fa...@de.bosch.com> 
>>>> 
>>>> Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
>>>> Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar 
>>>> Denner,
>>>> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Rolf Bulander, 
>>>> Dr. Stefan Hartung, Dr. Markus Heyn,
>>>> Dr. Dirk Hoheisel, Christoph Kübel, Uwe Raschke, Peter Tyroller 
>>>> 
>>>> 
>>>> Von: osgi-dev-boun...@mail.osgi.org 
>>>> <mailto:osgi-dev-boun...@mail.osgi.org> 
>>>> [mailto:osgi-dev-boun...@mail.osgi.org 
>>>> <mailto:osgi-dev-boun...@mail.osgi.org>] Im Auftrag von Tim Ward via 
>>>> osgi-dev
>>>> Gesendet: Donnerstag, 12. Juli 2018 10:22
>>>> An: Peter Kriens mailto:peter.kri...@aqute.biz>>; 
>>>> OSGi Developer Mail List >>> <mailto:osgi-dev@mail.osgi.org>>
>>>> Betreff: Re: [osgi-dev] Double config
>>>>  
>>>> Hi David,
>>>> 
>>>> 
>>>> This is a gift! :-) It means your code is not handling the dynamics 
>>>> correctly and now you know it! 
>>>>  
>>>> I’m not sure that there’s quite enough evidence to come to this 
>>>> conclusion. It could simply be that everything is working fine with a 
>>>> static policy, and working the way that DS is supposed to. In general a DS 
>>>> component being bounced like this is nothing to worry about, it’s just a 
>>>> change rippling through the system, and it’s typically resolved in very 
>>>> short order. If you do want to reason about why, however, then there are 
>>>> several places to look.
>>>>  
>>>> For some reason, the Component gets activated, deactivated, then activated 
>>>> again, which is not desirable.
>>>> ...
>>>> 1. How can I figure out why this is happening. I have tried many 
>>>> approaches, but can’t seem to get a clue as to why this is happening. 
>>>> Since it generally doesn’t seem to happen for other configured components, 
>>>> I am assuming that it is not a Configurator problem.
>>>>  
>>>> From the rest of your email it is clear that the configured component 
>>>> provides a service. This will make it lazy by default. You have also 
>>>> stated that you’re using has a required configuration policy. Therefore it 
>>>> will only be activated when:
>>>>  
>>>> There is a configuration present
>>>> All of the mandatory services are satisfied
>>>> Someone is actually using the service
>>>>  
>>>> The component will then be deactivated when any of these things are 
>>>> no-longer true, and so this means that your component may be being bounced 
>>>> for several reasons, only one of which is to do with configuration. 
>>>>  
>>>> While it seems unlikely to me, you seem to be fairly convinced that the 
>>>> configuration is at fault so lets start there.
>>>>  
>>>> Does the generated XML file in your bundle actually say that the 
>>>> configuration is required. This is the configuration used by DS (not the 
>>>> annotations) at runtime. It’s unlikely that this has gone wrong, but it’s 
>>>> an easy check
>>>> Do you have more than one way of putting configuration into the runtime? 
>>>> If you are also using File Install, or some other configuration management 
>>>> agent, then it’s entirely possible that you’re seeing a configuration 
>>>> update occurring.
>>>> Do you have multiple configuration bundles which both contain a 
>>>> configuration for this component? The configurator will process these one 
>>>> at a time, and it will result in configuration bouncing
>>>> Is it possible that something is forcing a re-resolve of your 
>>>> configuration bundle or the configurator? This could easily trigger the 
>>>> configurations to be reprocessed.
>>>>  
>>>> Now in my view the most likely reason for this behaviour is that the 
>>>> configured component isnot being bounced due to a configuration change. 
>>>> The most likely suspect is that the component is simply not being used at 
>>>> that time, and so it is being disposed (DS lazy behaviour). This could 
&

Re: [osgi-dev] Double config

2018-07-12 Thread David Leangen via osgi-dev

On the topic of bouncing…

What is a good way to test for robustness against this phenomenon? Is there 
some kind of Monkey Testing [1] framework to help with this?


[1] https://en.wikipedia.org/wiki/Monkey_testing 
<https://en.wikipedia.org/wiki/Monkey_testing>




> On Jul 12, 2018, at 23:11, Peter Kriens  wrote:
> 
> Well, OSGi has a tendency to punish short cuts. In the long run this is very 
> valuable, I’ve seen too many software disasters to sleep well at night. 
> However, for a developer behind his screen and a looming deadline the long 
> term goals are unfortunately not so urgent :-(
> 
> Kind regards,
> 
>   Peter Kriens
> 
> 
> 
>> On 12 Jul 2018, at 13:38, David Leangen via osgi-dev > <mailto:osgi-dev@mail.osgi.org>> wrote:
>> 
>> 
>> As always, thank you VERY much to all of you for your great suggestions. I 
>> will look into this from tomorrow. I didn’t know the concept of “bouncing”. 
>> That is interesting. Now that I am aware, I will give more thought to it.
>> 
>> One thing I have seen stated before on this list is that OSGi is indeed 
>> complex, but it is not accidental complexity. It is simply making complexity 
>> explicit, and providing the tools to work with complexity. I completely 
>> agree with this statement.
>> 
>> I am still learning all the time, and am always amazed that there is still 
>> so much deeper to go. Thanks for helping me along this journey.
>> 
>> 
>> Cheers,
>> =David
>> 
>> 
>> 
>>> On Jul 12, 2018, at 17:42, Fauth Dirk (AA-AS/EIS2-EU) via osgi-dev 
>>> mailto:osgi-dev@mail.osgi.org>> wrote:
>>> 
>>> If it is a bouncing problem, than you could also try to configure the 
>>> behavior using the property
>>>  
>>> ds.delayed.keepInstances=true
>>> 
>>> From the Apache Felix documentation:
>>>  
>>> Whether or not to keep instances of delayed components once they are not 
>>> referred to any more. The Declarative Services specifications suggests that 
>>> instances of delayed components are disposed off if there is not used any 
>>> longer. Setting this flag causes the components to not be disposed off and 
>>> thus prevent them from being constantly recreated if often used. Examples 
>>> of such components may be EventHandler services. The default is to dispose 
>>> off unused components. See FELIX-3039 
>>> <https://issues.apache.org/jira/browse/FELIX-3039> for details.
>>>  
>>> http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html
>>>  
>>> <http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html>
>>>  
>>> Equinox DS had this set by default, while the default in Felix SCR is 
>>> false. With the usage of Felix SCR in Eclipse we needed to explicitly set 
>>> this parameter to true so the previous behavior persists.
>>>  
>>> Mit freundlichen Grüßen / Best regards 
>>> 
>>> Dirk Fauth
>>> 
>>> Automotive Service Solutions, ESI application (AA-AS/EIS2-EU) 
>>> Robert Bosch GmbH | Postfach 11 29 | 73201 Plochingen | GERMANY | 
>>> www.bosch.com <http://www.bosch.com/> 
>>> Tel. +49 7153 666-1155 | dirk.fa...@de.bosch.com 
>>> <mailto:dirk.fa...@de.bosch.com> 
>>> 
>>> Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
>>> Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar 
>>> Denner,
>>> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Rolf Bulander, 
>>> Dr. Stefan Hartung, Dr. Markus Heyn,
>>> Dr. Dirk Hoheisel, Christoph Kübel, Uwe Raschke, Peter Tyroller 
>>> 
>>> 
>>> Von: osgi-dev-boun...@mail.osgi.org <mailto:osgi-dev-boun...@mail.osgi.org> 
>>> [mailto:osgi-dev-boun...@mail.osgi.org 
>>> <mailto:osgi-dev-boun...@mail.osgi.org>] Im Auftrag von Tim Ward via 
>>> osgi-dev
>>> Gesendet: Donnerstag, 12. Juli 2018 10:22
>>> An: Peter Kriens mailto:peter.kri...@aqute.biz>>; 
>>> OSGi Developer Mail List >> <mailto:osgi-dev@mail.osgi.org>>
>>> Betreff: Re: [osgi-dev] Double config
>>>  
>>> Hi David,
>>> 
>>> 
>>> This is a gift! :-) It means your code is not handling the dynamics 
>>> correctly and now you know it! 
>>>  
>>> I’m not sure that there’s quite enough evidence to come to this conclusion. 
>>> It could simply be that everythin

Re: [osgi-dev] Double config

2018-07-12 Thread Peter Kriens via osgi-dev
Well, OSGi has a tendency to punish short cuts. In the long run this is very 
valuable, I’ve seen too many software disasters to sleep well at night. 
However, for a developer behind his screen and a looming deadline the long term 
goals are unfortunately not so urgent :-(

Kind regards,

Peter Kriens



> On 12 Jul 2018, at 13:38, David Leangen via osgi-dev  
> wrote:
> 
> 
> As always, thank you VERY much to all of you for your great suggestions. I 
> will look into this from tomorrow. I didn’t know the concept of “bouncing”. 
> That is interesting. Now that I am aware, I will give more thought to it.
> 
> One thing I have seen stated before on this list is that OSGi is indeed 
> complex, but it is not accidental complexity. It is simply making complexity 
> explicit, and providing the tools to work with complexity. I completely agree 
> with this statement.
> 
> I am still learning all the time, and am always amazed that there is still so 
> much deeper to go. Thanks for helping me along this journey.
> 
> 
> Cheers,
> =David
> 
> 
> 
>> On Jul 12, 2018, at 17:42, Fauth Dirk (AA-AS/EIS2-EU) via osgi-dev 
>> mailto:osgi-dev@mail.osgi.org>> wrote:
>> 
>> If it is a bouncing problem, than you could also try to configure the 
>> behavior using the property
>>  
>> ds.delayed.keepInstances=true
>> 
>> From the Apache Felix documentation:
>>  
>> Whether or not to keep instances of delayed components once they are not 
>> referred to any more. The Declarative Services specifications suggests that 
>> instances of delayed components are disposed off if there is not used any 
>> longer. Setting this flag causes the components to not be disposed off and 
>> thus prevent them from being constantly recreated if often used. Examples of 
>> such components may be EventHandler services. The default is to dispose off 
>> unused components. See FELIX-3039 
>> <https://issues.apache.org/jira/browse/FELIX-3039> for details.
>>  
>> http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html
>>  
>> <http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html>
>>  
>> Equinox DS had this set by default, while the default in Felix SCR is false. 
>> With the usage of Felix SCR in Eclipse we needed to explicitly set this 
>> parameter to true so the previous behavior persists.
>>  
>> Mit freundlichen Grüßen / Best regards 
>> 
>> Dirk Fauth
>> 
>> Automotive Service Solutions, ESI application (AA-AS/EIS2-EU) 
>> Robert Bosch GmbH | Postfach 11 29 | 73201 Plochingen | GERMANY | 
>> www.bosch.com <http://www.bosch.com/> 
>> Tel. +49 7153 666-1155 | dirk.fa...@de.bosch.com 
>> <mailto:dirk.fa...@de.bosch.com> 
>> 
>> Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
>> Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar 
>> Denner,
>> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Rolf Bulander, Dr. 
>> Stefan Hartung, Dr. Markus Heyn,
>> Dr. Dirk Hoheisel, Christoph Kübel, Uwe Raschke, Peter Tyroller 
>> 
>> 
>> Von: osgi-dev-boun...@mail.osgi.org <mailto:osgi-dev-boun...@mail.osgi.org> 
>> [mailto:osgi-dev-boun...@mail.osgi.org 
>> <mailto:osgi-dev-boun...@mail.osgi.org>] Im Auftrag von Tim Ward via osgi-dev
>> Gesendet: Donnerstag, 12. Juli 2018 10:22
>> An: Peter Kriens mailto:peter.kri...@aqute.biz>>; 
>> OSGi Developer Mail List > <mailto:osgi-dev@mail.osgi.org>>
>> Betreff: Re: [osgi-dev] Double config
>>  
>> Hi David,
>> 
>> 
>> This is a gift! :-) It means your code is not handling the dynamics 
>> correctly and now you know it! 
>>  
>> I’m not sure that there’s quite enough evidence to come to this conclusion. 
>> It could simply be that everything is working fine with a static policy, and 
>> working the way that DS is supposed to. In general a DS component being 
>> bounced like this is nothing to worry about, it’s just a change rippling 
>> through the system, and it’s typically resolved in very short order. If you 
>> do want to reason about why, however, then there are several places to look.
>>  
>> For some reason, the Component gets activated, deactivated, then activated 
>> again, which is not desirable.
>> ...
>> 1. How can I figure out why this is happening. I have tried many approaches, 
>> but can’t seem to get a clue as to why this is happening. Since it generally 
>> doesn’t seem to happen for other configured components, I am

Re: [osgi-dev] Double config

2018-07-12 Thread David Leangen via osgi-dev

As always, thank you VERY much to all of you for your great suggestions. I will 
look into this from tomorrow. I didn’t know the concept of “bouncing”. That is 
interesting. Now that I am aware, I will give more thought to it.

One thing I have seen stated before on this list is that OSGi is indeed 
complex, but it is not accidental complexity. It is simply making complexity 
explicit, and providing the tools to work with complexity. I completely agree 
with this statement.

I am still learning all the time, and am always amazed that there is still so 
much deeper to go. Thanks for helping me along this journey.


Cheers,
=David



> On Jul 12, 2018, at 17:42, Fauth Dirk (AA-AS/EIS2-EU) via osgi-dev 
>  wrote:
> 
> If it is a bouncing problem, than you could also try to configure the 
> behavior using the property
>  
> ds.delayed.keepInstances=true
> 
> From the Apache Felix documentation:
>  
> Whether or not to keep instances of delayed components once they are not 
> referred to any more. The Declarative Services specifications suggests that 
> instances of delayed components are disposed off if there is not used any 
> longer. Setting this flag causes the components to not be disposed off and 
> thus prevent them from being constantly recreated if often used. Examples of 
> such components may be EventHandler services. The default is to dispose off 
> unused components. See FELIX-3039 
> <https://issues.apache.org/jira/browse/FELIX-3039> for details.
>  
> http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html
>  
> <http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html>
>  
> Equinox DS had this set by default, while the default in Felix SCR is false. 
> With the usage of Felix SCR in Eclipse we needed to explicitly set this 
> parameter to true so the previous behavior persists.
>  
> Mit freundlichen Grüßen / Best regards 
> 
> Dirk Fauth
> 
> Automotive Service Solutions, ESI application (AA-AS/EIS2-EU) 
> Robert Bosch GmbH | Postfach 11 29 | 73201 Plochingen | GERMANY | 
> www.bosch.com <http://www.bosch.com/> 
> Tel. +49 7153 666-1155 | dirk.fa...@de.bosch.com 
> <mailto:dirk.fa...@de.bosch.com> 
> 
> Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
> Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar 
> Denner,
> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Rolf Bulander, Dr. 
> Stefan Hartung, Dr. Markus Heyn,
> Dr. Dirk Hoheisel, Christoph Kübel, Uwe Raschke, Peter Tyroller 
> 
> 
> Von: osgi-dev-boun...@mail.osgi.org [mailto:osgi-dev-boun...@mail.osgi.org] 
> Im Auftrag von Tim Ward via osgi-dev
> Gesendet: Donnerstag, 12. Juli 2018 10:22
> An: Peter Kriens ; OSGi Developer Mail List 
> 
> Betreff: Re: [osgi-dev] Double config
>  
> Hi David,
> 
> 
> This is a gift! :-) It means your code is not handling the dynamics correctly 
> and now you know it! 
>  
> I’m not sure that there’s quite enough evidence to come to this conclusion. 
> It could simply be that everything is working fine with a static policy, and 
> working the way that DS is supposed to. In general a DS component being 
> bounced like this is nothing to worry about, it’s just a change rippling 
> through the system, and it’s typically resolved in very short order. If you 
> do want to reason about why, however, then there are several places to look.
>  
> For some reason, the Component gets activated, deactivated, then activated 
> again, which is not desirable.
> ...
> 1. How can I figure out why this is happening. I have tried many approaches, 
> but can’t seem to get a clue as to why this is happening. Since it generally 
> doesn’t seem to happen for other configured components, I am assuming that it 
> is not a Configurator problem.
>  
> From the rest of your email it is clear that the configured component 
> provides a service. This will make it lazy by default. You have also stated 
> that you’re using has a required configuration policy. Therefore it will only 
> be activated when:
>  
> There is a configuration present
> All of the mandatory services are satisfied
> Someone is actually using the service
>  
> The component will then be deactivated when any of these things are no-longer 
> true, and so this means that your component may be being bounced for several 
> reasons, only one of which is to do with configuration. 
>  
> While it seems unlikely to me, you seem to be fairly convinced that the 
> configuration is at fault so lets start there.
>  
> Does the generated XML file in your bundle actually say that the 
> configuration is required. This is the configuration used by DS (not the

Re: [osgi-dev] Double config

2018-07-12 Thread Fauth Dirk (AA-AS/EIS2-EU) via osgi-dev
If it is a bouncing problem, than you could also try to configure the behavior 
using the property

ds.delayed.keepInstances=true
From the Apache Felix documentation:

Whether or not to keep instances of delayed components once they are not 
referred to any more. The Declarative Services specifications suggests that 
instances of delayed components are disposed off if there is not used any 
longer. Setting this flag causes the components to not be disposed off and thus 
prevent them from being constantly recreated if often used. Examples of such 
components may be EventHandler services. The default is to dispose off unused 
components. See FELIX-3039<https://issues.apache.org/jira/browse/FELIX-3039> 
for details.

http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html

Equinox DS had this set by default, while the default in Felix SCR is false. 
With the usage of Felix SCR in Eclipse we needed to explicitly set this 
parameter to true so the previous behavior persists.

Mit freundlichen Grüßen / Best regards

Dirk Fauth

Automotive Service Solutions, ESI application (AA-AS/EIS2-EU)
Robert Bosch GmbH | Postfach 11 29 | 73201 Plochingen | GERMANY | 
www.bosch.com<http://www.bosch.com>
Tel. +49 7153 666-1155 | dirk.fa...@de.bosch.com<mailto:dirk.fa...@de.bosch.com>

Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar 
Denner,
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Rolf Bulander, Dr. 
Stefan Hartung, Dr. Markus Heyn,
Dr. Dirk Hoheisel, Christoph Kübel, Uwe Raschke, Peter Tyroller


Von: osgi-dev-boun...@mail.osgi.org [mailto:osgi-dev-boun...@mail.osgi.org] Im 
Auftrag von Tim Ward via osgi-dev
Gesendet: Donnerstag, 12. Juli 2018 10:22
An: Peter Kriens ; OSGi Developer Mail List 

Betreff: Re: [osgi-dev] Double config

Hi David,


This is a gift! :-) It means your code is not handling the dynamics correctly 
and now you know it!

I’m not sure that there’s quite enough evidence to come to this conclusion. It 
could simply be that everything is working fine with a static policy, and 
working the way that DS is supposed to. In general a DS component being bounced 
like this is nothing to worry about, it’s just a change rippling through the 
system, and it’s typically resolved in very short order. If you do want to 
reason about why, however, then there are several places to look.

For some reason, the Component gets activated, deactivated, then activated 
again, which is not desirable.
...
1. How can I figure out why this is happening. I have tried many approaches, 
but can’t seem to get a clue as to why this is happening. Since it generally 
doesn’t seem to happen for other configured components, I am assuming that it 
is not a Configurator problem.

From the rest of your email it is clear that the configured component provides 
a service. This will make it lazy by default. You have also stated that you’re 
using has a required configuration policy. Therefore it will only be activated 
when:


  *   There is a configuration present
  *   All of the mandatory services are satisfied
  *   Someone is actually using the service

The component will then be deactivated when any of these things are no-longer 
true, and so this means that your component may be being bounced for several 
reasons, only one of which is to do with configuration.

While it seems unlikely to me, you seem to be fairly convinced that the 
configuration is at fault so lets start there.


  1.  Does the generated XML file in your bundle actually say that the 
configuration is required. This is the configuration used by DS (not the 
annotations) at runtime. It’s unlikely that this has gone wrong, but it’s an 
easy check
  2.  Do you have more than one way of putting configuration into the runtime? 
If you are also using File Install, or some other configuration management 
agent, then it’s entirely possible that you’re seeing a configuration update 
occurring.
  3.  Do you have multiple configuration bundles which both contain a 
configuration for this component? The configurator will process these one at a 
time, and it will result in configuration bouncing
  4.  Is it possible that something is forcing a re-resolve of your 
configuration bundle or the configurator? This could easily trigger the 
configurations to be reprocessed.

Now in my view the most likely reason for this behaviour is that the configured 
component is not being bounced due to a configuration change. The most likely 
suspect is that the component is simply not being used at that time, and so it 
is being disposed (DS lazy behaviour). This could easily happen if one of the 
dependent services that you mention starts using your component, and then is 
bounced (by a configuration update or whatever) which causes your component to 
be released. If nobody else is using your component at the time then it wi

Re: [osgi-dev] Double config

2018-07-12 Thread Tim Ward via osgi-dev
Hi David,

> This is a gift! :-) It means your code is not handling the dynamics correctly 
> and now you know it! 

I’m not sure that there’s quite enough evidence to come to this conclusion. It 
could simply be that everything is working fine with a static policy, and 
working the way that DS is supposed to. In general a DS component being bounced 
like this is nothing to worry about, it’s just a change rippling through the 
system, and it’s typically resolved in very short order. If you do want to 
reason about why, however, then there are several places to look.

>> For some reason, the Component gets activated, deactivated, then activated 
>> again, which is not desirable.
>> ...

>> 1. How can I figure out why this is happening. I have tried many approaches, 
>> but can’t seem to get a clue as to why this is happening. Since it generally 
>> doesn’t seem to happen for other configured components, I am assuming that 
>> it is not a Configurator problem.


From the rest of your email it is clear that the configured component provides 
a service. This will make it lazy by default. You have also stated that you’re 
using has a required configuration policy. Therefore it will only be activated 
when:

There is a configuration present
All of the mandatory services are satisfied
Someone is actually using the service

The component will then be deactivated when any of these things are no-longer 
true, and so this means that your component may be being bounced for several 
reasons, only one of which is to do with configuration. 

While it seems unlikely to me, you seem to be fairly convinced that the 
configuration is at fault so lets start there.

Does the generated XML file in your bundle actually say that the configuration 
is required. This is the configuration used by DS (not the annotations) at 
runtime. It’s unlikely that this has gone wrong, but it’s an easy check
Do you have more than one way of putting configuration into the runtime? If you 
are also using File Install, or some other configuration management agent, then 
it’s entirely possible that you’re seeing a configuration update occurring.
Do you have multiple configuration bundles which both contain a configuration 
for this component? The configurator will process these one at a time, and it 
will result in configuration bouncing
Is it possible that something is forcing a re-resolve of your configuration 
bundle or the configurator? This could easily trigger the configurations to be 
reprocessed.

Now in my view the most likely reason for this behaviour is that the configured 
component is not being bounced due to a configuration change. The most likely 
suspect is that the component is simply not being used at that time, and so it 
is being disposed (DS lazy behaviour). This could easily happen if one of the 
dependent services that you mention starts using your component, and then is 
bounced (by a configuration update or whatever) which causes your component to 
be released. If nobody else is using your component at the time then it will be 
deactivated and released. The easiest way to verify this is to make your 
component immediate. This will remove the laziness, and you will get a good 
idea as to whether the bounce is caused by things that you depend on, or by 
things that depend on you. If making your component immediate removes the 
“problem” then it proves that this isn’t a problem at all (and you can then 
remove the immediate behaviour again).

If making your component immediate doesn’t stop the bouncing then the third set 
of things to check is the list of services that your component depends on. Is 
it possible that one of them is being bounced due to a configuration update, or 
perhaps one of their service dependencies being unregistered/re-registered? As 
I mentioned before, bouncing of DS components is simply the way that updates 
propagate through the system when services use a static policy. It isn’t 
inherently a bad thing, but if you want to avoid it you have to be dynamic all 
the way down the dependency graph. Usually this is a lot more effort than it’s 
worth!

I hope this helps,

Tim


> On 12 Jul 2018, at 08:39, Peter Kriens via osgi-dev  
> wrote:
> 
> This is a gift! :-) It means your code is not handling the dynamics correctly 
> and now you know it! 
> 
> The cause is that that the DS starts the components before the Configurator 
> has done its work. The easiest solution seems to be to use start levels. If 
> your code CAN handle the dynamics, then this is one of the few legitimate 
> places where startlevels are useful. I usually oppose it because people do 
> not handle the dynamics correctly and want a short cut. This is fine until it 
> is not. And the ‘not’ happens guaranteed one day. So first fix the dynamics, 
> and then think of solutions that improve the experience.
> 
> For this purpose, enRoute Classic had a 
> ‘osgi.enroute.configurer.api.ConfigurationDone’ service. If you made an 
> @Reference to 

Re: [osgi-dev] Double config

2018-07-12 Thread Peter Kriens via osgi-dev
This is a gift! :-) It means your code is not handling the dynamics correctly 
and now you know it! 

The cause is that that the DS starts the components before the Configurator has 
done its work. The easiest solution seems to be to use start levels. If your 
code CAN handle the dynamics, then this is one of the few legitimate places 
where startlevels are useful. I usually oppose it because people do not handle 
the dynamics correctly and want a short cut. This is fine until it is not. And 
the ‘not’ happens guaranteed one day. So first fix the dynamics, and then think 
of solutions that improve the experience.

For this purpose, enRoute Classic had a 
‘osgi.enroute.configurer.api.ConfigurationDone’ service. If you made an 
@Reference to ConfigurationDone then you were guaranteed to not start before 
the Configurer had done its magic. Since you did not want to depend on such a 
specific service for reasons of cohesion, I developed AggregateState. One of 
the aggregated states was then the presence of the ConfigurationDone service. 
Although this is also not perfectly cohesive it at least aggregates all the 
uncohesive things in one place and it is configurable.

Although this works the customer still is not completely happy since also the 
Aggregate State feels uncohesive. So we’ve been discussing a new angle. I want 
to try to make the Configuration Records _transient_. In Felix Config Admin you 
can provide a persistence service (and it was recently made useful). I was 
thinking of setting a special property in the configuration (something like 
‘:persistence:transient’). The persistence layer would then _not_ persist it. 
I.e. after a restart there would be no configuration until the Configurer sets 
it. This will (I expect) seriously diminish the bouncing caused for these kind 
of components.

And if you’re asking why I am still on the enRoute classic Configurer. Well, it 
has ‘precious’ fields and they solved a nasty problem. We needed to use a well 
defined value but if the user set one of those values, we wanted to keep the 
user’s value. Quite a common scenario. With `precious` fields you rely on 
default values (so no value for a precious field in the configurer’s input) but 
copy the previous value to the newer configuration, if present. Works quite 
well.

I think `transient` and `precious` could be nice extensions to the new 
Configurator for R8.

Hope this helps. Kind regards,

Peter Kriens

> On 12 Jul 2018, at 00:48, David Leangen via osgi-dev  
> wrote:
> 
> 
> Hi!
> 
> A question about component configuration.
> 
> I have a component that has a required configuration policy. Using a (pre R7) 
> Configurator to configure the component. For some reason, the Component gets 
> activated, deactivated, then activated again, which is not desirable.
> 
> Questions:
> 
> 1. How can I figure out why this is happening. I have tried many approaches, 
> but can’t seem to get a clue as to why this is happening. Since it generally 
> doesn’t seem to happen for other configured components, I am assuming that it 
> is not a Configurator problem.
> 
> 2. Is there a way to prohibit this from happening?
> 
> 
> In the meantime, I will make the dependent services more dynamic so they are 
> not thrown off by this change, but their behavior is actually correct: the 
> expectation is that the configured service should only get instantiated once, 
> so a static @Reference is correct.
> 
> 
> Thanks!
> =David
> 
> 
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev