Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-19 Thread Pierre De Rop
Yes David, indeed our scenarios are different. In mine, I'm measuring the assembly of components using parallel dependency manager where components are concurrently activated, registered, and bound with each other, using a shared thread pool. cheers; /Pierre On Tue, May 19, 2015 at 3:40 PM, Da

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-19 Thread David Bosschaert
Hi Pierre, Good to hear that the problem is now gone. I guess the performance improvement measured hugely depends on what you are testing. My test focuses on multiple clients/threads/bundles accessing the same service (either singleton or PSF) in a very raw manner (via ctx.getServiceReference()).

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-19 Thread Pierre De Rop
Hi David, Excellent. I'm glad to confirm that the issue is resolved, and my DM loader is now running seamlessly. I'm observing an overall gain of 16% compared to the previous 5.0.0. (but this has to be taken with care,because I only made a quick test). I did not have time but I guess I could obs

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-19 Thread David Bosschaert
Thanks Pierre for submitting a unit test to FELIX-4866 that helped me enormously in identifying the issue. I have fixed the bug in my code (without degrading performance) and at least your concurrency test, my concurrency tests and all the framework unit tests now consistently pass. I would be ver

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
the threadump did not help. I will investigate (may be a bug somewhere in my part; if this is the case, I would be sorry to make all this noise). hope to let you know soon. by the way, do you know how to run the SCR integration tests with the framework from the trunk ? I know that there are some

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
ok, it's a bit late, I will continue tomorrow. What I just found is that when the test fails, we are in the following situation: A DM component C1 that is part of the test remains inactive because it is awaiting for a service dependency on C2. But C2 is actually registered in the OSGi service regi

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
the threadump did not help. I will investigate (may be a bug somewhere in my part; if this is the case, I would be sorry to make all this noise). hope to let you know soon. by the way, do you know how to run the SCR integration tests with the framework from the trunk ? I know that there are some

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread David Bosschaert
Hi Pierre, It would indeed be useful to find out more about why your test is hanging. Maybe analysing a threaddump might give some more information? Cheers, David On 14 May 2015 at 19:54, Pierre De Rop wrote: > Thanks David; I just gave a try, and indeed the parallel test passed. I > observed

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
oops, I just realise that it's not making sense to include the SCR test bundle in the benchmark tools, since it's not a concurrent test. So for now, I have clues about where the problem may come from. cheers; /Pierre On Thu, May 14, 2015 at 7:54 PM, Pierre De Rop wrote: > Thanks David; I just

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
Thanks David; I just gave a try, and indeed the parallel test passed. I observed a gain of around 7/10%. The tool is described in [1]. But I only have 4 cores on my laptop and I will make more tests in my lab at work (next week) where we have some servers having 32 or even 128 processors. This wil

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread David Bosschaert
I've fixed this now in svn.apache.org/viewvc?view=revision&revision=1679367 Pierre, your loadtest now runs to completion - thanks for reporting this issue! I can see that the results for the parallel tests are a little bit different than before, but I'm not sure how to read them so I'll leave the

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread David Bosschaert
I think I know what this is. I had some additional changes exactly in this area that I simply forgot to apply this morning. I should have it fixed sometime today. Cheers, David On 14 May 2015 at 14:03, David Bosschaert wrote: > Hi Pierre, > > I'll take a look today. > > Cheers, > > David > > On

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread David Bosschaert
Hi Pierre, I'll take a look today. Cheers, David On 14 May 2015 at 14:00, Pierre De Rop wrote: > I just committed the benchmark tool in > http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/, if you can > take a look. > > To run the scenario: > > - install jdk8: > > [nxuser@nx0012 pdero

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
I just committed the benchmark tool in http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/, if you can take a look. To run the scenario: - install jdk8: [nxuser@nx0012 pderop]$ java -version java version "1.8.0_40" Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) 64-

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread Pierre De Rop
Hi David, I don't know if it's me (a bug in my benchmark tool) or if if there is a regression somewhere in the framework, by my parallel test does not pass anymore. The test first starts with a single-threaded scenario, which passes OK (org.apache.felix.dependencymanager.benchmark.dependencymanag

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-14 Thread David Bosschaert
I've committed this now in http://svn.apache.org/viewvc?view=revision&revision=1679327 Curious to see what others are measuring. My tests were focused on multiple bundles/threads obtaining the same service, as that's were I saw a bit of contention. Cheers, David On 13 May 2015 at 15:10, Pierre

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-13 Thread Pierre De Rop
Hi David, I'm looking forward to test your improvements using the dependencymanager benchmark tool ([1]). [1] http://svn.apache.org/viewvc/felix/trunk/dependencymanager/org.apache.felix.dependencymanager.benchmark/ /Pierre On Wed, May 13, 2015 at 3:02 PM, David Bosschaert < david.bosscha...@gm

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-05-13 Thread David Bosschaert
I have implemented the performance improvements that I was thinking of using Java 5 concurrency tools, they can be viewed at [1]. I wrote a little performance test suite [2] that tests multithreaded service registry performance (10 threads) from single / multiple bundles with either singleton serv

Re: Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-03-23 Thread Richard S. Hall
On 3/23/15 10:17 , David Bosschaert wrote: On 23 March 2015 at 13:39, Richard S. Hall wrote: On 3/23/15 03:55 , Guillaume Nodet wrote: There's a call to interrupt() in Felix#acquireBundleLock(), not sure if it can be the culprit though. Interrupts could also be caused by a bundle being shutdow

Service Registry refactor using Java-5 concurrency libraries (Was Re: [Framework] ServiceRegistry.getService() endless loop with lock?)

2015-03-23 Thread David Bosschaert
On 23 March 2015 at 13:39, Richard S. Hall wrote: > On 3/23/15 03:55 , Guillaume Nodet wrote: >> >> There's a call to interrupt() in Felix#acquireBundleLock(), not sure if it >> can be the culprit though. >> Interrupts could also be caused by a bundle being shutdown while one of >> its >> thread i

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-23 Thread Richard S. Hall
On 3/23/15 03:55 , Guillaume Nodet wrote: There's a call to interrupt() in Felix#acquireBundleLock(), not sure if it can be the culprit though. Interrupts could also be caused by a bundle being shutdown while one of its thread is waiting for a service, which should is a valid use case imho. Anywa

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-23 Thread Robert Munteanu
On Fri, Mar 20, 2015 at 7:41 PM, David Bosschaert wrote: > Some more thoughts about this... > > The wait() call in the getService() method is as follows: > > synchronized (this) > { > // First make sure that no existing operation is currently > // being performed by

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-23 Thread Guillaume Nodet
There's a call to interrupt() in Felix#acquireBundleLock(), not sure if it can be the culprit though. Interrupts could also be caused by a bundle being shutdown while one of its thread is waiting for a service, which should is a valid use case imho. Anyway, I think sanely reacting to a thread being

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-23 Thread Carsten Ziegeler
Am 23.03.15 um 01:25 schrieb Richard S. Hall: > On 3/21/15 05:52 , Carsten Ziegeler wrote: >> >> Question remains, why the thread got interrupted in the first place. > > It was something that you did as part of FELIX-4806: > Yes, I noticed this as well - and I have no idea why I did it. I know th

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-22 Thread Richard S. Hall
On 3/21/15 05:52 , Carsten Ziegeler wrote: Am 20.03.15 um 18:41 schrieb David Bosschaert: Some more thoughts about this... The wait() call in the getService() method is as follows: synchronized (this) { // First make sure that no existing operation is currently

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-22 Thread David Bosschaert
Thanks Carsten! I removed that interrupt() call on trunk. On why it got interrupted? I'm not sure. I guess anyone could call Thread.interrupt()... Cheers, David On 21 March 2015 at 09:52, Carsten Ziegeler wrote: > Good question - The call to interrupt() is wrong. If the thread gets > interrup

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-21 Thread Carsten Ziegeler
Am 20.03.15 um 18:41 schrieb David Bosschaert: > Some more thoughts about this... > > The wait() call in the getService() method is as follows: > > synchronized (this) > { > // First make sure that no existing operation is currently > // being performed by another

Re: [Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-20 Thread David Bosschaert
Some more thoughts about this... The wait() call in the getService() method is as follows: synchronized (this) { // First make sure that no existing operation is currently // being performed by another thread on the service registration. for (Object o =

[Framework] ServiceRegistry.getService() endless loop with lock?

2015-03-20 Thread David Bosschaert
Hi all, I'm looking at an issue that I'm experiencing (with Felix 4.6.1/Java 7) where the ServiceRegsitry.getService() [1] method seems to be in an endless loop. It doesn't happen very often, but when it does happen the thread executing getService() seems to never exit that method apparently switc