Hi Carsten, Sorry, I think didn’t make myself clear.
I am only replacing ONE listener with a BlockingListener, and then I am blocking ONE event delivery. So I do expect to loose ONE event, not 70k… Or am I missing something? Erwin @Test public void testEventing() throws Exception { // this.addListener(PREFIX + "/0", null); this.addBlockingListener(PREFIX + "/0", null); this.addListener(PREFIX + "/1", null); this.addListener(PREFIX + "/2", null); ... On Nov 29, 2017, at 09:21, Carsten Ziegeler <cziege...@apache.org<mailto:cziege...@apache.org>> wrote: Thanks I'm not sure if event admin could/should do anything in this case. Each blocking listener takes away one thread from the thread pool. As soon as you have as many blocking listeners as the pool has threads, event admin will not deliver any event anymore Regards Carsten Erwin Hogeweg wrote Hi Carsten, After analyzing a bunch of log files we found some evidence that suggested that an EventAdminThread might be blocked. I was able to duplicate it with a JUnit test. I created a BlockingListener with extends Listener and override the handleEvent() method. As you can see below we are loosing almost 70000 events because of this one block. I know, eventHandlers are not supposed to block, so we definitely have a design issue here, but I figured I share the results with you because I am not sure this is what’s expected. Kind Regards, Erwin @Override public void handleEvent(final Event event) { this.test.handleEvent(event, this.payload); count++; if (count == 10) { logger.info<http://logger.info>("{} is blocking forever", Thread.currentThread().getName()); while (true) { try { Thread.sleep(60000); } catch (InterruptedException e) { // } } } } [org.apache.felix.eventadmin.ittests.StressTestIT] : Starting eventing test StressTestIT [org.apache.felix.eventadmin.ittests.StressTestIT] : Preparing test with 15 threads and 10000 events per thread. [org.apache.felix.eventadmin.ittests.StressTestIT] : Expecting 1050000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Started test with 15 threads and 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : [org.apache.felix.eventadmin.ittests.StressTestIT] : [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestITStarted thread Started thread ] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 12 events so far. [org.apache.felix.eventadmin.ittests.BlockingListener] : EventAdminThread #6 is blocking forever [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far. … On Nov 15, 2017, at 21:04, Carsten Ziegeler <cziege...@apache.org<mailto:cziege...@apache.org><mailto:cziege...@apache.org>> wrote: Hi, we haven't experienced anything like this so far and I wouldn't know if any better out of the box way to trouble shoot. I guess the best would be to add additional logging to event admin, so you can follow what's happening inside event admin. If it is somehow reproducible, then switching from postEvent to sendEvent could also help in identifying whether there is a problem in the postEvent handling (which is more complicated than sendEvent). Regards Carsten Erwin Hogeweg wrote Hi, I have a really bizarre problem. It looks like the eventAdmin is intermittent but frequently loosing events. I find this very hard to believe but the evidence seems to support my suspicion. I spit out a log msgs just before the event is posted and also when the event is handled. There is only one eventHandler for this topic, and sure enough every now and again I see and event be posted but it never comes out at the other end. I have the event admin log level set to 4, but there aren’t any msgs to indicate that something wrong. I am really at a loss here. Any suggestions for further trouble shooting would be greatly appreciated. BTW… this seems to happen (and gradually increase) after the system has been up for several weeks which points in the direction of a resource leak, but I don’t see that either. Regards, Erwin Java8 EventAdmin-1.4.6 Equinox: 3.10.2.v20150203-1939 Producer side: ... if (eventAdmin != null) { if (LOG.isDebugEnabled()) { LOG.debug("Posting: " + newEvent); } eventAdmin.postEvent(newEvent); } else { LOG.error("eventAdmin is null."); } Consumer side: @Override public void handleEvent( Event event ) { LOG.info<http://LOG.info><http://LOG.info><http://LOG.info>("handleEvent: " + event); ... Erwin Hogeweg CTO 3690 Airport Road Boca Raton, FL 33431 P. +1 (954) 556-6565 M. +1 (561) 306-7395 F. +1 (561) 948-2730 [Seecago]<http://www.seecago.com> -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org<mailto:cziege...@apache.org><mailto:cziege...@apache.org> Erwin Hogeweg CTO 3690 Airport Road Boca Raton, FL 33431 P. +1 (954) 556-6565 M. +1 (561) 306-7395 F. +1 (561) 948-2730 [Seecago]<http://www.seecago.com> -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org<mailto:cziege...@apache.org> --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@felix.apache.org For additional commands, e-mail: users-h...@felix.apache.org Erwin Hogeweg CTO 3690 Airport Road Boca Raton, FL 33431 P. +1 (954) 556-6565 M. +1 (561) 306-7395 F. +1 (561) 948-2730 [Seecago]<http://www.seecago.com>