Hi Erwin, ah sorry, right :) I was reading modulo 10 instead of equals 10...
Yes, you're absolutely right in this case, that should not happen. This looks like a problem in postEvent somewhere Carsten Erwin Hogeweg wrote > Hi Carsten, > > Sorry, I think didn’t make myself clear. > > I am only replacing ONE listener with a BlockingListener, and then I am > blocking ONE event delivery. So I do expect to loose ONE event, not 70k… > > Or am I missing something? > > Erwin > > > @Test > public void testEventing() throws Exception { > // this.addListener(PREFIX + "/0", null); > this.addBlockingListener(PREFIX + "/0", null); > this.addListener(PREFIX + "/1", null); > this.addListener(PREFIX + "/2", null); > ... > > > > On Nov 29, 2017, at 09:21, Carsten Ziegeler > <cziege...@apache.org<mailto:cziege...@apache.org>> wrote: > > Thanks > > I'm not sure if event admin could/should do anything in this case. Each > blocking listener takes away one thread from the thread pool. As soon as > you have as many blocking listeners as the pool has threads, event admin > will not deliver any event anymore > > Regards > > Carsten > > > Erwin Hogeweg wrote > Hi Carsten, > > After analyzing a bunch of log files we found some evidence that suggested > that an EventAdminThread might be blocked. > > I was able to duplicate it with a JUnit test. I created a BlockingListener > with extends Listener and override the handleEvent() method. As you can see > below we are loosing almost 70000 events because of this one block. > > I know, eventHandlers are not supposed to block, so we definitely have a > design issue here, but I figured I share the results with you because I am > not sure this is what’s expected. > > > Kind Regards, > > Erwin > > > @Override > public void handleEvent(final Event event) { > this.test.handleEvent(event, this.payload); > count++; > if (count == 10) { > logger.info<http://logger.info>("{} is blocking forever", > Thread.currentThread().getName()); > while (true) { > try { > Thread.sleep(60000); > } catch (InterruptedException e) { > // > } > } > } > > } > > > [org.apache.felix.eventadmin.ittests.StressTestIT] : Starting eventing test > StressTestIT > [org.apache.felix.eventadmin.ittests.StressTestIT] : Preparing test with 15 > threads and 10000 events per thread. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Expecting 1050000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started test with 15 > threads and 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : > [org.apache.felix.eventadmin.ittests.StressTestIT] : > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestITStarted thread > Started thread > ] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 12 events so > far. > [org.apache.felix.eventadmin.ittests.BlockingListener] : EventAdminThread #6 > is blocking forever > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events > so far. > … > > > > On Nov 15, 2017, at 21:04, Carsten Ziegeler > <cziege...@apache.org<mailto:cziege...@apache.org><mailto:cziege...@apache.org>> > wrote: > > Hi, > > we haven't experienced anything like this so far and I wouldn't know if > any better out of the box way to trouble shoot. I guess the best would > be to add additional logging to event admin, so you can follow what's > happening inside event admin. > > If it is somehow reproducible, then switching from postEvent to > sendEvent could also help in identifying whether there is a problem in > the postEvent handling (which is more complicated than sendEvent). > > Regards > > Carsten > > > Erwin Hogeweg wrote > Hi, > > I have a really bizarre problem. It looks like the eventAdmin is intermittent > but frequently loosing events. I find this very hard to believe but the > evidence seems to support my suspicion. > > I spit out a log msgs just before the event is posted and also when the event > is handled. There is only one eventHandler for this topic, and sure enough > every now and again I see and event be posted but it never comes out at the > other end. > > I have the event admin log level set to 4, but there aren’t any msgs to > indicate that something wrong. > > I am really at a loss here. Any suggestions for further trouble shooting > would be greatly appreciated. > > BTW… this seems to happen (and gradually increase) after the system has been > up for several weeks which points in the direction of a resource leak, but I > don’t see that either. > > > Regards, > > Erwin > > Java8 > EventAdmin-1.4.6 > Equinox: 3.10.2.v20150203-1939 > > Producer side: > ... > if (eventAdmin != null) { > if (LOG.isDebugEnabled()) { > LOG.debug("Posting: " + newEvent); > } > eventAdmin.postEvent(newEvent); > } else { > LOG.error("eventAdmin is null."); > } > > > Consumer side: > @Override > public void handleEvent( Event event ) { > LOG.info<http://LOG.info><http://LOG.info><http://LOG.info>("handleEvent: " > + event); > ... > > Erwin Hogeweg > CTO > 3690 Airport Road > Boca Raton, FL 33431 > P. +1 (954) 556-6565 > M. +1 (561) 306-7395 > F. +1 (561) 948-2730 > [Seecago]<http://www.seecago.com> > > -- > Carsten Ziegeler > Adobe Research Switzerland > cziege...@apache.org<mailto:cziege...@apache.org><mailto:cziege...@apache.org> > > Erwin Hogeweg > CTO > 3690 Airport Road > Boca Raton, FL 33431 > P. +1 (954) 556-6565 > M. +1 (561) 306-7395 > F. +1 (561) 948-2730 > [Seecago]<http://www.seecago.com> > > -- > Carsten Ziegeler > Adobe Research Switzerland > cziege...@apache.org<mailto:cziege...@apache.org> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@felix.apache.org > For additional commands, e-mail: users-h...@felix.apache.org > > > Erwin Hogeweg > CTO > 3690 Airport Road > Boca Raton, FL 33431 > P. +1 (954) 556-6565 > M. +1 (561) 306-7395 > F. +1 (561) 948-2730 > [Seecago]<http://www.seecago.com> > -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@felix.apache.org For additional commands, e-mail: users-h...@felix.apache.org