yes, concretely (to adapt to your case) you will need to play with: 1. timeout (not something a small slow down can hit but not something too big, ~1mn is not bad for slow providers) 2. caching (jcache in local mode is great) 3. memory
Allocating blindly threads will lead to out of memory (try to have core=15000 tasks in your test should be close to make it happening) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://blog-rmannibucau.rhcloud.com> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory <https://javaeefactory-rmannibucau.rhcloud.com> 2017-01-29 15:51 GMT+01:00 cocorossello <[email protected]>: > Sorry, I hit enter too early. > > > > > I see the problem. > > > I didn't have that problem with @Asynchronous, but it is probably because I > didn't properly test it and we haven't hit max. > > Our application searches for several type of products (flights, trains, > hotels, tickets, transfers, cars) in several different providers for each > type of product. (that would be about 20-25 threads opened per search). > > -For each type of product we open a thread (product thread) > -For each provider of that specific product we open a thread to search > the product (search thread) > -We mix all results from each provider in the product thread > > > > So I think that the best solution for us would be to use different pools > for each task and always use timeouts (we have proper timeouts in all web > service searchs, I didn't think about this case). > > > > On Sun, Jan 29, 2017 at 3:43 PM, Vicente Rossello <[email protected]> > wrote: > > > Hi, > > > > I see the problem. > > > > I don't really understand why max is 100 > > > > On Sun, Jan 29, 2017 at 3:27 PM, Romain Manni-Bucau [via TomEE & OpenEJB] > > <[email protected]> wrote: > > > >> I think you just ensured to lock yourself - but agree it is well hidden > >> ;). > >> > >> You have one layer waiting for N sub async tasks to be done but next > >> layer > >> submits a lot of task and first layer keeps continuing so you just block > >> yourself since at some point second layer is blocked by first layer > which > >> filled the pool. Not it is not a deadlock by itself but just a code > logic > >> lock if that phrasing means anything. > >> > >> Note that configuring a queue that big will always queue instead of > >> increasing the pool size which will always be 100 and never 700 > >> (reference > >> to your config). Not an issue but means Max=100 concretely. It means > that > >> if com.tr2.test.MyStateless#sayHello submits 99 tasks instead of 200 > the > >> blocking will likely not happen that easily if you followed me. > >> > >> Said otherwise: it is not linked to tomee but the code design (nesting > >> thread pool tasks between them for a *same* executor is very dangerous. > >> > >> Here a simpler example: you have an executor of core=1, queue=1, first > >> task > >> submits 1 other tasks. You submitted therefore 2 tasks which passes > cause > >> it fills our core and queue but doesn't overpass it. However the second > >> task submitted by the first one will never be executed and therefore the > >> first task will keep waiting for it except if you have a timeout. In > such > >> a > >> case the "facade"/first task will fail and let the child one be executed > >> and release slowly the pool. > >> > >> Timeouts are key with thread pools. > >> > >> Side note, this works in the interceptor: > >> > >> @Resource(name = "TravelcAsynchronousPool") > >> private ManagedExecutorService executor; > >> > >> > >> > >> > >> Romain Manni-Bucau > >> @rmannibucau <https://twitter.com/rmannibucau> | Blog > >> <https://blog-rmannibucau.rhcloud.com> | Old Blog > >> <http://rmannibucau.wordpress.com> | Github < > >> https://github.com/rmannibucau> | > >> LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory > >> <https://javaeefactory-rmannibucau.rhcloud.com> > >> > >> 2017-01-29 13:48 GMT+01:00 cocorossello <[hidden email] > >> <http:///user/SendEmail.jtp?type=node&node=4680972&i=0>>: > >> > >> > Hi, > >> > > >> > I'm running into problems with this, I think it might be another > >> openejb > >> > bug. > >> > > >> > If I have two levels of EJB @async threads get stuck at: > >> > > >> > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > >> > at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429) > >> > at java.util.concurrent.FutureTask.get(FutureTask.java:191) > >> > at org.apache.openejb.threads.future.CUFuture.get(CUFuture. > java:55) > >> > >> > at > >> > com.tr2.util.interceptor.async.FutureDelegator.get( > >> > FutureDelegator.java:19) > >> > > >> > The setup is very simple > >> > someStateless -> AsyncTask1 -> asyncTask2 > >> > > >> > I made an ApplicationComposer test (AsyncInterceptorIT) in that github > >> > project (with latest SNAPSHOT) > >> > > >> > https://github.com/cocorossello/tomee-example > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > -- > >> > View this message in context: http://tomee-openejb.979440. > >> > n4.nabble.com/MDC-and-Asynchronous-tp4680927p4680971.html > >> > Sent from the TomEE Users mailing list archive at Nabble.com. > >> > > >> > >> > >> ------------------------------ > >> If you reply to this email, your message will be added to the discussion > >> below: > >> http://tomee-openejb.979440.n4.nabble.com/MDC-and-Asynchrono > >> us-tp4680927p4680972.html > >> To unsubscribe from MDC and @Asynchronous, click here > >> <http://tomee-openejb.979440.n4.nabble.com/template/ > NamlServlet.jtp?macro=unsubscribe_by_code&node=4680927&code= > Y29jb3Jvc3NlbGxvQGdtYWlsLmNvbXw0NjgwOTI3fC05Mzc2MzQ4MzY=> > >> . > >> NAML > >> <http://tomee-openejb.979440.n4.nabble.com/template/ > NamlServlet.jtp?macro=macro_viewer&id=instant_html% > 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > BasicNamespace-nabble.view.web.template.NabbleNamespace- > nabble.view.web.template.NodeNamespace&breadcrumbs= > notify_subscribers%21nabble%3Aemail.naml-instant_emails% > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > >> > > > > > > > > > -- > View this message in context: http://tomee-openejb.979440. > n4.nabble.com/MDC-and-Asynchronous-tp4680927p4680975.html > Sent from the TomEE Users mailing list archive at Nabble.com. >
