But doesn't that mean it's local to the thread, and any thread that accesses
it, will use the same one? I'm trying to send the exchange from the same
thread....
On Mon, Jun 23, 2008 at 3:07 AM, Guillaume Nodet <[EMAIL PROTECTED]> wrote:
> The currentRequest is a thread local variable, so if another exchange
> is handled it should not damage the processing of another thread.
>
> On Mon, Jun 23, 2008 at 8:45 AM, Ryan Moquin <[EMAIL PROTECTED]>
> wrote:
> > Could someone on the servicemix team verify this rationale for me? I
> think
> > I figured out my problem, which is a race condition in Servicemix (which
> of
> > course is exploited more on a faster machine). I'm not sure what the
> best
> > solution to work around this problem is, and I am on a short timeframe
> since
> > of course this problem really only plagues our production environment and
> > not our dev or test environment. Anyhow, I have a servicemix-bean
> component
> > that is taking an input, performing some analysis on it, and then sending
> it
> > back out to a JMS topic. In order to send it back out, I'm using the
> > DeliveryChannel that is injected into my bean. This sharing of the
> delivery
> > channel is what is causing the race condition, here is the reason. The
> > exceptions I've been mentioning are referring to the following line:
> >
> > requests.put(messageExchange.getExchangeId(), currentRequest.get());
> >
> > This line is dependant upon the request I'm building in order to send out
> to
> > the JMS queue (at least I believe so). The thing is that currentRequest
> is
> > global to the current thread, so when a message comes into my
> > servicemix-bean component, which I believe would be this method snippet:
> >
> > protected void onProviderExchange(MessageExchange exchange) throws
> Exception
> > {
> > ...
> > currentRequest.set(req);
> > synchronized (req) {
> > ...
> > }
> > checkEndOfRequest(req, corId);
> > currentRequest.set(null);
> >
> > Doesn't this mean that anytime a message is delivered (inside the
> > synchronized block), there is a good chance that when my bean is trying
> to
> > use the delivery channel to send a message, that the "currentRequest"
> member
> > variable is being clobbered? I get an NPE .. it seems like that's most
> > likely a sideeffect of the currentRequest.set(null). Such as my send
> inside
> > the bean, clobbers the currentRequest, and then in turn, the BeanEndpoint
> > clobbers my request and makes it null, which causes my send to blow up?
> >
> > It appears as though I maybe should be creating my own deliverychannel or
> > using another method to send out from my bean? Or is this something that
> > could be fixed, and if so, how soon? Or is it something I could do with
> > some guidance? Or is there something else I can do? I'm going to go
> look
> > through the docs for another option, but I really have to work around
> this,
> > since my app is pretty much dead right now, which is the worst time.
> >
> > Thanks for any help!!
> >
> > Ryan
> >
> > On Sun, Jun 22, 2008 at 7:17 PM, Ryan Moquin <[EMAIL PROTECTED]>
> wrote:
> >
> >> Just a few details on this.. so servicemix deadlocks every time I
> startup,
> >> where servicemix needs to redeploy all SEs and all SUs... here is the
> >> deadlock trace.. all my threads are sitting like this:
> >>
> >> "pool-flow.seda.servicemix-bean-thread-1" prio=6 tid=0x2879c400
> nid=0x1728
> >> waiting for monitor entry [0x3512f000..0x3512
> >> fb94]
> >> java.lang.Thread.State: BLOCKED (on object monitor)
> >> at
> >>
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.createQueue(SedaFlow.java:189)
> >> - waiting to lock <0x07db2050> (a
> >> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow)
> >> at
> >>
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:179)
> >> at
> >>
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:162)
> >> at
> >>
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
> >> at
> >>
> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
> >> at
> >>
> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88)
> >> at
> >>
> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:830)
> >> at
> >>
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:395)
> >> at
> >>
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:431)
> >> at
> >>
> org.apache.servicemix.common.EndpointDeliveryChannel.send(EndpointDeliveryChannel.java:79)
> >> at
> >>
> org.apache.servicemix.bean.BeanEndpoint$PojoChannel.send(BeanEndpoint.java:571)
> >>
> >>
> >> then if I force it closed, since it's hanging and unresponsive... I do
> get
> >> a ton of NPE's on startup and then it APPEARS to start behaving
> normally...
> >> so I'm not sure if this latest 13 build has solved the problem enough or
> if
> >> some code changes I made did, which again is here:
> >>
> >> Caused by: java.lang.NullPointerException
> >> at
> >> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
> >> at
> >>
> org.apache.servicemix.bean.BeanEndpoint$PojoChannel.send(BeanEndpoint.java:569)
> >> at
> >>
> >> and results in some illegal sendSync errors.... is it possible that my
> >> MessageExchangeFactory is giving back a null message or something like
> that
> >> which is causing the NPE?
> >>
> >>
> >> On Sun, Jun 22, 2008 at 5:38 PM, Ryan Moquin <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >>> Hey Bruce,
> >>>
> >>> Out of curiousity.. did you guys do anything with this issue in the
> last
> >>> day or two? I've been working on some stuff on my end that I was
> hoping may
> >>> have had some effect.. when I run it on the build tagged "12" in the
> nightly
> >>> builds, I have the same issue. When I try the one tagged "13" all my
> >>> outgoing JMS requests on the DeliveryChannel deadlock the first time I
> start
> >>> up Servicemix. The second time I start up it, I get a ton of errors
> about
> >>> illegally calling sendSync but my threads don't deadlock that second
> time.
> >>> The deadlock is way down in the bowels of servicemix... I don't see
> that NPE
> >>> anymore on the latest one, so I'm just curious so I know exactly what
> I'm
> >>> trying to solve at this point! :)
> >>>
> >>> Thanks!!
> >>>
> >>> Ryan
> >>>
> >>>
> >>> On Sat, Jun 21, 2008 at 12:48 AM, Ryan Moquin <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>>> I hate to say this Bruce, but unfortunately 3.2.2 is working pretty
> bad
> >>>> for me and has the same problem :( It only seems to affect our
> production
> >>>> server and in a few minutes after start up, I start getting the
> >>>> NullPointerException.. slowly all my services start doing it and they
> all
> >>>> stop working. If I stop and restart servicemix, then servicemix-jms
> >>>> components are no longer routable. I'm guessing this error causes
> >>>> Servicemix to shut them down and not deploy them anymore (Servicemix
> does
> >>>> the same thing if a service unit starts up, tries to make a Joram
> connection
> >>>> to a server that isn't up, servicemix will shutdown that SU and will
> always
> >>>> shut it down immediately after it starts on any subsequent run).
> >>>>
> >>>> It seems like this problem must be related to a race condition. When
> >>>> doing development testing, I never see this problem on my laptop even
> under
> >>>> high load. One our fast test server, I see this error popup once on
> startup
> >>>> and then it doesn't seem to happen again. On our even faster
> production
> >>>> server, the whole thing looses it's wheels and falls apart after a few
> >>>> minutes.
> >>>>
> >>>> I'm supposed to be deploying this system in a few days and of course
> >>>> that's the only spot where I can't temporarily limp by. Is it
> possible you
> >>>> could give me some hints on what the problem is and I'll debug it this
> >>>> weekend to see if I can fix it, or at least patch it temporarily? I
> really
> >>>> need to figure out way to get around this problem. Other than that,
> 3.2.2
> >>>> seems to work perfectly fine.
> >>>>
> >>>> Here is the caused by error again in case it's any bit different than
> the
> >>>> 3.2.1 one was:
> >>>>
> >>>> Caused by: java.lang.NullPointerException
> >>>> at
> >>>> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
> >>>> at
> >>>>
> org.apache.servicemix.bean.BeanEndpoint$PojoChannel.send(BeanEndpoint.java:569)
> >>>> at
> >>>>
> com.notification.impl.JbiNotificationHandlerImpl.sendNotification(JbiNotificat
> >>>> ionHandlerImpl.java:80)
> >>>>
> >>>> Also, is the delivery channel component threadsafe? I'm curious if
> >>>> multiple threads accessing it is a problem or if I should keep access
> to it
> >>>> synchronized? I'm currently synchronizing, but don't want to if I
> don't
> >>>> need to.
> >>>>
> >>>> Thanks!
> >>>> Ryan
> >>>>
> >>>>
> >>>> On Thu, Jun 19, 2008 at 2:19 AM, Bruce Snyder <[EMAIL PROTECTED]
> >
> >>>> wrote:
> >>>>
> >>>>> On Wed, Jun 18, 2008 at 7:56 PM, Ryan Moquin <[EMAIL PROTECTED]
> >
> >>>>> wrote:
> >>>>> > I'm using Servicemix 3.2.1, so I'll give 3.2.2 a try. I was kind
> of
> >>>>> waiting
> >>>>> > until it was released, but this problem is now cropping up on a
> >>>>> regular
> >>>>> > basis on a server so I'll definitely give it a shot. Hopefully
> this
> >>>>> will
> >>>>> > allow me to get this project done so I can then get that Joram
> write
> >>>>> up done
> >>>>> > since I should have ran across hopefully most of the gotchas for it
> at
> >>>>> that
> >>>>> > point.
> >>>>>
> >>>>> 3.2.2 will be released very soon and I know it's pretty stable. Maybe
> >>>>> we can release it this weekend.
> >>>>>
> >>>>> Bruce
> >>>>> --
> >>>>> perl -e 'print
> >>>>> unpack("u30","D0G)[EMAIL
> >>>>> PROTECTED]&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> >>>>> );'
> >>>>>
> >>>>> Apache ActiveMQ - http://activemq.org/
> >>>>> Apache Camel - http://activemq.org/camel/
> >>>>> Apache ServiceMix - http://servicemix.org/
> >>>>>
> >>>>> Blog: http://bruceblog.org/
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >
>
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
>
