Re: [xmlblaster] Client connection not being restored properly?
Hi Marcel, In fact, that is what I have done for now (set session to unlimited). The only reason it has been an issue is because the default client session timeout is 24 hours and in some cases, I forgot to override the default in some of our low-traffic situations. Oops. If I were to file some bugs against this it would be: 1) What is the purpose of the session timing out on the server side when the client is still connected/alive (as far as a network connection is concerned, eg - ping requests are still being answered by the client)? If the client's alive, keep the session alive and just eliminate the server-side session timeout perhaps? 2) If the client reaches the 'DEAD' state, it shouldn't accept and queue more messages for publishing that it will never/can never publish. -- David Kerry On Thu, Apr 03, 2008 at 09:32:41AM +0200, Marcel Ruff wrote: > Hi David, > > I think your solution is to keep the server session alive by setting > the session to unlimited. > > Further Notes: > You could call xmlBlasterAccess.refreshSession() > to refresh the server side session (but this can fail if the network is > broken and the refresh is not reaching the server in time). > > If you don't want to use an unlimited server session you need to > destroy the XmlBlasterAccess instance (without deleting the client queue > entries) > when going to DEAD (xmlBlasterAccess.disconnect(disConnectQos)). > Use a disConnectQos.clearClientQueue(false) to keep the entries. > Re-establish the connection with a new XmlBlasterAccess instance > which then sends the existing queue entries. > However - this will, i think, not work if you use a RAM,1.0 queue (no DB). > > As said in the last mail, XmlBlasterAccess behavior is fishy after going > to DEAD, > which needs to be fixed by us. > We more clearly need to invalidate the XmlBlasterAccess instance in this > case. > > And finally: > If your scenario is a use case which we should support we > can think of changing the client library behavior so that a > server session timeout triggers a POLLING (and not a DEAD) > which would solve your case. Is there any logical reason > to keep the to-DEAD pattern (Michele?) > > regards > Marcel > > > David Kerry wrote: > >On Wed, Apr 02, 2008 at 07:42:54PM +0200, Marcel Ruff wrote: > > > >>Hi David, > >> > >>if the server side session times out the expected behavior is > >>that the client goes to dead (what you client seem to do). > >>That your client recovers from DEAD is not intended > >>and is considered a bug, see > >>http://www.xmlblaster.org/xmlBlaster/doc/requirements/client.failsafe.html#failsave > >>The transition (9) and (8) should not happen. > >> > >> > >>Thanks for reporting > >>best regards > >>Marcel > >> > > > >Hi Marcel, > > > >Whoops! > > > >Then I guess the original issue is still outstanding then. > > > >If my client goes from ALIVE->DEAD, simply because of a session > >timeout on the server side, what are my options on the client > >side to reconnect gracefully and to prevent the loss of the > >messages still queued on the client side? > > > >The client still happily accepts messages for queueing even > >though the connection state is DEAD and returns no errors. > > > > -- > Marcel Ruff > http://www.xmlBlaster.org > http://watchee.net > Phone: +49 7551 309371 >
Re: [xmlblaster] Client connection not being restored properly?
On Wed, Apr 02, 2008 at 07:42:54PM +0200, Marcel Ruff wrote: > Hi David, > > if the server side session times out the expected behavior is > that the client goes to dead (what you client seem to do). > That your client recovers from DEAD is not intended > and is considered a bug, see > http://www.xmlblaster.org/xmlBlaster/doc/requirements/client.failsafe.html#failsave > The transition (9) and (8) should not happen. > > > Thanks for reporting > best regards > Marcel Hi Marcel, Whoops! Then I guess the original issue is still outstanding then. If my client goes from ALIVE->DEAD, simply because of a session timeout on the server side, what are my options on the client side to reconnect gracefully and to prevent the loss of the messages still queued on the client side? The client still happily accepts messages for queueing even though the connection state is DEAD and returns no errors. -- David Kerry
Re: [xmlblaster] Client connection not being restored properly?
:45:36,605 ERROR [STDERR] 1-Apr-2008 1:45:36 PM org.xmlBlaster.util.dispatch.DispatchManager switchToASyncMode INFO: connection:client/app_prod/12: Switched to asynchronous message delivery -- David Kerry
Re: [xmlblaster] Client connection not being restored properly?
On Wed, Feb 27, 2008 at 06:28:59PM +0100, Marcel Ruff wrote: > > > >My way of working around this for now is to create yet another queue, > >in my application that keeps a copy of the saved messages until a > >connection is re-established after which it puts them back in the > >xmlblaster queue for publishing. > > > >This works, but it's, well, ugly. Is this the intended client behaviour > >in this situation? Keep in mind that I'm not concerned about subscriptions > >and incoming messages to this client, if that makes a difference. > > > Hi David, > > what about starting the client in fail safe mode? > > Like this the client retries to establish a connection and sends the > queued messages on reconnect, > > regards, > Marcel Hi Marcel, I think I was mistaken. I think I am connecting in fail-safe mode, if 'fail-safe' mode means setting up the connection reconnect/ping/retry values. I am not specifying an explicit session id, however (I get a negative id back upon connect). For reference, here are the connect properties I'm using: protocol=SOCKET dispatch/connection/pingInterval=3 dispatch/connection/retries=-1 dispatch/connection/delay=5000 dispatch/connection/plugin/socket/port=7607 dispatch/connection/plugin/socket/hostname= queue/connection/defaultPlugin=RAM,1.0 The rest are whatever defaults that xmlblaster client provides. >From reading the reference docs, I think this actually does enable failsafe mode, correct? -- David Kerry
Re: [xmlblaster] Client connection not being restored properly?
On Mon, Feb 25, 2008 at 02:48:33PM +0100, Marcel Ruff wrote: > David Kerry wrote: > >Hello All, > > > >I'm seeing some odd behaviour on one of my clients here and I'm > >not sure if it's a bug, feature, or configuration problem. Client > >and server are running the 1.6.2 version of code. > > > Hi David, > > 1. Why is secretSessionId invalid? > We would need to see the server log for this, possibly the session > timed out. > > 2. Is the client connecting in fail safe mode? > o With public session ID > 0: -session.name spiderman_test/1 > o -dispatch/callback/retries -1 > > To detect going DEAD you can register > xmlBlasterAccess.registerConnectionListener() > and react Kerry-like (see xmlBlaster/demo/HelloWorld4.java). > > Still it would be good to know why the secret sessionId got invalid. > > best regards > Marcel, Michele Hi Marcel, Well, I figured this out - it was a bit of a misunderstanding on the session timeout parameter. I didn't look close enough and thought it was in seconds instead of milliseconds. Thus, my client's sessions kept timing out on the server side much too fast! The client wasn't connecting in fail safe mode, and I was using a connection listener to monitor the connection, which helped my figure out what was going on. However, the client behaviour when the server times out the session is still a little off, I believe. What I see happening is the server kills the client session (fine), but the client doesn't notice that fact until it tries to use the connection (ie: publish), even though it has a connection-ping setup for every 10 seconds. Once the publish happens, the client goes 'oh, my connection is dead' and then starts queuing messages. At this point, however, the client never goes into a polling mode and the only way to re-establish communications is to do a disconnect(), re-instatiate a new XmlBlasterAccess and do another connect(). This whole process also has the side-effect of erasing all the queued messages. If I try to simply call connect() again, I get an exception stating "connect() rejected, you are connected already, please check your code". My way of working around this for now is to create yet another queue, in my application that keeps a copy of the saved messages until a connection is re-established after which it puts them back in the xmlblaster queue for publishing. This works, but it's, well, ugly. Is this the intended client behaviour in this situation? Keep in mind that I'm not concerned about subscriptions and incoming messages to this client, if that makes a difference. -- David Kerry
[xmlblaster] Client connection not being restored properly?
tStream.close(): java.net.SocketException: Socket is closed Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchConnectionsHandler removeDispatchConnection FINE: connection:client/ejb_test1203703214006: Destroyed one callback connection, 0 remain. Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchConnectionsHandler updateState FINE: connection:client/ejb_test1203703214006: updateState() oldState=ALIVE conList.size=0 Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager shutdownFomAnyState FINER: connection:client/ejb_test1203703214006: Switch from ALIVE to DEAD Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.XmlBlasterAccess toDead FINER: client/ejb_test/-42: Changed from connection state ALIVE to DEAD connectInProgress=false Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager givingUpDelivery FINE: connection:client/ejb_test1203703214006: Entering givingUpDelivery(), state is DEAD Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.ClientErrorHandler handleError FINER: Entering handleError for 0 messages Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.protocol.socket.SocketCallbackImpl run FINE: Terminating socket callback thread Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager shutdown FINER: connection:client/ejb_test1203703214006: Entering shutdown ... Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchConnectionsHandler shutdown FINER: connection:client/ejb_test1203703214006: Entering shutdown ... Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager handleSyncWorkerException FINER: connection:client/ejb_test1203703214006: Sync delivery failed connection state is DEAD: errorCode=communication.noConnection.dead message=#exported Your secretSessionId is invalid, no access to spiderman_test. : O riginal erroCode=user.security.authentication.accessDenied Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager checkSending FINE: connection:client/ejb_test1203703214006: The dispatcher is shutdown, can't activate callback worker thread false Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager switchToASyncMode INFO: connection:client/ejb_test1203703214006: Switched to asynchronous message delivery Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.dispatch.ClientDispatchConnectionsHandler createFakedReturnObjects FINER: Entering createFakedReturnObjects() for 1 entries Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.dispatch.ClientDispatchConnectionsHandler createFakedReturnObjects FINE: Creating faked return for 'publish' invocation Feb 22, 2008 2:36:02 PM org.xmlBlaster.util.dispatch.DispatchManager handleSyncWorkerException FINE: connection:client/ejb_test1203703214006: Delivery failed, pushed 1 entries into tail back queue Feb 22, 2008 2:36:02 PM org.xmlBlaster.client.XmlBlasterAccess queueMessage FINE: client/ejb_test/-42: Forwarded one 'publish' message, current state is DEAD Feb 22, 2008 2:36:59 PM org.xmlBlaster.util.dispatch.DispatchConnection finalize FINE: DispatchConnection-connection:client/ejb_test1203703214006 finalize - garbage collected Feb 22, 2008 2:36:59 PM org.xmlBlaster.util.queuemsg.MsgQueueEntry finalize FINE: finalize - garbage collect At this point, the client is still running, but it never reconnects back to the server, regardless of how many publish() calls are made - the messages are effectively lost in the client-side queue now. Is this a bug, or a configuration issue or is there some method I should be calling to 'kick' the connection back up and allow the queued messages to get published to the server? -- David Kerry
[xmlblaster] NullPointer exception
Hello all, We just ran into another exception in the server that seems to have totally confused our clients: Jun 28, 2006 8:41:17 AM org.xmlBlaster.engine.AvailabilityChecker checkException SEVERE: publishArr() failed: XmlBlasterException serverSideException=true node=[xmlBlaster_172_23_254_15_10412] location=[TopicHandler] java.lang.NullPointerException stackTrace=java.lang.NullPointerException at org.xmlBlaster.authentication.SessionInfo.toXml(SessionInfo.java:594) at org.xmlBlaster.engine.TopicHandler.checkIfAllowedToSend(TopicHandler.java:1219) at org.xmlBlaster.engine.TopicHandler.invokeCallback(TopicHandler.java:1342) at org.xmlBlaster.engine.TopicHandler.invokeCallbackAndHandleFailure(TopicHandler.java:1202) at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:660) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1704) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1495) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1489) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:198) at org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:405) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:232) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:354) at java.lang.Thread.run(Thread.java:595) errorCode=internal.unknown message=#15171M This is an unknown and unexpected error, usually a Java runtime exception, please post it to the mailing list. -> http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.unknown at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:669) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1704) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1495) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1489) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:198) at org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:405) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:232) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:354) at java.lang.Thread.run(Thread.java:595) versionInfo=version=1.2,revision=15171M,os.name=Linux,os.version=2.6.15.6,java.vm.vendor=Sun Microsystems Inc.,java.vm.version=1.5.0_06-b05,os.arch=i386,build.timestamp=05/24/2006 09:56 AM,build.java.vendor=Sun Microsystems Inc.,build.java.version=1.5.0_04 errorCode description=http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.unknown Jun 28, 2006 8:41:17 AM org.xmlBlaster.protocol.socket.HandleClient handleMessage FINE: Can't handle message, throwing exception back to client: errorCode=internal.unknown message=#15171M This is an unknown and unexpected error, usually a Java runtime exception, please post it to the mailing list. -> http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.unknown : java.lang.NullPointerException Also, since I'm here - any word on resolving some of the other multi-thread race issues we keep banging into? For some reason, our access patterns seem to expose them more than anyone else it seems. -- David Kerry
Re: [xmlblaster] Missing messages
Some more info on this... Not sure if it will help, but I discovered this related exception in our logs that is tied to this event as well: java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1158) at org.xmlBlaster.engine.TopicHandler.entryDestroyed(TopicHandler.java:952) at org.xmlBlaster.engine.MsgUnitWrapper.toDestroyed(MsgUnitWrapper.java:596) at org.xmlBlaster.engine.MsgUnitWrapper.setReferenceCounter(MsgUnitWrapper.java:308) at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:681) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1704) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1495) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1489) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:198) at org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:405) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:232) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:354) at java.lang.Thread.run(Thread.java:595) -- David Kerry On Thu, May 25, 2006 at 01:54:18PM -0400, David Kerry wrote: > Hello All, > > We've managed to quell all the recent exceptions we've been having > with the head version of xmlblaster. However, we're still experiencing > a case of occasional lost messages. > > Every once in a long while, we'll lose a message. It gets published > successfully by the client, but no subscribers are ever notified that > they have a message and it seems to just disappear into the ether. > > This may or may not be related to the race condition we found earlier > in regards to topic handling (destroy/publish overlap). > > In our environment, it seems to happen on average once to 3 times per > day. > > Below is the error message we see in the logs when it happens (starts at > 'MsgUnitCache is unexpected null'): > > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.MsgUnitWrapper > FINE: Created new MsgUnitWrapper instance '[EMAIL PROTECTED]' > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler publish > FINE: Message http_post/2006-05-24 18:50:13.758 handled, now we can send > updates to all interested clients. > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler > invokeCallbackAndHandleFailure > FINE: Going to update dependent clients for http_post, subscriberMap.size() = > 0 > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.MsgUnitWrapper > setReferenceCounter > FINE: Reference count changed from 1 to 0, this='[EMAIL PROTECTED]' > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler entryDestroyed > FINER: Entering entryDestroyed(http_post/2006-05-24 18:50:13.758) > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler entryDestroyed > SEVERE: MsgUnitCache is unexpected null, topic: > state='UNCONFIGURED'> > http_post >id='__subId:client/edserver_prod1148479503488-114850521601701:114851101381300'/> > true > > > msgUnitWrapper is: > state='DESTROYED'> > > >/node/xmlBlaster_172_23_254_15_10412/client/app_post/-10 > > >false > > timestamp='114851101375800' dirtyRead='false'/> > > > > > > http_postresponse68970E0FFE0E00F5C3A617D2D9A7post > > base-64 content removed > > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler toUnreferenced > FINER: Entering toUnreferenced(oldState=UNCONFIGURED, > onAdministrativeCreate=false) > 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler toDead > FINER: Entering toDead(oldState=UNREFERENCED) > > -- > David Kerry >
[xmlblaster] Missing messages
Hello All, We've managed to quell all the recent exceptions we've been having with the head version of xmlblaster. However, we're still experiencing a case of occasional lost messages. Every once in a long while, we'll lose a message. It gets published successfully by the client, but no subscribers are ever notified that they have a message and it seems to just disappear into the ether. This may or may not be related to the race condition we found earlier in regards to topic handling (destroy/publish overlap). In our environment, it seems to happen on average once to 3 times per day. Below is the error message we see in the logs when it happens (starts at 'MsgUnitCache is unexpected null'): 24-May-2006 6:50:13 PM org.xmlBlaster.engine.MsgUnitWrapper FINE: Created new MsgUnitWrapper instance '[EMAIL PROTECTED]' 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler publish FINE: Message http_post/2006-05-24 18:50:13.758 handled, now we can send updates to all interested clients. 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler invokeCallbackAndHandleFailure FINE: Going to update dependent clients for http_post, subscriberMap.size() = 0 24-May-2006 6:50:13 PM org.xmlBlaster.engine.MsgUnitWrapper setReferenceCounter FINE: Reference count changed from 1 to 0, this='[EMAIL PROTECTED]' 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler entryDestroyed FINER: Entering entryDestroyed(http_post/2006-05-24 18:50:13.758) 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler entryDestroyed SEVERE: MsgUnitCache is unexpected null, topic: http_post true msgUnitWrapper is: /node/xmlBlaster_172_23_254_15_10412/client/app_post/-10 false http_postresponse68970E0FFE0E00F5C3A617D2D9A7post base-64 content removed 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler toUnreferenced FINER: Entering toUnreferenced(oldState=UNCONFIGURED, onAdministrativeCreate=false) 24-May-2006 6:50:13 PM org.xmlBlaster.engine.TopicHandler toDead FINER: Entering toDead(oldState=UNREFERENCED) -- David Kerry
[xmlblaster] More exceptions
icHandler.publish(TopicHandler.java:549) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1704) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1495) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1489) at org.xmlBlaster.engine.XmlBlasterImpl.publish(XmlBlasterImpl.java:165) at org.xmlBlaster.protocol.xmlrpc.XmlBlasterImpl.publish(XmlBlasterImpl.java:115) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.xmlrpc.Invoker.execute(Unknown Source) at org.apache.xmlrpc.XmlRpcServer$Worker.executeInternal(Unknown Source) at org.apache.xmlrpc.XmlRpcServer$Worker.execute(Unknown Source) at org.apache.xmlrpc.XmlRpcServer.execute(Unknown Source) at org.apache.xmlrpc.WebServer$Connection.run(Unknown Source) at org.apache.xmlrpc.WebServer$Runner.run(Unknown Source) at java.lang.Thread.run(Thread.java:595) errorCode=internal.nullpointer message=#15163 errorCode=internal.publish: An internal error occurred when processing a publish() request. /node/xmlBlaster_172_23_254_15_10412/client/VERID_prod/-16 at org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:689) at org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:678) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1724) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1495) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1489) at org.xmlBlaster.engine.XmlBlasterImpl.publish(XmlBlasterImpl.java:165) at org.xmlBlaster.protocol.xmlrpc.XmlBlasterImpl.publish(XmlBlasterImpl.java:115) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.xmlrpc.Invoker.execute(Unknown Source) at org.apache.xmlrpc.XmlRpcServer$Worker.executeInternal(Unknown Source) at org.apache.xmlrpc.XmlRpcServer$Worker.execute(Unknown Source) at org.apache.xmlrpc.XmlRpcServer.execute(Unknown Source) at org.apache.xmlrpc.WebServer$Connection.run(Unknown Source) at org.apache.xmlrpc.WebServer$Runner.run(Unknown Source) at java.lang.Thread.run(Thread.java:595) versionInfo=version=1.2,revision=15163,os.name=Linux,os.version=2.6.15.6,java.vm.vendor=Sun Microsystems Inc.,java.vm.version=1.5.0_06-b05,os.arch=i386,build.timestamp=05/19/2006 09:53 AM,build.java.vendor=Sun Microsystems Inc.,build.java.version=1.4.2_09 errorCode description=http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.nullpointer -- David Kerry
[xmlblaster] Missing volatile messages?
Hello, Looks like we're having another problem with the latest 1.1.1 release. A large number of our messages are volatile, with some persistent ones thrown in for good measure. In our previous version (1.0.6), all was fine - both volatile and persistent messages worked. However, in 1.1.1, we're having a big problem with the volatile messages. If we do a clean start, everything seems ok - messages are distributed properly. But then, randomly, it seems, the volatile messages will just start disappearing into a black hole. Here's the pattern: Client A publishes a volatile message on oid 'X', xmlblaster receives it and acknowledges it. Subscriber B, subscribed with an XPATH subscription, is never notified. However, if client A now publishes the exact same message again, but using oid 'Z', subscriber B *does* receive it. Very weird. It's like a given oid randomly turns into a black hole for messages. We're still in the process of tracing this one down, but it looks to be very difficult to pintpoint. Even with the server logs cranked up all the way (call,trace,dump), there are no obvious problems indicated. Messages just plain disappear. Looking through the changelog, I note that #14007 might somehow be related? -- David Kerry
[xmlblaster] NullPointerException
Hi, We've just upgraded to the latest 1.1.1 version, and it looks like the spurious NullPointerException problem we've been experiencing is still present. We see some strange behaviour once in awhile too - messages disappearing, for example. The client shows successfully published, but the subscriber never gets notified. I looked through all the logs and I only found one exception (which I believe coincided roughly with the time that the message disappeared): java.lang.NullPointerException at org.xmlBlaster.engine.TopicHandler.entryDestroyed(TopicHandler.java:928) at org.xmlBlaster.engine.MsgUnitWrapper.toDestroyed(MsgUnitWrapper.java:595) at org.xmlBlaster.engine.MsgUnitWrapper.setReferenceCounter(MsgUnitWrapper.java:307) at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:658) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1677) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1483) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1477) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:185) at org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:402) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:231) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:352) at java.lang.Thread.run(Thread.java:595) Now, I have discovered what I think it is a race condition that may trigger this. I found it using the demo client code to publish some test messages to our clients. If client A publishes a persistent message, client B receives it, then client A *and* client B try to erase it at around the same time, the xmlblaster server will very often explode with a nullpointerexception that gets thrown back to the client, like this: Mar 8 08:56:55 wolverine1 error publishing response message errCode=internal.nullpointer, message=XmlBlasterException serverSideException=true node=[xmlBlaster_172_23_254_15_10412] location=[RequestBroker] Mar 8 08:56:55 wolverine1 java.lang.NullPointerException Mar 8 08:56:55 wolverine1 stackTrace=java.lang.NullPointerException Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.TopicHandler.entryDestroyed(TopicHandler.java:928) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.MsgUnitWrapper.toDestroyed(MsgUnitWrapper.java:595) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.MsgUnitWrapper.setReferenceCounter(MsgUnitWrapper.java:307) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:658) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1677) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1483) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1477) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:185) Mar 8 08:56:55 wolverine1 at org.xmlBlaster.util.protocol.RequestReplyExecutor.receiveReply(RequestReplyExecutor.java:402) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(Han This was fairly easy to reproduce depending on how long I took to hit a key in the HelloWorldPublish demo after it published the message, but before it erased it. -- David Kerry
[xmlblaster] NullPointer exception
Hello, We've been using the latest xmlblaster for a bit now and have started getting these popping out once in a while in the logs: [Sep 28, 2005 8:56:55 AM ERROR XmlBlaster.SOCKET.tcpListener-edserver_prod AvailabilityChecker] publishArr() failed: XmlBlasterException serverSideException=true node=[xmlBlaster_172_23_254_15_10412] location=[RequestBroker-/node/xmlBlaster_172_23_254_15_10412] java.lang.NullPointerException stackTrace=java.lang.NullPointerException at org.xmlBlaster.engine.TopicHandler.entryDestroyed(TopicHandler.java:946) at org.xmlBlaster.engine.MsgUnitWrapper.toDestroyed(MsgUnitWrapper.java:580) at org.xmlBlaster.engine.MsgUnitWrapper.setReferenceCounter(MsgUnitWrapper.java:309) at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:676) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1665) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1471) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1465) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:194) at org.xmlBlaster.protocol.socket.Executor.receive(Executor.java:312) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:211) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:320) at java.lang.Thread.run(Thread.java:534) errorCode=internal.nullpointer message=#exported errorCode=internal.publish: An internal error occurred when processing a publish() request. /node/xmlBlaster_172_23_254_15_10412/client/edserver_prod/ -14 at org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:664) at org.xmlBlaster.util.XmlBlasterException.convert(XmlBlasterException.java:653) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1685) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1471) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1465) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:194) at org.xmlBlaster.protocol.socket.Executor.receive(Executor.java:312) at org.xmlBlaster.protocol.socket.HandleClient.handleMessage(HandleClient.java:211) at org.xmlBlaster.protocol.socket.HandleClient.run(HandleClient.java:320) at java.lang.Thread.run(Thread.java:534) versionInfo=version=1.0.6,revision=exported,os.name=Linux,os.version=2.4.23,java.vm.vendor=Sun Microsystems Inc.,java.vm.version=1.4.2_03-b02,os.arch=i386,build.timestamp=08/10/2005 01:45 PM,build.ja va.vendor=Sun Microsystems Inc.,build.java.version=1.4.2_06 errorCode description=http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.nullpointer Restarting xmlblaster makes it go away (for now), but I suspect it may return. While all this was going on, I also got errors from the client attempting to subscribe. The error in the server log is: [Oct 1, 2005 7:24:18 PM WARN XmlBlaster.SOCKET.tcpListener-core900_staging_ctb XmlKeyDom.InvalidQuery] Sorry, can't access, query syntax is wrong for '//service[.="ivs-123.com" and ../type[.="response"]]|//service[.="ivs-abc" and ../type[.="response"]]' : java.lang.NullPointerException The error the client received was: 2005-10-01 19:24:18,358 ERROR [XmlBlaster.PingTimer] [logic.XmlBlasterSyncLogic]: exception subscribing to messages:XmlBlasterException errorCode=[user.query.invalid] serverSideException=true location=[XmlKeyDom] message=[#exported Sorry, can't access, query syntax of '//service[.="ivs-123.com" and ../type[.="response"]]|//service[.="ivs-abc" and ../type[.="response"]]' is wrong : java.lang.NullPointerException] [See URL http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#user.query.invalid] errorCode=user.query.invalid message=#exported Sorry, can't access, query syntax of '//service[.="ivs-123.com" and ../type[.="response"]]|//service[.="ivs-abc" and ../type[.="response"]]' is wrong : java.lang.NullPointerException at org.xmlBlaster.util.XmlBlasterException.parseByteArr(XmlBlasterException.java:637) at org.xmlBlaster.protocol.socket.Parser.getException(Parser.java:482) at org.xmlBlaster.protocol.socket.Executor.receive(Executor.java:426) at org.xmlBlaster.client.protocol.socket.SocketCallbackImpl.run(SocketCallbackImpl.java:189) at java.lang.Thread.run(Thread.java:534) -- David Kerry
Re: [xmlblaster] ANNOUNCE xmlBlaster release 1.RC3 *stable*
On Wed, Dec 08, 2004 at 06:44:06PM +0100, Marcel Ruff wrote: > Hi, > > xmlBlaster version 1.RC3 *stable* is available. > > > This is the third release candidate for version 1.0. > We plan to release version 1.0 in a few days. > > Please download this version and test it in your environment > a.s.a.p and report all problems to our mailing list, > it is the last chance to add fixes for version 1.0. > Having covered your problems for 1.0 is for your own benefit. Hello, I have found one issue, not sure if it's a show stopper, but it is rather messy... The c/c++ source code all assumes it can use the constants LOG_INFO, LOG_ERROR, etc freely (src/c/util/helper.h, for example, similar in c++ section). Unfortunately, every unix syslog.h pre-defined some of those constants on you many decades ago... I cannot build anything against the xmlblaster c/c++ libraries if I need to include syslog support in my application (parse errors in xmlblaster header files is the result). I would recommend renaming those constants (XBLOG_xxx maybe?), or using the syslog.h versions instead. -- David Kerry
Re: [xmlblaster] Server exception
On Fri, Jun 11, 2004 at 10:30:03AM +0200, Marcel Ruff wrote: > Marcel Ruff wrote: > >David Kerry wrote: > > > >>Hi, > >> > >>I've been trying out the latest cvs code of xmlblaster and the > >>following error got thrown out into the log. > >> > >>After this, one of our clients started misbehaving as well > >>and restarting didn't seem to help (the server thought the > >>client was still logged in - the session was never disconnected). > >> > > > >Hi David, > > > >this looks like a bug. > >Is it reproducable? > >Could you please send to me directly the complete log file, > >if possible a stack dump and xml dump > >and a description of the publishQos and subscriber involved? > > > >thanks > > > >Marcel > > > David, > > is it possible that you updated xmlBlaster to 0.903 and > worked on old data (from persistent messages in DB) > from a previous xmlBlaster release? > > regards, > > Marcel Hi, I don't think so. In the scenario where I saw this, I updated our v.0901 xmlblaster by just overwriting the lib/xmlBlaster.jar file with the latest cvs version. After that, I restarted the server. The server itself is configured to use only ram-based storage, so it all should have been flushed out on a restart. Unless the server is keeping some hidden files around somewhere I don't know about... :-) The bug isn't really reproducible at the moment - it has happened only twice so far in a full day of operation. You are in luck however - I happened to do a server dump after it happened and before I restarted it. I'd rather send it off-list, so if you can send me an address to send it to I'll gladly forward it. I'll also include the log file too. As for our situation, we have probably 10 clients, all pub-sub capable and using xpath subscriptions. They are all long-lived and never shut down (unless unexpectedly). All messages are volatile. -- David Kerry
[xmlblaster] Server exception
Hi, I've been trying out the latest cvs code of xmlblaster and the following error got thrown out into the log. After this, one of our clients started misbehaving as well and restarting didn't seem to help (the server thought the client was still logged in - the session was never disconnected). -- David Kerry ([EMAIL PROTECTED]) [Jun 10, 2004 8:37:46 PM ERROR XmlBlaster.DispatchWorkerPool.xmlBlaster_172_23_254_15_3412-1 DispatchManager-callback:/node/xmlBlaster_172_23_254_15_3412/client/eds_prod/-12] Callback failed: errorCode=internal.unknown message=Message 108691426684501 not found [Jun 10, 2004 8:37:46 PM ERROR XmlBlaster.DispatchWorkerPool.xmlBlaster_172_23_254_15_3412-1 DispatchManager-callback:/node/xmlBlaster_172_23_254_15_3412/client/eds_prod/-12] PANIC: Internal error, doing shutdown: XmlBlasterException serverSideException=true node=[xmlBlaster_172_23_254_15_3412] location=[MsgQueueUpdateEntry] stackTrace=errorCode=internal.unknown message=Message 108691426684501 not found at org.xmlBlaster.engine.queuemsg.ReferenceEntry.getMsgUnit(ReferenceEntry.java:264) at org.xmlBlaster.engine.dispatch.CbDispatchConnection.doSend(CbDispatchConnection.java:119) at org.xmlBlaster.util.dispatch.DispatchConnection.send(DispatchConnection.java:203) at org.xmlBlaster.util.dispatch.DispatchConnectionsHandler.send(DispatchConnectionsHandler.java:362) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:166) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:732) at java.lang.Thread.run(Thread.java:534) versionInfo=version=0.903,os.name=Linux,os.version=2.4.23,java.vm.vendor=Sun Microsystems Inc.,java.vm.version=1.4.2_03-b02,os.arch=i386,build.timestamp=06/10/2004 11:16 AM,build.java.vendor=Sun Microsystems Inc.,build.java.version=1.4.2_03 errorCode description=http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#internal.unknown
Re: [xmlblaster] When does a topic get marked dead?
On Wed, Jun 09, 2004 at 10:47:46PM +0200, Michele Laghi wrote: > Hi David, > > David Kerry wrote: > >I hit enter to publish a message and it gets published, the > >topic gets created on the server and is subsequently marked 'DEAD'. > > > >Two messages show up in the subscriber window - the publish and > >the erase event. This is fine. > > > >Now, I exit the publish client and run it again, exactly the same. > > > When you exit the publisher an erase is done. This erase kills the > subscriptions on that topic (The subscriber demo does not subscribe on > an xpath but on the topic). Subscribing again (or not erasing the topic) > solves the problem here but this is not the problem you describe below > (since xpath subscriptions are not deleted). Ahh.. so that explains that bit. > >>From this point forwards, no messages ever get received by the > >subscriber client, regardless of how many times I run the > >publish client. > > > >I was trying various combinations of topic destroy delays, maxentry > >settings and whatnot to get things to work in my environment, and > >this pretty much seems to be the behaviour I'm seeing. The publish > >is made successfully by the publisher, the topic is cleaned up by > >the server, but any subscribers to that message (using an xpath > >subscription on the message key) never receive the messages. > > > >Any ideas? Is this a bug, or am I likely doing something wrong > >still? > > > Could you send a small example of your subscription and publish keys > together with the dump of the xmlBlaster ? > > >Oh, fyi - I'm using version 0.901 at the moment. > > > Could you also try the same on 0.903 ? > > Michele Ok, here's a simple setup that demonstrates the problem quite nicely: Note that in this case, the subscriber receives the published message, but in my case, this doesn't happen. I think I may have something goofed up in the qos (I'm using a custom rolled c client - written before the current c/c++ library was up to snuff). run the server - all defaults (v.903 has the same problem, btw). Run a subscriber client, but use an xpath subscription: java javaclients.HelloWorldSubscribe -xpath "//org.xmlBlaster" Then, run a publisher (disabling erase, enabling volatile messages and unique oids per message): java javaclients.HelloWorldPublish -lifeTime 0 -destroyDelay 0 -session.name foobar -oid "" -erase false -numPublish 100 Then just hit enter a whole lot of times to create 100 new Topics on the server. Note that they never get deleted. Or rather, they never get deleted as long as the subscriber client is still connected. The *do* seem to get deleted if the subscriber is not using an xpath subscription... -- David Kerry
Re: [xmlblaster] When does a topic get marked dead?
On Tue, Jun 08, 2004 at 06:08:19PM +0200, Marcel Ruff wrote: > Try this: > > java org.xmlBlaster.Main -call[core] true > > java javaclients.HelloWorldPublish -lifeTime 0 -destroyDelay 0 > > you can see in the server log how the topic disappears > (type 'd' in the server console to verify it). Ok, I tried this, and it did not work quite as expected. Here's what I actually did: Run the server: "java org.xmlBlaster.Main" In another window, ran the subscribe demo client: "java javaclients.HelloWorldSubscribe" I hit a key and let the subscribe client enter a subscription. Then, in another window, the publish client: "java javaclients.HelloWorldPublish -lifeTime 0 -destroyDelay 0" I hit enter to publish a message and it gets published, the topic gets created on the server and is subsequently marked 'DEAD'. Two messages show up in the subscriber window - the publish and the erase event. This is fine. Now, I exit the publish client and run it again, exactly the same. >From this point forwards, no messages ever get received by the subscriber client, regardless of how many times I run the publish client. I was trying various combinations of topic destroy delays, maxentry settings and whatnot to get things to work in my environment, and this pretty much seems to be the behaviour I'm seeing. The publish is made successfully by the publisher, the topic is cleaned up by the server, but any subscribers to that message (using an xpath subscription on the message key) never receive the messages. Any ideas? Is this a bug, or am I likely doing something wrong still? Oh, fyi - I'm using version 0.901 at the moment. -- David Kerry > > Another way is to switch off history: > > java javaclients.HelloWorldPublish -destroyDelay 0 > -queue/history/maxEntries 0 > > as the message instance is not kept in history > as soon as it is delivered to all subscribers the topic > is unreferenced and dies immediately. > -- > > You can start the server with > > java org.xmlBlaster.Main -call[core] true -topic.destroyDelay 0 > > (or put it into xmlBlaster.properties). Now the default > topic destroy delay is 0. If a client publishes a longer > destroy delay this will win. > > > You should use version 0.903 as this is the most stable > on the 0.9x branch. > > best regards > > Marcel > > > -- > http://www.xmlBlaster.org
Re: [xmlblaster] When does a topic get marked dead?
On Tue, Jun 08, 2004 at 06:08:19PM +0200, Marcel Ruff wrote: > David Kerry wrote: > >So I guess the question is - how can I force xmlblaster to check > >for dead topics more often and clean them out? > > Hi David, > > you've summarized it quite exactly already. > The best solution is to use named topics as communication > channels. If you use key oid='' we generate > a complete topic on the fly for every message instance send. > > If you still want to stick to tempary topics please use a > destroyDelay == 0. > In this case the topic is > destroyed immediately after dieing of your single messsage instance. > The default is 6 millisec (= one minute) which is > the problem you see. > > You should also send the messages volatile otherwise > the message instance lives forever in the history queue > and holds the topic alive as well > (switching off the history queue > with '-queue/history/maxEntries 0' has the > same effect in your case). > best regards Hmm.. in our case, we're only sending about 10-20 messages per minute, so the volume isn't very high. The empty topics are hanging around for far longer than 1 minute. I see some that are many hours old before I see them getting marked dead (4+ hours). All our messages are already sent volatile so they're emtpy almost immediately. Thank you for the examples - I'll give them a try shortly and investigate further. > > Marcel > > > -- > http://www.xmlBlaster.org -- David Kerry
Re: [xmlblaster] When does a topic get marked dead?
Hi Michele, Right now, we're using PubSub, all clients are connected before a publish (losing the odd message is ok in our environment) and we are setting the isVolatile() on all messages as well. Shouldn't setting the volatile flag (as we're already doing) cause the topics to get destroyed right away? Limiting the number of topics isn't an option at this moment, but we'll probably start moving in that general direction going forward. -- David Kerry On Tue, Jun 08, 2004 at 05:25:38PM +0200, Michele Laghi wrote: > hallo David, > > As I see it you have several possibilities to solve your problem: > first of all you should consider structuring your system in such a way > to get a limited amount of topics. Topic creation is timeconsuming and > should be kept to a minimum. > > An alternative to the described approach would be to limit the lifetime > of the topics by reducing the destroyDelay of the topic or the lifetime > of the message. Read more about it in the following requirement: > > http://www.xmlBlaster.org/xmlBlaster/doc/requirements/engine.message.lifecycle.html > > If you are using a PtP pattern or a PubSub where all subscribers are > connected before any publishing then you could also consider using the > volatile flag in the publish qos. > > As you see you have several options. Personally I would focus in trying > to limit the amount of topics. > > > Saluti > Michele > > > David Kerry wrote: > >Hello, > > > >We used to run xmlblaster v0.8 for quite some time, but have now > >moved to v0.9. > > > >Everything is running fine, but we have a problem where xmlblaster > >keeps running out of memory. > > > >We're using ram based storage for now (which we also used in the > >v0.8 version). This shouldn't be a problem as our traffic is > >fairly low and all messages are volatile. > > > >The problem seems to be that a new topic gets created for every > >message (when we publish we give a blank oid and let the server > >fill it in). The server then gets all plugged up with hundreds > >or thousands of topics with no messages in them. > > > >We also see this behaviour when we use a database (Oracle) and > >use it for all the topic/message storage. The database tables > >just keep getting bigger and bigger and never seem to shrink. > >This causes a problem when we restart xmlblaster too because it goes > >and loads in all the old entries from the database which can take > >a long time (10-15 minutes). > > > >So I guess the question is - how can I force xmlblaster to check > >for dead topics more often and clean them out? > > > > > -- > Michele Laghi > mailto:[EMAIL PROTECTED] > tel. +46 8 7492952 / mob. +46 70 4103964 > http://eclettic.tripod.com > http://www.xmlBlaster.org
[xmlblaster] When does a topic get marked dead?
Hello, We used to run xmlblaster v0.8 for quite some time, but have now moved to v0.9. Everything is running fine, but we have a problem where xmlblaster keeps running out of memory. We're using ram based storage for now (which we also used in the v0.8 version). This shouldn't be a problem as our traffic is fairly low and all messages are volatile. The problem seems to be that a new topic gets created for every message (when we publish we give a blank oid and let the server fill it in). The server then gets all plugged up with hundreds or thousands of topics with no messages in them. We also see this behaviour when we use a database (Oracle) and use it for all the topic/message storage. The database tables just keep getting bigger and bigger and never seem to shrink. This causes a problem when we restart xmlblaster too because it goes and loads in all the old entries from the database which can take a long time (10-15 minutes). So I guess the question is - how can I force xmlblaster to check for dead topics more often and clean them out? -- David Kerry
Re: [xmlblaster] Deliver to one of N subscribers?
Ahh... the description sounds perfect for my needs. In all my searching I missed that one somehow. I'll give it a go here and see how it works out. Thanks! -- David Kerry On Fri, May 07, 2004 at 07:47:42PM +0200, Michele Laghi wrote: > Hi David, > what you describe should be handled by the msgDistributorPlugin in > org.xmlBlaster.engine.distributor.* and described in the requirement > http://www.xmlBlaster.org/xmlBlaster/doc/requirements/msgDistributor.plugin.html > > The requirement is still open since extensive testing has not been done > yet. There is however already some simple testing in > xmlBlaster/testsuite/src/java/org/xmlBlaster/test/distributor which of > course could be extended. > > Have you tried this plugin already ? > > Regards > Michele > > > David Kerry wrote: > >Hello, > > > >I've seen this tossed around here before, but I haven't yet seen a > >straightforward solution to this problem. > > > >We have a need for a published message to be delivered to a group > >of identical clients, with the exception that each message should > >only be delivered to one client in the group at a time. > > > >I suspect this would be a new plugin type to make this work properly > >as the exsting ones don't quite seem to work at the level we need. > > > >The goal here is to have multiple clients all logged in for load > >sharing and/or failover purposes. To make this work, the message > >can only be delivered to one of the clients at a time (otherwise > >we'll end up with duplicated work by all of the clients). > > > >I have no simple way to pre-determine which client the message > >should be delivered to since that depends on which clients are > >connected at the time. > > > >Also, if the message is unsuccessfully delivered to one client, it > >should be possible to retry delivery to the next client, and so on, > >and if it can't be delivered at all, wait until a client connects > >that does satisfy the criteria and try again... > > > >I guess you could call this a clustered client problem. > > > >Anyone have any suggestions on how to make this work? > > > >Thanks > > > -- > Michele Laghi > mailto:[EMAIL PROTECTED] > tel. +46 8 7492952 / mob. +46 70 4103964 > http://eclettic.tripod.com > http://www.xmlBlaster.org
[xmlblaster] Deliver to one of N subscribers?
Hello, I've seen this tossed around here before, but I haven't yet seen a straightforward solution to this problem. We have a need for a published message to be delivered to a group of identical clients, with the exception that each message should only be delivered to one client in the group at a time. I suspect this would be a new plugin type to make this work properly as the exsting ones don't quite seem to work at the level we need. The goal here is to have multiple clients all logged in for load sharing and/or failover purposes. To make this work, the message can only be delivered to one of the clients at a time (otherwise we'll end up with duplicated work by all of the clients). I have no simple way to pre-determine which client the message should be delivered to since that depends on which clients are connected at the time. Also, if the message is unsuccessfully delivered to one client, it should be possible to retry delivery to the next client, and so on, and if it can't be delivered at all, wait until a client connects that does satisfy the criteria and try again... I guess you could call this a clustered client problem. Anyone have any suggestions on how to make this work? Thanks -- David Kerry
Re: [xmlblaster] volatile message problem
On Wed, Oct 30, 2002 at 09:51:18AM +0100, Cyrille Giquello wrote: > hello, > not about the error, but about Perl. > > You are using xmlBlaster::*.pm. > Are that packages good enough ? > did you make some changes to make them better ? > > cyrille No, not yet. I should be using those packages, but I've been in a bit of a rush at the moment, and copy/paste of the older examples has been my source of code for quick hacking/testing. As a quick observation, I do note that the xmlBlaster::*.pm modules seem to be missing subscribe/unsubscribe methods however. -- David Kerry
Re: [xmlblaster] volatile message problem
On Wed, Oct 30, 2002 at 08:50:26AM +0100, Marcel Ruff wrote: > David Kerry wrote: > > >Ok folks, another volatile message bug... > > > Hi David, > > it is good to have such a volatile tester! > I'm busy the next days but this needs to be resolved a.s.a.p. > > thanks for reporting > > Marcel This particular bug has been a bit of a critical issue for us, so I delved into the code and have come up with a solution for it. Please find attached a patch that corrects this bug. This bug fix does not resolve the other problems we're seeing with volatile messages still, however (see Russell Chan's most recent post for details). -- David Kerry Index: src/java/org/xmlBlaster/engine/RequestBroker.java === RCS file: /opt/cvsroot/xmlBlaster/src/java/org/xmlBlaster/engine/RequestBroker.java,v retrieving revision 1.177 diff -c -r1.177 RequestBroker.java *** src/java/org/xmlBlaster/engine/RequestBroker.java 2002/10/25 08:30:33 1.177 --- src/java/org/xmlBlaster/engine/RequestBroker.java 2002/10/31 05:32:40 *** *** 1214,1227 --- 1214,1235 } } + // Increment counter so MsgQueue doesn't purge our message while we're +still + // pushing it out to all the subscribers. + msgUnitHandler.getMessageUnitWrapper().addEnqueueCounter(1); + //- 2. now we can send updates to all interested clients: if (log.TRACE) log.trace(ME, "Message " + xmlKey.getKeyOid() + " handled, now we can send updates to all interested clients."); + if (contentChanged || publishQos.forceUpdate()) // if the content changed of the publisher forces updates ... msgUnitHandler.invokeCallback(sessionInfo, Constants.STATE_OK); //- 3. check all known query subscriptions if the new message fits as well // TODO: Only check if it is a new message (XmlKey is immutable) checkExistingSubscriptions(sessionInfo, xmlKey, msgUnitHandler, publishQos); + + // Unlock the counter now that we're done publishing + msgUnitHandler.getMessageUnitWrapper().addEnqueueCounter(-1); // First all callback calls must be successful - the CbWorker checks it as well if (msgUnitHandler.isPublishedWithData() &&
[xmlblaster] volatile message problem
Ok folks, another volatile message bug... This time, it seems that not all subscribers get updated when a volatile message is published. Sometimes it makes it to all of them, sometimes only one. Attached is a perl test program that demonstrates the problem. Just run xmlblaster locally, then run the test program against it like this: perl server2.pl http://127.0.0.1:1/ (substitute appropriate xml-rpc port above as necessary). It starts 4 callback clients, all with different logins, but the same xpath subscription. Then it publishes one message that will match that xpath. Every update callback that gets invoked will print 'UPDATE' from the corresponding client. The problem is I only get one update, when there should be 4 of them (one for each subscriber). You'll have to kill off the forked perl processes by hand for now - I didn't bother putting in shutdown code to get it to clean up properly. -- David Kerry server2.pl Description: Perl program
[xmlblaster] Another NPE with volatile messages
Hello, I've run across another NullPointerException in xmlBlaster. This time, it happens when we unsubscribe. The NPE is preceeded by a couple of other exceptions as well. The first is this: [Oct 29, 2002 10:13:26 AM TRACE XmlKey] Creating new XmlKey for isPublish=true java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Unknown Source) at org.xmlBlaster.engine.ClientSubscriptions.removeFromClientSubscriptionMap(ClientSubscriptions.java:441) at org.xmlBlaster.engine.ClientSubscriptions.subscriptionRemove(ClientSubscriptions.java:374) at org.xmlBlaster.engine.ClientSubscriptions.messageErase(ClientSubscriptions.java:294) at org.xmlBlaster.engine.RequestBroker.fireMessageEraseEvent(RequestBroker.java:1670) at org.xmlBlaster.engine.RequestBroker.eraseVolatile(RequestBroker.java:1319) at org.xmlBlaster.engine.queue.MsgQueue.checkForVolatileErase(MsgQueue.java:224) at org.xmlBlaster.engine.callback.CbWorker.run(CbWorker.java:65) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:728) The next is this [Oct 29, 2002 10:13:26 AM ERROR SubscriptionInfo] Failed to remove XPATH children subscription __subId:XPATH88 java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Unknown Source) at org.xmlBlaster.engine.SubscriptionInfo.removeChildSubscription(SubscriptionInfo.java:174) at org.xmlBlaster.engine.SubscriptionInfo.shutdown(SubscriptionInfo.java:403) at org.xmlBlaster.engine.ClientSubscriptions.subscriptionRemove(ClientSubscriptions.java:385) at org.xmlBlaster.engine.ClientSubscriptions.messageErase(ClientSubscriptions.java:294) at org.xmlBlaster.engine.RequestBroker.fireMessageEraseEvent(RequestBroker.java:1670) at org.xmlBlaster.engine.RequestBroker.eraseVolatile(RequestBroker.java:1319) at org.xmlBlaster.engine.queue.MsgQueue.checkForVolatileErase(MsgQueue.java:224) at org.xmlBlaster.engine.callback.CbWorker.run(CbWorker.java:65) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:728) at java.lang.Thread.run(Unknown Source) And our friend, at the usual spot: [Oct 29, 2002 10:13:26 AM TRACE SubscriptionInfo] Removed XPATH __subId:XPATH90 children subscription [Oct 29, 2002 10:13:26 AM TRACE RequestBroker-/node/http:10.0.1.190:3412] Erase event occured for oid=template.response.581, removing message from my map ... [Oct 29, 2002 10:13:26 AM TRACE MessageUnitHandler-/node/http:10.0.1.190:3412/msg/template.response.581] Entering erase() [Oct 29, 2002 10:13:26 AM TRACE RequestBroker-/node/http:10.0.1.190:3412] Published message is marked as volatile, erasing it [Oct 29, 2002 10:13:26 AM TRACE BigXmlKeyDOM] Erase event occured ... [Oct 29, 2002 10:13:26 AM ERROR CbWorker-/node/http:10.0.1.190:3412/client/EDserver/2] Unexpected exception: java.lang.NullPointerException java.lang.NullPointerException at org.xmlBlaster.engine.ClientSubscriptions.getSubscriptionByOid(ClientSubscriptions.java:210) at org.xmlBlaster.engine.ClientSubscriptions.messageErase(ClientSubscriptions.java:285) at org.xmlBlaster.engine.RequestBroker.fireMessageEraseEvent(RequestBroker.java:1670) at org.xmlBlaster.engine.RequestBroker.eraseVolatile(RequestBroker.java:1319) at org.xmlBlaster.engine.queue.MsgQueue.checkForVolatileErase(MsgQueue.java:224) at org.xmlBlaster.engine.callback.CbWorker.run(CbWorker.java:65) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:728) at java.lang.Thread.run(Unknown Source) I've attached a full log of the sequence of events for reference as well (gzipped). Sorry, I have no simple testcase to reproduce this one... -- David Kerry xmllog.txt.gz Description: Binary data
[xmlblaster] Another volatile message bug
Ok, now that the NPE problems have been resolved, we've run into another problem, probably related, where volatile messages are not deleted properly. What happens is xmlBlaster seems to get itself confused and not delete a volatile message when it should. After it gets into this state, it will never delete the message and continue to re-deliver it to each new subscriber/login that it applies to. It will do this forever as far as I can tell - ie: it will never expire the message either. Attached are test scripts to reproduce the problem. Protocol is XML-RPC, BTW. To reproduce, run the scripts in the following order: 1) perl server2.pl (in xterm #1) 2) perl serverlogin.pl (in xterm #2) 3) perl testpub.pl (in xterm #2) (server2.pl will spit out 'GOT UPDATE' to note that it got the published message via callback) 4) kill server2.pl (ctrl-c in xterm #1) 5) perl testpub.pl (in xterm #2) (xmlblaster will kill server2's login because it can no longer reach the callback server) 6) perl server2.pl (in xterm #1) 7) perl serverlogin.pl (in xterm #1) Note that server2.pl now says 'GOT UPDATE' again. It should not receive a callback at this point - the message was volatile so it should have been deleted immediately. Continued restarting of server2/serverlogin will cause the same message to get continually posted again and again when it should receive nothing at all. -- David Kerry testpub.pl Description: Perl program server2.pl Description: Perl program serverlogin.pl Description: Perl program
Re: [xmlblaster] NPE in ClientSubscriptions.getSubscriptionByOid
On Fri, Oct 25, 2002 at 04:42:29PM +0200, Marcel Ruff wrote: > Ok, i have comited it to xmlBlaster/demo/perl/xmlrpc as is (i've only > added some credits). > > Volunteers may extend the examples :-) > > thanks for your contribution > > Marcel > > PS: Has the NPE disappeared? Actually, the NPEs went away for the test case, but we encountered more NPEs when we pointed our development code against it. However, I noticed there were some more changes this morning, so I did another update and now with the latest changes, it no longer seems to have any problems. Thank you! -- David Kerry
Re: [xmlblaster] NPE in ClientSubscriptions.getSubscriptionByOid
On Fri, Oct 25, 2002 at 01:29:11PM +0200, Marcel Ruff wrote: > David Kerry wrote: > >Alright - I've attached two scripts that reproduce the problem > >right away. > > > >Just run the server script, then follow up with the test.pl script. > >No other work needed - it will cause xmlBlaster to NPE right away. > > > > > > > Thanks - this way it was easy to locate the problem. > I have fixed it and it is available via cvs on the main branch. > Sorry about the inconvenience. > > Your Perl code is cool, may i add it to the xmlBlaster/demo directory to > Cyrilles > examples (after adding credits to you)? > > best regards, > > Marcel Certainly. I wouldn't quite call those 'good' examples though... ;-) It would probably be better to rewrite them to use the xmlBlaster perl modules instead though (as the xmlBlasterClient.pl example does). -- David Kerry
Re: [xmlblaster] NPE in ClientSubscriptions.getSubscriptionByOid
On Wed, Oct 23, 2002 at 11:19:03AM -0400, Russell Chan wrote: > >I'm using XMLBlaster 0.80. I'm getting a NPE in > >ClientSubscriptions.getSubscriptionByOid... > > > > > > > >[Oct 18, 2002 4:44:50 PM ERROR > >CbWorker-/node/http:10.0.1.157:3412/client/guest/3] Unexpected > >exception: java.lang.NullPointerException > >java.lang.NullPointerException > >at > >>org.xmlBlaster.engine.ClientSubscriptions.getSubscriptionByOid(ClientSubscriptions.java:210) > >at > >>org.xmlBlaster.engine.ClientSubscriptions.messageErase(ClientSubscriptions.java:285) > >at > >org.xmlBlaster.engine.RequestBroker.fireMessageEraseEvent(RequestBroker.java:1669) > >at > >org.xmlBlaster.engine.RequestBroker.eraseVolatile(RequestBroker.java:1318) > >at > >org.xmlBlaster.engine.queue.MsgQueue.checkForVolatileErase(MsgQueue.java:212) > >at org.xmlBlaster.engine.callback.CbWorker.run(CbWorker.java:61) > >at > >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:728) > >at java.lang.Thread.run(Thread.java:498) > > > > > >All of my subscriptions are XPATH based, rather than OID based.. For > >example: > > > >SubscribeKeyWrapper subscribeKey = new > >SubscribeKeyWrapper( > >"//service[.=\""+EventQueueConstants.SERVICE_TEMPLATE_PARSE+"\" and "+ > >"../type[.=\""+EventQueueConstants.TYPE_RESPONSE+"\" > >and "+ > >"../id[.=\""+ eventQueueId+"\"]]]","XPATH"); > > > > > >I'm using primarily volatile messages. I see the NPE while the sweeper > >thread (for volatile messages) appears to be running. > > > >I'm going to attach a dump of the xmlblaster state. I *DO* notice that > >even during a dump xmlblaster complains about the "Can't determine key > >oid" (See the top of the dump). > > > >I'm obviously doing something wrong here... Insights appreciated :-) > > > If there are NPEs its not you but xmlBlaster which is fishy. > Can you provide a little demo client reproducing this problem? > > thanks > > Marcel Here's some more details... The exception from the latest CVS cut I get is this: [Oct 23, 2002 11:10:51 AM ERROR CbWorker-/node/http:127.0.0.1:3412/client/test/2] Unexpected exception: java.lang.NullPointerException java.lang.NullPointerException at org.xmlBlaster.engine.ClientSubscriptions.getSubscriptionByOid(ClientSubscriptions.java:210) at org.xmlBlaster.engine.ClientSubscriptions.messageErase(ClientSubscriptions.java:285) at org.xmlBlaster.engine.RequestBroker.fireMessageEraseEvent(RequestBroker.java:1669) at org.xmlBlaster.engine.RequestBroker.eraseVolatile(RequestBroker.java:1318) at org.xmlBlaster.engine.queue.MsgQueue.checkForVolatileErase(MsgQueue.java:212) at org.xmlBlaster.engine.callback.CbWorker.run(CbWorker.java:61) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:728) at java.lang.Thread.run(Thread.java:536) The first time we get this, it happens under the following circumstances: 1) All clients use XML-RPC protocol 2) Client #1 logs in and subscribes using an XPATH query 3) Client #2 logs in and starts publishing simple messages 4) Client #1 receives messages published by client #2 5) Client #1 is killed 6) xmlBlaster throws the above NullPointer exception when it tries to publish to the now non-existent client #2. For reference, client#2 is publishing messages like this: post request 123 truefalse And client #1 is subscribing like this: //service false Note also that after this event, if I restart client #1, it again starts receiving messages, but xmlBlaster will then start throwing out NullPointerExceptions very frequently (same as one above) and the memory consumption starts growing. -- David Kerry
