Re: unable to reconnect after exception Channel was inactive for too long
There may very well be a bug in the implementation of the InactivityMonitor code in ActiveMQCPP compared to the Java implementation; I'd recommend you submit a bug in JIRA, ideally with the client code that produces this behavior. This one may be harder for someone to track down, because I don't know of a way to reliably produce Channel was inactive for too long. exceptions. (If someone knows of a good way to do that, I'd love to know it.) Once this occurs, if you stop your application (but leave the broker up) and start a new one, does that eliminate the error? Knowing that will let us determine whether the bad state is in the client or in the broker. I can't think of any URI options that would help you work around the issue, though if the restart I asked about in the previous paragraph results in correct behavior, you could simply exit when you detect the problem and have the script that started it recognize that it was dead and start a fresh one. It's not a great workaround, but it might be better than what you have right now. Or of course you could turn off inactivity monitoring; you said you don't want to do that, but it might be better than the other alternatives, at least till someone figures out what's going on and implements a fix. Tim On Wed, Jun 17, 2015 at 1:25 PM, jeffrey jeffreyhasm...@hotmail.com wrote: Tim, I'm not sure if the Java application ever gets the exception, but if it is, then it's reconnecting quickly. I don't think my application can really control which port it is getting moved to, that would be decided after I connect to the server on 61616 and we negotiate to a new port. Are there any transport connector options on the client or server that could help me here? Thanks, Jeffrey Tim Bain wrote Do you ever get the channel inactivity exception in the Java client? If so, does it reconnect gracefully? Those error messages make it sound like the TCP connection is being reused instead of starting a fresh one. Is there any chance you're reusing the same ephemeral port, where the server wouldn't be able to tell that it wasn't simply additional data on the old connection? Tim -- View this message in context: http://activemq.2283324.n4.nabble.com/unable-to-reconnect-after-exception-Channel-was-inactive-for-too-long-tp4697843p4697923.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Potential threading bug in MemoryMessageStore.
But that's the thing; I think that method is properly synchronized, and
many of the other methods are over-synchronized.
Here's the code in question: from
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.activemq/activemq-osgi/5.11.1/org/apache/activemq/store/memory/MemoryMessageStore.java#MemoryMessageStore.addMessage%28org.apache.activemq.broker.ConnectionContext%2Corg.apache.activemq.command.Message%29
messageTable is a synchronized map obtained from
Collections.synchronizedMap(), so get() acquires the intrinsic lock before
issuing the get() against the underlying Map. The various other methods in
the class explicitly acquire that intrinsic lock via synchronized
(messageTable) {} blocks, so they're each acquiring the same lock (and
hence thread-safe). The other methods generally invoke only one other
method inside the synchronized block, and that method will acquire the same
intrinsic lock the sychronized block already holds; this works (because
intrinsic locks are reentrant), but it's unnecessary and your point about
double synchronization is accurate (but applicable to more than just
addMessage()).
I am, however, concerned that lastBatchId should probably be volatile since
it looks like it could be set from multiple threads. Either that, or
resetBatching() and setBatch() both need to use a synchronized block.
Tim
On Sat, Jun 20, 2015 at 8:43 PM, Kevin Burton bur...@spinn3r.com wrote:
I didn’t.. it’s a race so if it’s happening is probably happening not that
often . Maybe a test could be created to try to introduce the race.
On Thu, Jun 18, 2015 at 9:57 PM, Tim Bain tb...@alumni.duke.edu wrote:
Did you look at this any further? Looking at the code, it looks like the
call will be protected without explicit synchronization by the intrinsic
lock on the synchronizedMap (and I think that some other methods such as
delete() and addMessage() that just call a method on the synchronizedMap
could have their synchronized blocks removed), though I might be looking
at
that wrong.
Tim
On Mon, Apr 6, 2015 at 1:58 PM, Kevin Burton bur...@spinn3r.com wrote:
Pretty sure getMessage() in MemoryMessageStore has a bug.
All access to messageTable is synchronized. this method is not. This
means that there’s a race where a message can go into the queue but the
thread reading it may have a cache copy of the data structure meaning
it
would get a cache miss
Also, it looks like “addMessage” is doubly synchronized.
public Message getMessage(MessageId identity) throws IOException {
return messageTable.get(identity);
}
… I’m going to migrate to using a PriorityBlockingQueue for this and
remove
all the synchronization and will try to submit a patch. Also I think
PriorityBlockingQueue will lower memory usage by 40%
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
Re: ActiveMQ master-slave topology issue[BUG]
Thanks for the feedback and questions. I hadn't considered any of the buffering/flushing/synchronization aspects of the underlying NFS configuration, but you're absolutely right that the guidelines for how to configure this solution need to acknowledge the interplay between the two sets of settings and provide guidelines for how to select settings for each one in order to achieve the goals. I have no experience with configuring NFS, so although I understand the theory that writes can be deferred, it would be great to have people who understand those aspects be involved when that documentation gets written. Re: the type of lock to be used in the solution I'm using, I have no idea (due to lack of experience with NFS other than as a user) and would appreciate suggestions from those who are more knowledgeable about NFS. I had assumed we'd use the same type of lock as is currently being used (with the current behavior in the face of network failures), but if another type would be more appropriate, that would be great to know about. Re: the master re-reading the data file, I wasn't planning on it but it could be done. What I proposed would have a separate thread checking the content of the file on the same periodicity (but not necessarily synchronized so not necessarily occurring at the same time) as the writing thread. If locking is working correctly and network access is reasonably stable, if another process has written to the lock file (such that the write thread would see a difference just before it wrote), then the write would fail if the other process still held the lock, and it would succeed (as you'd want it to) if the other process had lost the lock due to losing network access. So under normal conditions, there wouldn't be a need to check content before writing. The only scenario I can think of where it might gain you something is if network access is winking in and out rapidly for the different processes, so they get writes in and then immediately lose the lock and someone else gets a write in and immediately loses the lock. If that's happening, and if the reading thread is timed so that it reads the file after the write thread writes to the file and before the other processes write to the file during this cycle, then multiple processes could think each think they've got the lock long enough to become master. That seems like a very unlikely scenario, but easy enough to guard against by doing the read-before-write that you asked about, so I think it's worth doing. Tim On Fri, Jun 19, 2015 at 10:19 AM, James A. Robinson j...@highwire.org wrote: On Mon, Jun 15, 2015 at 7:08 AM Tim Bain tb...@alumni.duke.edu wrote: It seems pretty clear that the assumption that acquiring a single file lock without doing any further checks will provide thread-safety in all cases is not an accurate one. As I see it, here are the failures of the existing approach: - Failure #1 is that when an NFS failure occurs, the master broker never recognizes that an NFS failure has occurred and never recognizes that the slave broker has replaced it as the master. The broker needs to identify those things even when it has no other reason to write to NFS. - Failure #2 is that the slave broker believes that it can immediately become the master. This wouldn't be a problem if the master broker instantaneously recognized that it has been supplanted and immediately ceded control, but assuming anything happens instantaneously (especially across multiple machines) is pretty unrealistic. This means there will be a period of unresponsiveness when a failover occurs. - Failure #3 is that once the master recognizes that it no longer is the master, it needs to abort all pending writes (in a way that guarantees that the data files will not be corrupted if NFS returns when some have been cancelled and others have not). I don't know if this has been called out already, but it will important for users to coordinate how their NFS is configured. For example, activating asynchronous writes on the NFS side would probably make a mess of any assumptions we can make from the client side. I also wonder how buffering might affect how a heartbeat would have to work, whether or not mtime won't get propogated until enough data has been written to cause a flush to disk. I've got a proposed solution that I think will address all of these failures, but hopefully others will see ways to improve it. Here's what I've got in mind: 1. It is no longer sufficient to hold an NFS lock on the DB lock file. In order to maintain master status, you must successfully write to the DB lock file within some time period. If you fail to do so within that time period, you must close all use of the DB files and relinquish master status to another broker. 2. When the master shuts down during normal NFS circumstances, it
Re: Performance issues regarding authentication
Since you're not getting any response here, I'd recommend you submit an enhancement request in JIRA. If there's a better way to do this, you'll get that feedback, and if there's not, you'll get a fix (eventually). Tim On Mon, Jun 15, 2015 at 7:16 AM, Shilen Patel shi...@duke.edu wrote: Hi folks, I'm running 5.11.1 and trying to get ActiveMQ to authenticate users against an LDAP via JAAS. I'm trying out org.apache.activemq.jaas.LDAPLoginModule but noticing a couple of issues. First, it doesn't seem to re-use LDAP connections (i.e. pooling) and on top of that, it doesn't seem to close connections that it creates (until they eventually timeout due to idling). Second, there's no caching. So the result that I'm seeing is if I send 100,000 messages (via the REST API in this particular case) and I authenticate as the same user each time, then that ends up authenticating the user 100K times against the LDAP using 100K different LDAP connections and without closing any of the 100K connections. What am I missing? Is there a better way to authenticate users against an LDAP? Thanks! - Shilen
Re: How to get added to the support page as a listed company
I can make that change. All the other entries on that page use third-person voice, so to be consistent how about the following: TytoEASE http://www.tytoease.com/activemq offers support, consulting and training on enterprise ready open source projects including ActiveMQ. Support offerings are at three levels with guaranteed SLA's. The company's consulting work is short term to help troubleshoot, fine-tune, do architectural reviews or help with initial design. TytoEASE's focus is on the Enterprise with how customers integrate open source with their current technologies. Tim On Fri, Jun 19, 2015 at 7:25 AM, Matt Atwater matt.atwa...@tytoease.com wrote: I was hoping to find out who to contact about getting our company listed as one of the commercial support offerings. If we could get added to http://activemq.apache.org/support.html the below is the text we would like. I appreciate everyone help. Matt Atwater - TytoEASE offers support, consulting and training on enterprise ready open source projects including ActiveMQ. Support offerings are at three levels with guaranteed SLA's. Most of our consulting work is short term to help trouble shoot, fine tune, do architectural reviews or help with initial design. Our focus is on the Enterprise with how you integrate open source with your current technologies. Our site is www.tytoease.comhttp://www.tytoease.com, with details on our ActiveMQ offering at http://www.tytoease.com/activemq.
Re: Active MQ 5.11.1 MQTT Message is automatically sent to subscriber when it is connected
I'm at a bit of a loss. Your expectation that a durable subscriber should only get delivered those messages it hasn't already received is correct, so there's probably a bug here. I'd like you to submit a bug in JIRA along with your broker config and client code and a description of the behavior (creating a durable subscriber from the web console, etc.). But before we do that, there's one other thing I want you to check. Please use JMX (JMX != web console, use JConsole or your favorite JMX viewer) to confirm that for your topic only one subscriber exists, that the DequeueCount really is going up (not just the DispatchCount), and that all information about the subscriber is the same from one re-connection to the next. If all of those check out, then there's a bug and you should submit a JIRA entry. BTW, are only the messages from before the first connection being retransmitted, or all messages? That is, if you send messages 1-5 before the first connection, 6-8 while you're connected the first time, and 9-10 while you're disconnected, do you get 1-10 when you reconnect or only 1-5 and 9-10? Knowing that might help someone track down what that root cause is. Tim On Wed, Jun 17, 2015 at 10:37 PM, SummerScent scent.o.sum...@gmail.com wrote: Hi Tim, Here is an example of the dispatched queue, enqueue, dequeue count. http://activemq.2283324.n4.nabble.com/file/n4697942/subscriber.png -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-5-11-1-MQTT-Message-is-automatically-sent-to-subscriber-when-it-is-connected-tp4697874p4697942.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Using MessageListener to read up to a limit of message?
Right, you will continue to receive messages if you process them. The prefetch size is used as an optimization and doesn't have anything to do with message acknowledgement. The broker will always try and keep your prefetch full on your client so you can quickly consume messages. As you process messages the broker will dispatch more to you regardless of whether or not you have called commit or acknowledged the messages. The broker will keep track of all messages until you actually acknowledge them. If you want to pause message acknowledgement and wait before consuming more messages then you should probably use a synchronous consumer instead and just call consumer.receive() and not try and use a messageListener which is asynchronous. On Sun, Jun 21, 2015 at 6:35 PM, Kevin Burton bur...@spinn3r.com wrote: I have a threaded app .. so let’s say I have 100 threads waiting to do work. I want an async message listener to read these messages, UP TO 100 messages, until I can process and commit() them. But I don’t think there’s a way to do this. I had ASSUMED that setting a prefetch of say 10, and a message listener, would only give me 10 messages, until I acknowledged them. But this doesn’t seem to be the case. I keep receiving messages on the listener. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
Re: Do all Message usage have to be within onMessage?
In JMS 2.0 one has shared subscriptions for topics. There was more interest recently around JMS 2.0. I think all the primitives are in place in AMQ, it's just a matter of implementing it. What is a bit of a cause for stress (for me) is the time it takes to run the unit tests. @Christopher, in addition to not being able to roll back you basically take responsibility for the message. If you don't persist, you risk loosing the message if the consumer process goes down. The question is valid though, @Kevin, what exactly are you trying to achieve? Cheers, Hadrian On 06/21/2015 09:14 PM, Christopher Shannon wrote: Usually, if you think you might need to rollback a message you should just want to handle that message in onMessage. What is the reason to share messages between threads? Is it just to be able to consume more than one message at a time? If so I would encourage you to just start multiple consumers instead of trying to pass messages to another thread from one consumer. On of my favorite ways to have multiple consumers and to handle messages concurrently is to use Spring's DefaultMessageListenerContainer. JMS is really designed to handle messages in the same thread they were consumed. A Session and a MessageConsumer are both only single threaded and shouldn't be shared across threads, for example. In the past I have done something where i've had a message listener receive messages and then pass them to other threads in a thread pool to increase throughput but I immediately acknowledged the messages before passing them. This has the disadvantage of not being able to roll back if something happens during processing. If really want to try and pass messages to other threads you might be able to get something to work if you use individual acknowledgement since only a single message will be acknowledged. (I haven't tested this so I could be wrong) Trying to forward a message to another thread while using transactions won't work because when you call commit all messages received since the last commit would be committed. The same for client acknowledge mode. On Sun, Jun 21, 2015 at 6:41 PM, Kevin Burton bur...@spinn3r.com wrote: If you’re using a MessageListener, what’s the best way to use that Message in other threads? I was thinking of reading the message as a string, then forwarding the *string* to other threads, with just a reference to the message. Then stick it back in a queue so that the original thread can commit the message. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
Re: Active MQ 5.11.1 MQTT Message is automatically sent to subscriber when it is connected
Hi Tim, Thanks so much for your consideration. Last time the subscriber automatically receive the message when connected is because I publish the message with RETAINED flag. But I have the trouble with the durable subscription created by Web Console: Some durable subscriptions are working well with offline messages (It keep all offline message as Pending state and when the actual subscriber is connected, all these messages are delivered to it). But some durable subscriptions are not working ( they keep the offline message as Pending, but they does not deliver these message to the actual subscriber when it is connected). I don't know why the ActiveMQ remove all the durable subscription when it is re-started. I mean that: My actual subscriber is already connected to ActiveMQ and subscribe some durable topic with clean Session is FALSE but when ActiveMQ restart, it remove all these durable subscription. That's why I need to create some durable topic subscription when the ActiveMQ is started. Thanks so much! -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-5-11-1-MQTT-Message-is-automatically-sent-to-subscriber-when-it-is-connected-tp4697874p4698036.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
ActiveMQ Apollo Advisory Connection
Hi all, I am investigating ActiveMQ Apollo. In ActiveMQ, I use OpenWire protocol to receive ActiveMQ.Advisory.Connection topic to check the connection status of clients. Can anyone let me know how to do that with ActiveMQ Apollo? Thank many in advance. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-Apollo-Advisory-Connection-tp4698031.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Do all Message usage have to be within onMessage?
Usually, if you think you might need to rollback a message you should just want to handle that message in onMessage. What is the reason to share messages between threads? Is it just to be able to consume more than one message at a time? If so I would encourage you to just start multiple consumers instead of trying to pass messages to another thread from one consumer. On of my favorite ways to have multiple consumers and to handle messages concurrently is to use Spring's DefaultMessageListenerContainer. JMS is really designed to handle messages in the same thread they were consumed. A Session and a MessageConsumer are both only single threaded and shouldn't be shared across threads, for example. In the past I have done something where i've had a message listener receive messages and then pass them to other threads in a thread pool to increase throughput but I immediately acknowledged the messages before passing them. This has the disadvantage of not being able to roll back if something happens during processing. If really want to try and pass messages to other threads you might be able to get something to work if you use individual acknowledgement since only a single message will be acknowledged. (I haven't tested this so I could be wrong) Trying to forward a message to another thread while using transactions won't work because when you call commit all messages received since the last commit would be committed. The same for client acknowledge mode. On Sun, Jun 21, 2015 at 6:41 PM, Kevin Burton bur...@spinn3r.com wrote: If you’re using a MessageListener, what’s the best way to use that Message in other threads? I was thinking of reading the message as a string, then forwarding the *string* to other threads, with just a reference to the message. Then stick it back in a queue so that the original thread can commit the message. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
Re: Do all Message usage have to be within onMessage?
Hadrian, Right, thanks for clarifying that point. I realized I wasn't super clear but that's what I was trying to say, that the client takes responsibility of the message from the broker. The only time I've done that in an application was rare cases where losing a message was an acceptable trade off to increase performance. Chris On Sun, Jun 21, 2015 at 11:27 PM, Hadrian Zbarcea hzbar...@gmail.com wrote: In JMS 2.0 one has shared subscriptions for topics. There was more interest recently around JMS 2.0. I think all the primitives are in place in AMQ, it's just a matter of implementing it. What is a bit of a cause for stress (for me) is the time it takes to run the unit tests. @Christopher, in addition to not being able to roll back you basically take responsibility for the message. If you don't persist, you risk loosing the message if the consumer process goes down. The question is valid though, @Kevin, what exactly are you trying to achieve? Cheers, Hadrian On 06/21/2015 09:14 PM, Christopher Shannon wrote: Usually, if you think you might need to rollback a message you should just want to handle that message in onMessage. What is the reason to share messages between threads? Is it just to be able to consume more than one message at a time? If so I would encourage you to just start multiple consumers instead of trying to pass messages to another thread from one consumer. On of my favorite ways to have multiple consumers and to handle messages concurrently is to use Spring's DefaultMessageListenerContainer. JMS is really designed to handle messages in the same thread they were consumed. A Session and a MessageConsumer are both only single threaded and shouldn't be shared across threads, for example. In the past I have done something where i've had a message listener receive messages and then pass them to other threads in a thread pool to increase throughput but I immediately acknowledged the messages before passing them. This has the disadvantage of not being able to roll back if something happens during processing. If really want to try and pass messages to other threads you might be able to get something to work if you use individual acknowledgement since only a single message will be acknowledged. (I haven't tested this so I could be wrong) Trying to forward a message to another thread while using transactions won't work because when you call commit all messages received since the last commit would be committed. The same for client acknowledge mode. On Sun, Jun 21, 2015 at 6:41 PM, Kevin Burton bur...@spinn3r.com wrote: If you’re using a MessageListener, what’s the best way to use that Message in other threads? I was thinking of reading the message as a string, then forwarding the *string* to other threads, with just a reference to the message. Then stick it back in a queue so that the original thread can commit the message. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
Do all Message usage have to be within onMessage?
If you’re using a MessageListener, what’s the best way to use that Message in other threads? I was thinking of reading the message as a string, then forwarding the *string* to other threads, with just a reference to the message. Then stick it back in a queue so that the original thread can commit the message. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
Using MessageListener to read up to a limit of message?
I have a threaded app .. so let’s say I have 100 threads waiting to do work. I want an async message listener to read these messages, UP TO 100 messages, until I can process and commit() them. But I don’t think there’s a way to do this. I had ASSUMED that setting a prefetch of say 10, and a message listener, would only give me 10 messages, until I acknowledged them. But this doesn’t seem to be the case. I keep receiving messages on the listener. -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts
