Re: Artemis 1.5.1 : org.apache.activemq.artemis.core.server] AMQ222029: Could not locate page transaction messages under some load
Strangely, the situation is a bit complicated: In a prior test I received messages that the broker could not start a thread due to not having enough heap available. It would not even shutdown, because it could not start the shutdown thread. Eventually the broker got killed, which then was taken over by the backup. After this test, I manually removed all queues, but did not delete the journals. Also, I changed artemis.profile to use 4096M heap (up from 1024). All producers and consumers were stopped prior to the removal of all the queues. The producers and consumers were started and the test repeated... The test setup is like this: - We use servicemix to provide the webservice(s), like this blueprint: http://www.osgi.org/xmlns/blueprint/v1.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xmlns:camel="http://camel.apache.org/schema/blueprint; xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf; xsi:schemaLocation=" http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://camel.apache.org/schema/blueprint/cxf http://camel.apache.org/schema/blueprint/cxf/camel-cxf.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd;> http://camel.apache.org/schema/blueprint; id="Webservices" xmlns:ns="http://my.test/webservices/perform_load_test_request/1;> http://0.0.0.0:8383/webservices/perform_load_test_01; /> the broker connection is setup like this: http://www.osgi.org/xmlns/blueprint/v1.0.0;> This producer seems to do the job fine. The consumers/producers are running in a separate java process that uses code I have only limited influence upon, but I reckon each process has it's own (1) session with "many" (in the case of this process 40) threads. I see this message (Could not locate page transaction) only on the 1st queue, but do not seem to loose any messages. (That's good, reliability takes precedence over performance, although performance is quite important as well) Could the message also happen due to the default prefetch of 1000 messages? Is there actually any harm? Should I increase the global-max-size directive in broker.xml? And is 400MB (I reckon the value is in KB, so the default is 100MB) the safe choice with a 4GB heap? -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-1-5-1-org-apache-activemq-artemis-core-server-AMQ222029-Could-not-locate-page-transaction-med-tp4721587p4721634.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Artemis 1.5.1 : org.apache.activemq.artemis.core.server] AMQ222029: Could not locate page transaction messages under some load
Currently I'm able to start some load testing using Artemis; The load is nowhere near what I expect to need, but that's currently not the issue. Of course I test both Artemis performance and the producers / consumers. The performance itself is not at stake though, there's plenty of room for improvements anyway. Using JMeter I sent a total of 50.000 messages using 50 threads to my servicemix which puts the messages on the Artemis broker. Next a bunch of processes get a message, does it's work and puts it on the next queue (/endpoint) After some time, I noticed this: message repeatedly in the log: WARN [org.apache.activemq.artemis.core.server] AMQ222029: Could not locate page transaction 32,212,560,158, ignoring message on position PagePositionImpl [pageNr=12, messageNr=1851, recordID=-1] on address= What does this message mean exactly? A google on AMQ029 does not yield any usable results. Am I losing messages? What can I do to fix / avoid this, whatever it is I'm seeing? Please shed some light on what is happening. -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-1-5-1-org-apache-activemq-artemis-core-server-AMQ222029-Could-not-locate-page-transaction-med-tp4721587.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: High availability - multiple publishers scenario
Normally it is hard to see if two messages that are sent separately from each other are really the same. To implement this on the broker level might actually be a bit hard (imo) However, there are a few ways you can work around this: 1) from the producer end: make sure that the producers do not produce the same data. For example such a thing can be prevented by placing the trigger on which the producers have to produce their messages on a queue; only one producer will consume this trigger and will produce the messages. 2) on the receiver / consumer end: build in logic to filter out doubles, or -even better- make sure the receiving end is idempotent; meaning that receiving the same data does not yield any problems. Both are viable, and might even for all kinds of reasons be combined. Not sure if this answers your question though. -- View this message in context: http://activemq.2283324.n4.nabble.com/High-availability-multiple-publishers-scenario-tp4720767p4720787.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: [Artemis] WebConsole (Hawt.io artemis plugin)
good to see this one; Jolokia is offered out of the box and should work. I did alter the "localhost" to "0.0.0.0" to make it available remotely, but care should be taken, as this opens up the server for others. Also, I noticed that the Artemis documentation regarding Jolokia is not 100% accurate anymore. I expect some beans have been renamed or otherwise altered. For Hawt.io and the artemis plugin: I am not aware of any licensing, so can't answer on that. I did notice it's not 100% compatible with the current version of Artemis (first message as JSON returns an error, can't browse queues, ...) but it does give the info I use in my tests: which queues are deployed and how many messages they contain and how many were put and removed. -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-WebConsole-Hawt-io-artemis-plugin-tp4720702p4720728.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: LevelDB vs KahaDB for master/slave Zookeeper setup
Another thing: If you need a master/slave configuration... You may want to check out Apache Artemis, which does "store replication" -- View this message in context: http://activemq.2283324.n4.nabble.com/LevelDB-vs-KahaDB-for-master-slave-Zookeeper-setup-tp4720633p4720699.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: LevelDB vs KahaDB for master/slave Zookeeper setup
(Replicated)LevelDB has some issues, and is indeed deprecated. KahaDB does not officially have a replicating version. There has been some work on it, which was abandoned in favor of ReplicatedLevelDB... If and when this will be revived is unknown. Then to the statement: I need to implement a master/slave AMQ configuration using Zookeeper. In my opinion, you may state that you need to setup a master/slave AMQ configuration. However, "using zookeeper" cannot be a requirement. While Zookeeper is used in the ReplicatedLevelDB configuration, and is therefore a requirement for an ActiveMQ configuration using ReplicatedLevelDB, does not make it a requirement if this dependency falls away. If you need a Master/Slave configuration of ActiveMQ, currently the only supported way is using a shared store. This can be a JDBC connection to a (highly available) database, or a KahaDB on a shared store. On shared stores tehre are some good and some bad options; that's something to determine. In my opinion, if you're able to, I'd suggest GlusterFS (that is, if you're running on Linux) One thing to keep in mind is that you need the filesystem able to lock a file. NFS allows this, samba does not (afaik)... so not every type of storage sharing will work... you'll have to dig a bit into that. -- View this message in context: http://activemq.2283324.n4.nabble.com/LevelDB-vs-KahaDB-for-master-slave-Zookeeper-setup-tp4720633p4720698.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Couple of questions regarding Apache Artemis
pretty much what I would do if there wasn't a script that did just that (a script is in this case just a tool to make sure you don't forget a step and do the right thing in the right order... and make sure all is done safely) -- View this message in context: http://activemq.2283324.n4.nabble.com/Couple-of-questions-regarding-Apache-Artemis-tp4720241p4720328.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Persist messages in MySQL as text instead of blob
Why would you want this? What should happen to various types of encoding? Is everything UTF-8? Or ISO-8859-15 ...? Is your database configured according to what _all_ messages will have as their codepage? Apart from having a database being discouraged, changing this datatype is far more dangerous. I would highly discourage you to walk this path. -- View this message in context: http://activemq.2283324.n4.nabble.com/Persist-messages-in-MySQL-as-text-instead-of-blob-tp4720250p4720261.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Couple of questions regarding Apache Artemis
Since we're advancing in our tests and Apache Artemis seems to suit our needs most, a couple of questions I have not yet been able to find in the Artemis User Manual (which has been a great resource, so far; very well written indeed.) Running: - We run Artemis in a master/slave configuration, with each node in a serparate datacenter. We have two datacenters, with two physically distinguished network paths; Is it possible to setup replication so, that both paths will be used for replication, so that if one path fails, the broker will avoid a split brain situation (as it sees the live broker over the other network interface and use that one instead for replication... or use the replication lan for replication (and the primary lan only in case the replication lan fails) I'm thinking such a setup would be possible by using two connectors to the same broker in the cluster configuration. - When I SIGKILL a broker (for testing, no worries) I see it takes the slave about two minutes to become live. Even if I set the following two options in the cluster configuration 2000, 1000; Is there a way to influence this behavior? If not, is there a reason I should not want that? Future: - Given that Artemis will continue releases... What is the upgrade path from one version to the other? (provided that the upgrade path is unsafe?) I assume this will be along the following lines + (place binaries and stop all brokers in the cluster) + artemis data exp (this goes to stdout?) + artemis create ... + artemis data imp --input + (start the brokers in the cluster) -- View this message in context: http://activemq.2283324.n4.nabble.com/Couple-of-questions-regarding-Apache-Artemis-tp4720241.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Artemis 1.5.0 fails to discover libAIO on Linux, falls back to NIO
Quick reply: 1.5.0 source was easier to get atm (behind some proxies, so it's hard to get to git), so followed the instructions: they're clear, correct and worked like a charm. -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-1-5-0-fails-to-discover-libAIO-on-Linux-falls-back-to-NIO-tp4720184p4720196.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Artemis 1.5.0 fails to discover libAIO on Linux, falls back to NIO
Thanks; I'll try to get the new version and try to compile the libaio and see if that fixes this issue. -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-1-5-0-fails-to-discover-libAIO-on-Linux-falls-back-to-NIO-tp4720184p4720195.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Artemis 1.5.0 fails to discover libAIO on Linux, falls back to NIO
I have a RHEL 6.7 server wich has LibAIO installed, using Java 1.8.0_102 Artemis 1.4.0 finds and uses it with no problem, as this log line shows: 17:18:30,060 INFO [org.apache.activemq.artemis.core.server] AMQ221012: Using AIO Journal When I create a broker with the same parameters on the same machine, but instead using Artemis-1.5.0 it fills in NIO in the broker.xml; When I change that by hand to ASYNCIO I get the following in the log: 15:46:04,805 WARN [org.apache.activemq.artemis.core.server] AMQ222018: AIO was not located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal (I realize that having NIO in the broker.xml itself is a sign that the Linux AIO library could not be found) Yum shows that libaio.x86_64-0.3.107-10.el6 is installed. is this a bug, or some other mismatch I made? -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-1-5-0-fails-to-discover-libAIO-on-Linux-falls-back-to-NIO-tp4720184.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: OutOfMemoryError in Activemq
Just a question about this line: A singleton class that used to publish huge non-persistent messages to one topic with one connection. However, I got the error as follows: How many of these huge messages have you sent, when did it break and how big were those messages? This data might help to determine the problem and possible solutions (e.g. increase the heap size) -- View this message in context: http://activemq.2283324.n4.nabble.com/OutOfMemoryError-in-Activemq-tp4719068p4719117.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ leaks file descriptors
Could it be a (or more) messages staying on a queue, causing the logs not to be cleared? -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-leaks-file-descriptors-tp4718793p4718954.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Activemq EOFException
In fact, this always happens when a connection stops... If you have a network of brokers and you stop one, you'll see this exception in the other brokers. If your client stops, same... Maybe this exception deserves to be handled like "Connection has disconnected." but apart from that... it's a warning, and not severe afaics. -- View this message in context: http://activemq.2283324.n4.nabble.com/Activemq-EOFException-tp4681252p4718765.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
thanks for providing a good idea of what to expect; yet I think it's good to be aware of this. I did notice upon searching that there has been done some work to get Kahadb replicating; is that completely abandoned or is it still something that might get implemented (in a somewhat near future) I think it's a great idea to have the database replicating, to reduce the added complexity of shared storage (which may itself fail, so needs to have it's own fault tolerance) I'll explore what other options are available and work well. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4717517.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
Created a JIRA https://issues.apache.org/jira/browse/AMQ-6453 about this issue; please let me know if you need more information. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4717515.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
ActiveMQ persistency backends, good and best practices
Recently we started looking at ActiveMQ as a replacement of the JMS layer of our backend, and thus I have been doing some test setups, using ActiveMQ 5.14.0 The requirements I'm looking at are: - reliability: highly available, no corruption - able to handle peak loads of approx. 250 msgs per second (incoming) (other performance is less of an issue) - able to handle messages that may stay on the queues for a longer period (e.g. a few months) - able to guarantee order of delivery (fifo) Given above constraints, I made the following setups: - replicatedLevelDB: As may be seen in another post, I found this backend too easily getting corrupted under stress. - KahaDB in a network of brokers, but that does not guarantee order of delvery (fifo) - JDBC to a PostgreSQL database I have not done a single KahaDB on a shared storage; This is because in our experience the shared storage adds unreliability; besides, I think KahaDB is less of an option due to the messages that remain on a queue for an extended period. Given the load, I expect the logs to grow very large (a few hundred gigabytes in my estimation) and this makes it less of an option. The JDBC backend seems to perform. I did some tests, sending 100.000 messages and they were all persisted nicely; However, PostgreSQL does not offer HA out of the box; There is replication, and thus a master/slave, but no automatic promotion of a slave to master, and the slave is (of course) read-only. To tackle this, I have looked at PostgreSQL-BDR; It appears to come close, but having a third slave causes a lock race when the master broker goes away, a race that is won and lost in turns by each of the other brokers. Of course, this should be mitigated having only 2 brokers, of which one is active. I was wondering what the experiences of others are; what is good practice and what to avoid, common mistakes and "easy" wins (that are not that obvious); Good experiences and bad ones... -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-persistency-backends-good-and-best-practices-tp4717473.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: activem-admin command is missing in 5.13.0 for linux
Browsing jira I noticed the removal of this component. Didn't dive deeper, but it's no longer there. Probably the same things can be done, but in a different way. What would you like to do? What would you need the activemq-admin command for? -- View this message in context: http://activemq.2283324.n4.nabble.com/activem-admin-command-is-missing-in-5-13-0-for-linux-tp4716870p4716914.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
Gave the system a weekend time to see if it could recover automatically... alas that's not the case. However, when I stopped all brokers, changed one of the brokers with 'replicas="1"' (rather than 3); started it (kind of single mode) and started the other brokers one by one so they could catch up and then stopped them; restarted the master broker but now the replicas back to 3 and the backup brokers got started as well... Now the brokers are back online in a consistent state. There are some issues though... As the broker that is now primary is one that apparently failed earlier there are now messages on the queues that should've been consumed (I think); So, is it possible that the brokers try to heal themselves? (with warnings and errors in the logs, so the administrator knows to validate the situation; remaining messages that are uncertain in the 'exactly once' situation should be moved to the DLQ Going through the logs I noticed a few things; for simplicty I will call the HA brokers 'broker1, broker2 and broker3' although they're all 'broker1' :-) I see that broker1 was online and accepting messages when the test started; However, it missed some zookeeper pings and got demoted to slave. broker2 became the new master. After the test, all brokers got restarted; Raising the issue, described in my previous post. This morning I "recovered" broker1 (I now realize I should've done this on the last known master: broker2; alas...) there are now many messages that I _think_ (they're a bit too many and the test too generic to determine this easily) have been processed already. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4716912.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
Concluded the test now; No good news though.
After sending the 100.000 messages (which got consumed by services that in
turn also produced new messages and so had a flow from one queue to another
and another etc... resulting in about 1.500.000 messages in the few hours
that this test ran) I restarted the master broker. Catching up went fine, no
errors
Shortly thereafter I restarted the new master (the slaves had caught up, I
had made sure of that message); Now the new master is giving multiple times
the message: "2016-09-23 19:10:37,644 | WARN | Invalid log position: 0 |
org.apache.activemq.leveldb.LevelDBClient | Thread-118"
While both slaves are spewing out many log messages along the lines of this:
"2016-09-23 19:15:33,892 | WARN | No reader available for position: 0,
log_infos:
{209718314=LogInfo(/data/activemq/broker1-db/0c800c2a.log,209718314,104859264),
2831214880=LogInfo(/data/activemq/broker1-db/a8c0e920.log,2831214880,104859800),
3250651975=LogInfo(/data/activemq/broker1-db/c1c10347.log,3250651975,104862133),
3355514108=LogInfo(/data/activemq/broker1-db/c80114fc.log,3355514108,104859203),
4823542070=LogInfo(/data/activemq/broker1-db/00011f816936.log,4823542070,0)}
| org.apache.activemq.leveldb.RecordLog | Thread-3
"
Question arises if LevelDB is a good choice or is it too easily corrupted?
--
View this message in context:
http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4716869.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
possibly of note is that I had the sync option on the default (quorum_mem). I have moved the data directories for each broker as an ".org" (for safekeeping) and will try and see if quorum_disk is the way to prevent this situation from happening. It will take a little while though to get definite results. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4716850.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: ActiveMQ ReplicatedLevelDB corruption
Yes, the three brokers are on separate (virtual) machines that write to their own disk. It's hard to determine (at this point) whether the original master got corrupted, as it (at least) got corrupted after starting said broker again (or was already corrupted to begin with) -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831p4716842.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
ActiveMQ ReplicatedLevelDB corruption
Recently I installed Apache ActiveMQ in a few different ways. One of those is
using ReplicatedLevelDB for a master/slave/slave setup.
Yesterday I did a bit of loadtesting: sending 100.000 messages with 100
threads producing the messages (used jmeter for that) (so each thread
produced 1000 messages); I had another process moving the messages from one
broker to another and back again (the queues had the same names across each
broker, so that was easy moving) and then went about processing the messages
which caused the messages to flow across various queues.
All seemed fine, everything looked okay... until I stopped the active
broker. (this is hours after the last message was consumed and procsesed):
Then I notice a few bouncing brokers, one comes up but crashes on an
EOFException; a bit later the other broker does the same.
In the log I see many messages like this:
[quote]
2016-09-23 13:38:52,950 | WARN | No reader available for position: 0,
log_infos:
{11534500540=LogInfo(/data/activemq/broker1-db/0002af8282bc.log,11534500540,104858130),
12163654223=LogInfo(/data/activemq/broker1-db/0002d502a24f.log,12163654223,104858162),
12897666570=LogInfo(/data/activemq/broker1-db/000300c2c60a.log,12897666570,104859912),
13002526482=LogInfo(/data/activemq/broker1-db/00030702cf12.log,13002526482,104859038),
18455209795=LogInfo(/data/activemq/broker1-db/00044c042743.log,18455209795,104859837),
22020442500=LogInfo(/data/activemq/broker1-db/000520854984.log,22020442500,104859288),
23173898306=LogInfo(/data/activemq/broker1-db/00056545a042.log,23173898306,104860684),
24641928389=LogInfo(/data/activemq/broker1-db/0005bcc5fcc5.log,24641928389,0)}
| org.apache.activemq.leveldb.RecordLog | Thread-1039
[/quote]
Then I see messages like this:
[quote]
2016-09-23 13:38:46,324 | WARN | Invalid log position: 11409726550 |
org.apache.activemq.leveldb.LevelDBClient | ActiveMQ BrokerService[broker1]
Task-3
[/quote]
After that, the broker starts and logs a few messages like this:
[quote]
2016-09-23 13:40:49,041 | WARN | Invalid log position: 0 |
org.apache.activemq.leveldb.LevelDBClient | Thread-1040
[/quote]
and then we get exception:
[quote]
2016-09-23 13:41:09,748 | INFO | Stopping BrokerService[broker1] due to
exception, java.io.EOFException: File
'/data/activemq/broker1-db/00030702cf12.log' offset: 110647192 |
org.apache.activemq.util.DefaultIOExceptionHandler | LevelDB IOException
handler.
java.io.EOFException: File '/data/activemq/broker1-db/00030702cf12.log'
offset: 110647192
[/quote]
This is a rince and repeat situation; both living brokers are now
alternating this sequence.
It seems like the load I generated caused corruption on the database; but
this should not be possible... What information can I provide to see how
this situation can be avoided?
--
View this message in context:
http://activemq.2283324.n4.nabble.com/ActiveMQ-ReplicatedLevelDB-corruption-tp4716831.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: What is the proper way to configure multiple brokers on the same machine in regard to jmx
I guess that is what I will do indeed; In normal operations time I'll hardly use jconsole itself; but will have some centralized tools or just simply use jolokia; Thanks for the advise given! -- View this message in context: http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589p4715652.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: What is the proper way to configure multiple brokers on the same machine in regard to jmx
It was a human (mine) error; I modified it... apologies if I had you thinking it was a typo in the default. I had been fiddling around... and have been even more: I now realized a few things; The bind error I had earlier, which caused me to think the problem was the port, actually was not the port, but the path "/jmxrmi"; When I changed that on the second broker it could bind. But then the other (logical) problem arose when I restarted the first broker... then the jmx for broker2 could no longer be found... besides, both brokers would never end up in the same jmx connection (e.g. jconsole); The thing I was trying to accomplish was getting one jmx mbean server to "rule" all brokers (even remote ones) so I would have a single point of management (restarting, memory usage, cpu utilization, etc) I'm getting to the conclusion this will not be possible... although it would really be cool to have such a thing :-) -- View this message in context: http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589p4715636.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: What is the proper way to configure multiple brokers on the same machine in regard to jmx
Tim, Your comment got me thinking why port 1099 was getting used rather than 11099 (and 12099); so I checked my java process and it's parameters: (ps -ef | grep java); I noticed the parameter telling the port was not in it. There's a stupid typo in the "env" files: ACTIVEMQ_SUNJMX_START="$ACTIEVMQ_SUNJMX_START -Dcom.sun.management.jmx.authenticate=false" So, fixed that; And now the broker won't start: Error: Password file not found: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/jre/lib/management/jmxremote.password This one is a bit strange; I think I said to make an "unsafe, unauthenticate mbean server"; but at least that problem is out of the way now. Now to get both brokers connecting to the same MBean Server (or start one, if none is available)... to be continued. -- View this message in context: http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589p4715625.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: What is the proper way to configure multiple brokers on the same machine in regard to jmx
For no obvious reason, actually... At first I did not have these parameters, neither on broker1 nor on broker2; however, upon starting broker1 first, I noticed this line: JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi | org.apache.activemq.broker.jmx.ManagementContext | JMX connector So, I assumed (I know, assumption is the mother of all ehrm... ) there would be one "overall" process that would kind of redirect connections to the "real" process, acting like a kind of proxy so to say. As you can see, broker1 has the same configuration, with port 1099 whereas port 11099 is the port in the environment file; I'll remove both parameters in the activemq.xml So, broker1 now has this managementContext: and broker2 now has this managementContext: Broker1 reports: JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi And broker2: Failed to start JMX connector Cannot bind to URL [rmi://localhost:1099/jmxrmi]: javax.naming.NameAlreadyBoundException: jmxrmi [Root exception is java.rmi.AlreadyBoundException: jmxrmi]. Will restart management to re-create JMX connector, trying to remedy this issue. And, naturally it doesn't connect to the mbean server. -- View this message in context: http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589p4715623.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Replicated LevelDB uses Apache ZooKeeper problem
I think your ZooKeeper elects a master, look at the slave log: 2016-03-08 02:08:05,307 | INFO | Promoted to master | org.apache.activemq.leveldb.replicated.MasterElector | main-EventThread However, something goes wrong, and I think it is the hostname in your persistenceAdapter: hostname="192.168.5.200" If all servers have this hostname, the slaves cannot connect, because that master is down... Each server needs its own hostname in the replicatedLevelDB configuration. At least, that is how I have it and it works like a charm. -- View this message in context: http://activemq.2283324.n4.nabble.com/Replicated-LevelDB-uses-Apache-ZooKeeper-problem-tp4709020p4715593.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: What is the proper way to configure multiple brokers on the same machine in regard to jmx
I should note the following few things: - Java version is 1.7.0 (openJDK) - ActiveMQ is version 5.14.0 -- View this message in context: http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589p4715590.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
What is the proper way to configure multiple brokers on the same machine in regard to jmx
For various purposes, I wish to define various brokers on the same machine.
For example: some queues require ordering of data, and thus use the
replicatedLevelDb as persistence adapter, others don't even need persistence
or ordering and can use only memory in order to provide faster throughput,
etc...
Other than described in the page about jmx, just using the "default"
useJmx="true" parameter in the broker definition, does not change a thing.
("default" is quoted, as according to the docs this parameter is always in
effect)
So, I tried fiddling around, and got thus far:
In ACTIVEMQ_HOME/bin/env I had to add / uncomment these lines (for now I
won't use authentication; might be added in a later stage, but first want to
see it "just work")
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.port=11099 "
ACTIVEMQ_SUNJMX_START="$ACTIEVMQ_SUNJMX_START
-Dcom.sun.management.jmx.authenticate=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.ssl=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote"
in the broker definition (activemq.xml) I had to change this part:
This seems to work for the first broker. When starting the second broker,
which has a few different options:
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.port=12099 "
ACTIVEMQ_SUNJMX_START="$ACTIEVMQ_SUNJMX_START
-Dcom.sun.management.jmx.authenticate=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.ssl=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote"
and a managementContext like this:
I get the exception that a server is already active (which is true, it's
started by the frist broker)
Failed to start JMX connector Cannot bind to URL
[rmi://localhost:1099/jmxrmi]: javax.naming.NameAlreadyBoundException:
jmxrmi [Root exception is java.rmi.AlreadyBoundException: jmxrmi]. Will
restart man
agement to re-create JMX connector, trying to remedy this issue. |
org.apache.activemq.broker.jmx.ManagementContext | JMX connector
I had hoped that the second broker would try to find an active mbean service
and register at it, or otherwise starts one. (Same for the first broker);
Is this possible (or are my wishes actually desirable) at all, or not? And
how to get this configured properly?
--
View this message in context:
http://activemq.2283324.n4.nabble.com/What-is-the-proper-way-to-configure-multiple-brokers-on-the-same-machine-in-regard-to-jmx-tp4715589.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
