for 1) don't use journaled jdbc in a failover setup because the journal is not replicated. Use simple jdbc or revert to a shared file system setup.
2) producer order cannot be maintained across brokers. If it is vital you need a single broker or partition your data across destinations. 2009/11/10 denis_the_user <denis.ehr...@b-s-s.de>: > > Hey. > > I'm looking for a high-performance solution for message throughput of about > 10.000 messages per second. > I think thats no problem at all. > > My System: > - Some multi-core systems > - ActiveMQ 5.3 > - Apache tomcat + Java Webapp handling Producers > - Java tool handling Consumers and delivering to the target system. > - Messages of typ Add/Update/Delete > > But I have some limiting facts to take care about: > > 1) I need to make sure that no message is lost. > For that case i did some testing with redundant brokers on different hosts. > If one dies the other takes over, no prblem, but still loosing messages. I'm > using failover-configuration within Java-API for producer and consumer. API > works fine. But I think the activeMQ server uses transaction to persist data > to the MYSQL-Cluster and if I kill one Broker (using kill -9 <pid>) the > transaction is not done and the messages are lost. Using just kill <pid> no > message is lost, but some times there is one more than I sent. > > I hope that behaviour is just a configuration fault I did. > > I'm using just two little configuration for the factory I used to create > some Producer: > ActiveMQConnectionFactory connectionFactory = new > ActiveMQConnectionFactory(user, password, url); > connectionFactory.setAlwaysSessionAsync(false); > connectionFactory.setAlwaysSyncSend(true); > > My Session uses: > Session.AUTO_ACKNOWLEDGE > > Per message I run the following code: > this.session.getProducer().send(message); > if (transacted) this.session.getSession().commit(); > this.session.close(); > > I know that I create one new producer per Message. Sure could change that. > But why are messages lost on broker fault? > > 2) The second fact I need to care for: The order of the Messages is > importened. Reason: One Message tells the system to update, the other > message to delete. If the entry is deleted before updating system throws > error. > Is there any possibility for multiple brokers to take care of producing > order? UseCase: One Server slows down and the other one deliveres normal. > Messages got mixed up and Consuming system runs into error. > I'm sure I could just use one broker and ensure the order, but that's > perhaps to slow. > > For more information my broker configuration: > <beans xmlns="http://www.springframework.org/schema/beans"; > xmlns:amq="http://activemq.apache.org/schema/core"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans-2.0.xsd > http://activemq.apache.org/schema/core > http://activemq.apache.org/schema/core/activemq-core.xsd";> > > <bean > class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/> > > <broker xmlns="http://activemq.apache.org/schema/core"; > brokerName="localhost" useJmx="true"> > > <persistenceAdapter> > <journaledJDBC dataDirectory="${activemq.base}/data" > dataSource="#mysql-ds"/> > </persistenceAdapter> > > <transportConnectors> > <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/> > </transportConnectors> > > <destinationPolicy> > <policyMap> > <policyEntries> > <policyEntry queue=">" producerFlowControl="true" > memoryLimit="20mb"> > <deadLetterStrategy> > <individualDeadLetterStrategy queuePrefix="DLQ." > useQueueForQueueMessages="true" /> > </deadLetterStrategy> > </policyEntry> > <policyEntry topic=">" producerFlowControl="true" > memoryLimit="20mb"> > </policyEntry> > </policyEntries> > </policyMap> > </destinationPolicy> > > <managementContext> > <managementContext createConnector="true"/> > </managementContext> > > <systemUsage> > <systemUsage sendFailIfNoSpace="true"> > <memoryUsage> > <memoryUsage limit="1024 mb"/> > </memoryUsage> > <storeUsage> > <storeUsage limit="2 gb" name="foo"/> > </storeUsage> > <tempUsage> > <tempUsage limit="1000 mb"/> > </tempUsage> > </systemUsage> > </systemUsage> > > </broker> > > <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource" > destroy-method="close"> > <property name="driverClassName" > value="com.mysql.jdbc.Driver"/> > <property name="url" > value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/> > <property name="username" value="activemq"/> > <property name="password" value="activepwd"/> > <property name="maxActive" value="200"/> > <property name="poolPreparedStatements" value="true"/> > </bean> > > <jetty xmlns="http://mortbay.com/schemas/jetty/1.0";> > <connectors> > <nioConnector port="8161"/> > </connectors> > > <handlers> > <webAppContext contextPath="/admin" > resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/> > <webAppContext contextPath="/fileserver" > resourceBase="${activemq.base}/webapps/fileserver" logUrlOnStart="true"/> > </handlers> > </jetty> > > </beans> > > I hope some one can help me. Or spend me some good advice for cluster design > and a configuration example. > > -- > View this message in context: > http://old.nabble.com/General-Design-Help-tp26284819p26284819.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > -- http://blog.garytully.com Open Source Integration http://fusesource.com
