As a follow-on... The problem appears to be between the Java producer and the initial federated broker. The scenario is that after 8 mins, the connection times out on the broker, and the producer does failover, and resends a message that was sent previously, and received by a consumer. The log files show that the broker is attempting to send heartbeat messages to the producer, but there is no response. Specifically, the broker records a SENT event, but there is no corresponding RECV. Conversely, the heartbeat to the other federated broker does have the SENT/RECV pattern. After the 4th failed heartbeat, the connection is timed out and the failover happens. So this leads me to think that my producer is (a) either set up incorrectly in terms of heartbeats, or (2) the messages are not being ack'ed properly, and are hence resent upon failover. The producer is being started with -Damqj.heartbeat.delay=30, but the log shows a heartbeat only every two minutes (i.e. the broker default). For the ack, the producer is setting up the session to be auto-ack (and non-transactional): _session = _connection.createSession(false, Session.AUTO_ACKNOWLEDGE); and the message is sent as _producer.send(message, DeliveryMode.PERSISTENT, Message.DEFAULT_PRIORITY, Message.DEFAULT_TIME_TO_LIVE); The producer is essentially just the example producer as per the qpid documentation, with adjustments for hosts and queue names, where connection params are: connectionfactory.broker=amqp://guest:guest@/QPID?brokerlist='tcp://localhost:5672?retries='10'&connectdelay='10'' destination.mydest=direct://amq.direct/myqueue?durable='true'&exclusive='false'&autodelete='false'&routingkey='myroute' So, can anyone provide some insight into these more specific questions: (1) what is the proper configuration to set up a Java producer to heartbeat properly with a target broker? (1a) Or, to put it another way, what can cause a Java producer to not answer a heartbeat message from its broker? (3) What could cause a messages sent in a session set up as "auto-ack" to not be ack'd? (4) This is an additional bit of strange behavior. As you can see from the broker connect string above, it specifies a connectdelay. However, on the producer's failover, the log shows: FailoverSingleServer:114 - No delay between connect retries, use tcp://host:port?connectdelay='value' to enable. It seems that the connectdelay value has not been picked up. Can anyone conjecture why? Thanks for any help! -Simon
--- On Sun, 2/28/10, me really <[email protected]> wrote: From: me really <[email protected]> Subject: duplicate messages To: [email protected] Date: Sunday, February 28, 2010, 9:37 PM Hello, We are running a two-broker federation. We have a Java producer (0.5) sending messages to Broker 1, which is configured to push to Broker 2, from which a consumer is reading the data. The push link is configured with: 'qpid-route -d -s queue add broker2 broker1 myexchange myqueue' We are experiencing duplicate messages. Here's what happens. The producer sends a steady stream of messages, but we cansee that occasionally, the consumer receives duplicate messages. What we see in the logs is that the duplicate messages appear to be transmitted when there is a failover recovery. But the duplicated messages are duplicates of messagesthat were sent on the order of an hour ago. We are running a version of the C++ broker that has the fix mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=510301(inactive federation links timeout occasionally) -- we are seeing the timeouts at the 8 minute intervals of this bug (we're running 0.5.752581-34, specifically), though not continuously, only when idle enough time. Can anyone offer any insight into this rather strange behavior? Thanks,Simon
