On 09/23/2010 09:23 AM, Joe Nardone wrote:
I'm at a loss as to what's going on. This is repeatable, regardless of
queue size or contents. It consistently slows down at the same point.
How do I avoid this delay in these latter frames for large payloads?
Is there some kind of limiter that I'm not aware of? Is there a magic
number for message size?
Not that I am aware of, that is a curious result indeed. Do non-persistent
messages show the same thing (wondering if its the persistence of the very
large messages that is hitting some journal limit)?
Persistence makes no difference. I restarted the broker without
persistence, and disable durability for both the queue and message
creation.
Here's a complete program that shows the defect. It takes a file as a
parameter -- if that file is about 5MB or less, it will send the
message in under a second. If it's greater than about 5.1MB, it will
add 3 seconds for every additional 64k frame that is required to send.
#include<qpid/client/Connection.h>
#include<qpid/client/Session.h>
#include<qpid/client/AsyncSession.h>
#include<qpid/client/Message.h>
#include<string>
#include<iostream>
#include<vector>
#include<iterator>
#include<fstream>
#include<algorithm>
using namespace qpid::client;
using namespace qpid::framing;
int main(int argc, char *argv[])
{
Connection c;
c.open("localhost", 5672);
Session s = c.newSession();
s.queueDeclare(arg::queue = "logs", arg::durable=false);
s.exchangeBind(arg::exchange="amq.topic", arg::queue="logs",
arg::bindingKey="logs.#");
// read the file in
typedef std::istream_iterator<char> istream_iterator;
std::ifstream file(argv[1]);
std::vector<char> buffer;
file>> std::noskipws;
std::copy(istream_iterator(file), istream_iterator(),
std::back_inserter(buffer));
string mydata(buffer.begin(), buffer.end() );
// send the message
Message message;
message.getDeliveryProperties().setRoutingKey("logs.1001.10000");
message.setData(mydata);
// DELAY HAPPENS HERE. Run qpidd with -t to see the frame receive
timings...
s.messageTransfer(arg::content=message, arg::destination="amq.topic");
s.sync();
s.close();
c.close();
}
This is a showstopper to me. I'm interested in any ideas on where/why
this happens and how to dig in and fix it.
Have you looked at the process sizes? I'm wondering if there's some issue with
the overhead of storing messages on the client and/or broker that is making them
grow large enough to cause things to slow down. I can't think of anything that
would explain a 3 second delay that you describe but
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
