Steve,
Managed to run some more performance tests today using a RHEL5u4 VM on a
Dell R710 . Ran qpid-perftest with default values on the same VM as
qpidd, each test ran several times with the calculated average shown in
the table below.
CPUs RAM Publish Consume
2 4G 48K 46K
4 4G 65K 60K
6 4G 73K 66K
2 8G 46K 44K
4 8G 65K 61K
6 8G 74K 67K
Basically it confirms your assertion about the broker using more threads
under heavy load. Changing the VM memory had no discernible effect on
performance, but increasing the number of CPU's available to the VM had
a big effect on throughput.
So when defining a VM for QPID transient usage focus on CPU allocation!!!
Thanks for the advise and help.
Clive
On 03/05/2012 15:27, Steve Huston wrote:
Hi Clive,
The broker will use threads based on load - if the broker takes longer to
process a message than qpid-perftest takes to send the next message, the
broker would need more threads.
A more pointed test for broker performance would be to run the client on
another host - then you know the non-VM vs. VM differences are just the
broker's actions. It may be a little confusing weeding out the actual vs.
virtual NIC issues, but there would be no confusion about how much the
client is taking away from resources available to the broker.
-Steve
-----Original Message-----
From: CLIVE [mailto:[email protected]]
Sent: Wednesday, May 02, 2012 5:28 PM
To: [email protected]
Cc: Steve Huston; 'James Kirkland'
Subject: Re: QPID performance on virtual machines
Steve,
I thought about this as well. So re-started the broker on the physical
Dell
R710 with the threads option set to just 4 and saw the same throughput
values (85000 publish and 80000 subscribe). As reducing the threads
count
didn't seem to have much effect on the physical machine I thought that
this
probably wasn't the issue.
As the qpid-perftest application was only creating 1 producer and 1
consumer
I reasoned that perhaps the broker was only using two threads too
service
the read and writes from these clients. This was why reducing the thread
count on the broker had no effect. Would you expect the broker to use
more
than two threads to service the clients for this scenario?
I will rerun the test tomorrow based on an increased number of CPU's in
the
VM(s) just to double check whether it is a number of cores issue.
I did run 'strace -c' on qpidd while the test was running to count the
number
of system calls and I noted the big hitters were futex and write.
Interestingly the reads read in 64K chunks, but the writes were only
2048 bytes at a time. As a result the number writes occurring were an
order
of magnitude bigger than the reads; I left the detailed results at work
so
apologies for not quoting the actual figures.
Clive
On 02/05/2012 20:23, Steve Huston wrote:
The qpid broker learns how many CPUs are available and will run more
I/O threads when more CPUs are available (#CPUs + 1 threads). It would
be interesting to see the results if your VM gets more CPUs.
-Steve
-----Original Message-----
From: CLIVE [mailto:[email protected]]
Sent: Wednesday, May 02, 2012 1:30 PM
To: James Kirkland
Cc: [email protected]
Subject: Re: QPID performance on virtual machines
James,
qpid-perf-test (as supplied with the qpid-0.14 source tar ball) runs
a
direct
queue test when executed without any parameters; there is a command
line option that enables this to be be changed if required. The
message size
is
1024K (again default size when not explicitly set). And
500000 messages are published by the test (again the default when not
explicitly set). All messages are transient so I wouldn't expect any
file I/O
overhead to interfere with the test and this is confirmed by the
vmstat results I am seeing. The only jump in the vmstat output is the
number of context switches that are occurring which jumps up into the
thousands.
Clive
On 02/05/2012 18:10, James Kirkland wrote:
What sort of messging scenario is it? Are the messages persisted?
How big are they? If they are persisted are you using virtual disks
or physical devices?
CLIVE wrote:
Hi all,
I have been undertaking some performance profiling of QPID version
0.14 over the last few weeks and I have found a significant
performance drop off when running QPID in a virtual machine.
As an example if I run qpidd on an 8 core DELL R710 with 36G RAM
(RHEL5u5) and then run qpid-perf-test (on the same machine to
discount any network problems) without any command line
parameters
I am seeing about 85,000 publish transfers/sec and 80000 consume
transfers/sec. If I run the same scenario on a VM (tried both KVM
and VMWare ESXi 4.3 running RHEL5u5) with 2 cores and 8G RAM, I am
seeing only 45000 publish transfers/sec and 40000 consume
transfers/sec. A significant drop off in performance. Looking at
the cpu and memory usage these would not seem to be the limiting
factors as the memory consumption of qpidd stays under 200 MBytes
and its CPU is up at about 150%; hence the two core machine.
I have even run the same test on my Mac Book at home using VMWare
Fusion 4 ( 2 Core 4G RAM) and see the same 45000/40000
transfers/sec results.
I would expect a small drop off in performance when running in a
VM, but not to the extent that I am seeing.
Has anyone else seen this and if so were they able to get to the
bottom of the issue.
Any help would be appreciated.
Clive Lilley
--
James Kirkland
Principal Enterprise Solutions Architect
3340 Peachtree Road, NE,
Suite 1200
Atlanta, GA 30326 USA.
Phone (404) 254-6457<https://www.google.com/voice#phones>
RHCE Certificate: 805009616436562
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For
additional commands, e-mail: [email protected]
.
.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
