Gino Ruopolo wrote:
When I get a chance, I'll reproduce many
connections/s (instead of
bytes/s) across zones, compared to intra-zone, using
Apache's 'ab'
tool, and report what I get.  -- Jeff

that would be very similar to my setup. thankyou
This message posted from
As promised, I've done some tests on the scenario Gino described: higher CPU overhead for a workload containing many connections per second when client and server processes are in different zones than when in the same zone. Sorry for the delay, but besides other activities getting in the way, I wanted to conduct good tests where I had a reasonable degree of confidence that I was correctly measuring the right thing. My apologies for the long e-mail - send to bit bucket if it's too much detail :-)

I tested with Nevada build 55 (perhaps I would get different numbers on stock Solaris 10, so maybe I'll retry this next week on a machine running S10), with most tests done on a fairly old PC (Pentium III at 1Ghz). I created zones named zone1 and zone2 (very creative, I know...), and configured Apache 2. For workload, I used the Apache benchmark tool ('ab'), and the http_load from In each test, I fetched the same URL, pointing to static HTML, in order to eliminate variable overhead from dynamic content and/or disk I/O.

Many thousands of web hits later and multiple runs with an otherwise idle box, I measured an average 692.82 hits per second when both http_load and httpd were in the same zone, and 679.27 going from zone2 to zone1. That's a ratio of 0.9804, so inter-zone results were 98.04% of the rate of same-zone traffic. Normally I'd be inclined to consider under 2% difference "noise", but the results over multiple trials (and also when using 'ab' instead of http_load) seemed consistent: a small difference in the same direction.

I then used DTrace to see where I was spending time, both elapsed and CPU time (timestamp and vtimestamp) and got the following results (shown is output from same-zone traffic) for the most dominant system calls:

vtime by function
httpd writev 71642973 httpd accept 71720121 httpd shutdown 76473328 http_load close 89913014 http_load connect 149196319

elapsed time by function
httpd writev 85014366 httpd shutdown 90492746 http_load close 102127944 http_load connect 162967191 httpd accept 17908653425

total vtime by execname:        http_load     351417154, httpd 434824250
total wall time by execname:  http_load     456124949; httpd 18503892793

The actual times aren't of consequence: what is significant is where time was spent (the same system calls consistently appeared in both same-zone and different-zone cases), and the different between same-zone and different zone tests. I measured CPU time slightly higher in the different-zone case: 5.76% higher for client (http_load), and 2.04% higher for Apache httpd. So pending further investigation, it does seem there is a small difference in time when an application with many connections per second is in the same zone compared to different zones.

This does not explain the issues Gino raised in his original post, where he reported a bigger change in performance. It does not seem to be related to the cost of network connections, and we have a few ideas on where they may be. We've corresponded privately, and I've sent him some DTrace scripts, so we will pursue this further together. Thank goodness for DTrace!

cheers, Jeff

zones-discuss mailing list

Reply via email to