Re: [Zope] Re: Give it a rest + answers. (Re: Re: Zope + Apache on Quad Debian machine)

2006-03-23 Thread Chris Withers

Andrew Sawyers wrote:

The other engineer was me, and it was close enough to double the load
per instance, per cpu. 


tangential question: I'm assuming you were using LVS in front of these 
boxes? If so, how'd you get the both clients on the box accessible by 
LVS? 2 ip addresses?


cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Give it a rest + answers. (Re: Re: Zope + Apache on Quad Debian machine)

2006-03-23 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Chris Withers wrote:
 Andrew Sawyers wrote:
 
 The other engineer was me, and it was close enough to double the load
 per instance, per cpu. 
 
 
 tangential question: I'm assuming you were using LVS in front of these
 boxes? If so, how'd you get the both clients on the box accessible by
 LVS? 2 ip addresses?

Nope, different port numbere.  LVS is configured per-service, rather
than per-IP.


Tres.
- --
===
Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEIo3E+gerLs4ltQ4RAmBaAKCwiU6E/cnJNRQBTSNDk32j+hKRkACgjhgn
awM/z7gLOzscAeCzKY5Aexk=
=0HJ9
-END PGP SIGNATURE-
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


[Zope] Re: Give it a rest + answers. (Re: Re: Zope + Apache on Quad Debian machine)

2006-03-22 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dario Lopez-Kästen wrote:

snip

 Let me be the first to admit my total lack of knowledge of kernel task
 schedulers, but generally speaking, unless the scheduler makes sure that
 a threaded python process never ever gets distributed over two
 processors simultaniously,  then the GIL *will* be an issue.

The hypothesis is that the scheduler notices that threads in a single
Python process are always blocking on a single kernel resource (the
semaphore / mutex which is the GIL), and hence migrates them all to the
same CPU (over time).

I could imagine that this is like the hand-rolled assembly vs.
compiler-generated code case:  older kernels (like the one your old app
was dpeloyed atop, perhaps?) didn't have such smarts, but newsr ones do.


 In any event, I'd love to be proved wrong about the need for taskset
 (especially if someone comes up with measurable data to that effect)
 because it means that my sysadmins can simplify their setup for managing
 Zope (making Zope even more acceptable :-).

WHile at ZC, one of the other engineers and I did some testing on SMB
boxes, and found that one appserver per CPU gave us near linear
scaling of the application, without any explicit affinity set.  I don't
have the numbers (we were using stock Dell 1U dual CPU boxes, I think),
but the win seemed clear enough that we quit invesigating taskset.


Tres.
- --
===
Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEIVAK+gerLs4ltQ4RAnb9AKC7qw38+BqNdAbY79bqPR4/G7USCwCbBmit
UX+GcftjNQ5fUKajALsEbSk=
=+E5c
-END PGP SIGNATURE-

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Give it a rest + answers. (Re: Re: Zope + Apache on Quad Debian machine)

2006-03-22 Thread Andrew Sawyers
On Wed, 2006-03-22 at 08:24 -0500, Tres Seaver wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 
 WHile at ZC, one of the other engineers and I did some testing on SMB
 boxes, and found that one appserver per CPU gave us near linear
 scaling of the application, without any explicit affinity set.  I don't
 have the numbers (we were using stock Dell 1U dual CPU boxes, I think),
 but the win seemed clear enough that we quit invesigating taskset.
 
The other engineer was me, and it was close enough to double the load
per instance, per cpu.  If he needs to test it, the numbers are easy to
get.  Run a single instance on a SMP Server, and hit it with load. You
will get Y.  Run another instance and spread that load across both
instances and you will get nearly 2xY.  Take a single cpu server and run
1 instance and load test it, and you'll get Y.  Take that same single
cpu server and run 2 instance, and load test it across both instances,
and you'll get Y.  These tests were done on both Linux and FreeBSD and
both were approximately the same.  The conclusion was, no further
complexity is required to get a substantial performance benfit by
running SMP Servers with multiple Zeo Clients instances, where the # of
CPUS == the # of Zeo Client instances.  This has been posted to the
lists serveral times BTW.

The only snafu is to make sure you have enough RAM to run several zeo
client instances.  If you've got a 4way server, I'd recommend 16GBs of
RAM if it's a big site.  I personally like to put as much ram into a
server as it can physically handle.

Andrew Sawyers
 
 Tres.
 - --
 ===
 Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
 Palladion Software   Excellence by Designhttp://palladion.com
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.1 (GNU/Linux)
 Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
 
 iD8DBQFEIVAK+gerLs4ltQ4RAnb9AKC7qw38+BqNdAbY79bqPR4/G7USCwCbBmit
 UX+GcftjNQ5fUKajALsEbSk=
 =+E5c
 -END PGP SIGNATURE-
 
 ___
 Zope maillist  -  Zope@zope.org
 http://mail.zope.org/mailman/listinfo/zope
 **   No cross posts or HTML encoding!  **
 (Related lists - 
  http://mail.zope.org/mailman/listinfo/zope-announce
  http://mail.zope.org/mailman/listinfo/zope-dev )

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )