Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-26 Thread Bob Ippolito
On 8/25/07, Cliff Wells [EMAIL PROTECTED] wrote: On Sat, 2007-08-25 at 20:24 +0300, Pekka Jääskeläinen wrote: On 8/25/07, Ben Bangert [EMAIL PROTECTED] wrote: I'd highly suggest memcached rather than database backend. It's easy to setup, and of course, very fast.

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-26 Thread Pekka Jääskeläinen
On 8/26/07, Bob Ippolito [EMAIL PROTECTED] wrote: Well if you have a cache that fills up with active sessions then you The cache is used to store also other objects, thus it eventually will fill up. Sure, if I have a separate memcached instance only for session data, this should not be a

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-26 Thread Pekka Jääskeläinen
On 8/26/07, Bob Ippolito [EMAIL PROTECTED] wrote: It will eventually fill up, but data in memcached is a combination of (opt-in) expiration and LRU. Sessions shouldn't be least recently used, and they only get expired if you gave them an expiration time (which probably makes sense to do).

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-25 Thread Ben Bangert
On Aug 24, 2007, at 4:15 PM, Cliff Wells wrote: Memcached seems the easiest (and probably best) solution. As Philip mentions, however, it isn't well-documented how to use Beaker with Memcached. If you decide to go this route, maybe update the wiki? I'd highly suggest memcached rather than

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-25 Thread Pekka Jääskeläinen
On 8/25/07, Ben Bangert [EMAIL PROTECTED] wrote: I'd highly suggest memcached rather than database backend. It's easy to setup, and of course, very fast. :) Yes, but as far as I know, memcached objects can be always replaced, that is, you cannot define an object to be persistent, can you?

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-25 Thread Cliff Wells
On Sat, 2007-08-25 at 20:24 +0300, Pekka Jääskeläinen wrote: On 8/25/07, Ben Bangert [EMAIL PROTECTED] wrote: I'd highly suggest memcached rather than database backend. It's easy to setup, and of course, very fast. :) Yes, but as far as I know, memcached objects can

implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Pekka Jääskeläinen
Hello, In our new project we want to implement the web application from the beginning to be easily scalable to 1) multiple cores (on the same server) and to 2) multiple separate servers. Due to the infamous GIL ruining multithreading scalability of Python, the only sensible way to implement both

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Philip Jenvey
On Aug 24, 2007, at 4:41 AM, Pekka Jääskeläinen wrote: Hello, In our new project we want to implement the web application from the beginning to be easily scalable to 1) multiple cores (on the same server) and to 2) multiple separate servers. Due to the infamous GIL ruining

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Cliff Wells
On Fri, 2007-08-24 at 14:41 +0300, Pekka Jääskeläinen wrote: However, we started to think the practical issues with this in Pylons. In principle, making this work reliably means to distribute the session data so all server processes can access each session's data. I'm curious about this

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Cliff Wells
On Fri, 2007-08-24 at 15:16 -0700, Cliff Wells wrote: On Fri, 2007-08-24 at 14:41 +0300, Pekka Jääskeläinen wrote: However, we started to think the practical issues with this in Pylons. In principle, making this work reliably means to distribute the session data so all server processes

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Bob Ippolito
On 8/24/07, Cliff Wells [EMAIL PROTECTED] wrote: On Fri, 2007-08-24 at 15:16 -0700, Cliff Wells wrote: On Fri, 2007-08-24 at 14:41 +0300, Pekka Jääskeläinen wrote: However, we started to think the practical issues with this in Pylons. In principle, making this work reliably means to

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Pekka Jääskeläinen
Apache 2.2 has a mod_proxy_balancer. If performance is a concern, you should go with the CherryPy WSGI server. use = egg:PasteScript#cherrypy instead of use egg:Paste#httpserver This didn't work, I changed the line use = egg:Paste#http to use egg:Paste#cherrypy Even though I just

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Pekka Jääskeläinen
On 8/25/07, Cliff Wells [EMAIL PROTECTED] wrote: My only possible explanation is that either a) Nginx makes some attempt to track sessions itself and always passes the same IP back to the same backend Pylons process or b) Pylons automagically makes it work. ...or c) you don't use much the

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Cliff Wells
On Sat, 2007-08-25 at 01:46 +0300, Pekka Jääskeläinen wrote: On 8/25/07, Cliff Wells [EMAIL PROTECTED] wrote: My only possible explanation is that either a) Nginx makes some attempt to track sessions itself and always passes the same IP back to the same

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Cliff Wells
On Sat, 2007-08-25 at 01:42 +0300, Pekka Jääskeläinen wrote: Apache 2.2 has a mod_proxy_balancer. If performance is a concern, you should go with the CherryPy WSGI server. use = egg:PasteScript#cherrypy instead of

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Pekka Jääskeläinen
On 8/25/07, Cliff Wells [EMAIL PROTECTED] wrote: This is wrong AFAIK. I'm using CP's wsgiserver (which is a standalone app and included with Paste, so you don't actually need to install CP3, although you certainly can), and this is my entry: [server:main] # use = egg:Paste#http use =

Re: implementing a scalable (to multiple processors and multiple servers) Pylons webapp

2007-08-24 Thread Peter Hansen
On 8/24/07, Pekka Jääskeläinen [EMAIL PROTECTED] wrote: Apache 2.2 has a mod_proxy_balancer. If performance is a concern, you should go with the CherryPy WSGI server. use = egg:PasteScript#cherrypy instead of use egg:Paste#httpserver This didn't work, I changed the line use