Thanks Massimo, Bottiger, good to know some keys that I didn't know,
like tricks to speed up the app, and the python efficiency in multicore.
Excuse my ignorance , but let me enquire a bit more...
* 10x more requests by only installing a simple package (flup)? or
is an special server? or is superman?
o Bottiger, I tried to find out some information and
documentation about flup but -apart from official trac wiki-
it's lack of
o Could you give us some introduction about Flup, Prefork
server (or whatever it is) and how to install it to improve
the performance?
* So (according to what I've been reading) I conclude that a great
performance machine for stressed production enviroment would be
made up of:
o Multiple core machine +
o Flup Server packages +
o Cherokee +
o Postgres +
o Web2py Tricks
* Massimo, in a old thread I read something about a prebuilt vmware
with cherokee and web2py, ready to install, is it any news about this?
o In terms of "marketing", I think that if we could upload a
vmware turnkey appliance to here
<http://www.vmware.com/appliances/> (like Django one
<http://www.vmware.com/appliances/directory/82433>), it
would be increase our presence
Thanks,
alex f
El 22/07/2009 10:48, mdipierro escribió:
> With 8 cores and Flup you can probably get close to 1000 requests/
> second.
> if you can do any test, let us know what you get.
>
> Massimo
>
> On Jul 22, 3:22 am, Bottiger<[email protected]> wrote:
>
>> "There are some things that definitively will NOT help: a multicore
>> machine. because the python interpreter cannot use
>> multiple cores efficiently "
>>
>> If you use the prefork flup server not included in Web2Py, each
>> request is handled by a seperate process so this will bypass the GIL.
>> It makes a huge difference on my 8 core server. Presumably, Jython
>> which is GILless would yield the same performance on a multicore
>> machine.
>>
>> "I will actually argue most of the map/reduce DB out there are slower
>> then postresql. "
>>
>> For minimal latency, this is probably true, but I am willing to bet
>> that the latency grows far slower in a map/reduce DB as the number of
>> simultaneous connection rises.
>>
>> On Jul 22, 1:03 am, mdipierro<[email protected]> wrote:
>>
>>
>>> This is a complex issues that spans different topics: speed,
>>> efficiency, scalability.
>>>
>>> I am not aware of any major bottle neck in web2py, execpt of the
>>> database (not the DAL , the actual database) so it is efficiently.
>>> There are many little tricks you can use to speed applications more:
>>> - use connection pooling
>>> - run your app bytecode compiled (press the button)
>>> - move as much code as you can modules instead of models.
>>> - discard sessions unless you they have modified
>>> - store sessions in a memory mapped file
>>>
>>> How fast is it (with or without optimizations) depends on the
>>> architecture. Is there a machine that can give you 1000 request/
>>> seconds. I do know. It is possible. On my virtual machine I get about
>>> 100.
>>>
>>> There are some things that definitively will NOT help:
>>> - a multicore machine. because the python interpreter cannot use
>>> multiple cores efficiently
>>> - a key/value database. This helps with scalability (i.e. run lots of
>>> concurrent servers) but not necessarily speeds up a single server. I
>>> will actually argue most of the map/reduce DB out there are slower
>>> then postresql.
>>>
>>> Massimo
>>>
>>> On Jul 21, 8:40 pm, Alex Fanjul<[email protected]> wrote:
>>>
>>>> Hello Massimo (all) this days I'm reading about horizontal scale
>>>> architectures, key/value and graphs db's, etc. and the awakening in
>>>> cloud computing enviroment
>>>> In the last reading, I saw "Redit" benchmark of about 50 to 100
>>>> thousands req/sec with standard linux box.
>>>>
>>>> I know those values are due to DB architecture of key/value (and they
>>>> are really incredible), but:
>>>> -what thing is really limiting 1000 req/sec in web2py?
>>>> cherrypy/apache? mysql/postgres? wscgi/fastcgi? web2pyframework? python?
>>>> -what do you think would the upper limit (req/sec) be in the better
>>>> production enviroment case (great linux server/s, apache/cherokee?, best
>>>> connection)
>>>> -As a matter of curiosity, have you ever though in implementing the
>>>> API for any of such databases? Redit? Tokyo? couchDB?
>>>>
>>>> regards,
>>>> alex f
>>>>
>>>> P.S as always i'm sorry for my poor english
>>>>
>>>> El 21/07/2009 15:52, mdipierro escribió:
>>>>
>>>>> - Are there any large web2py installations that I can quote as an
>>>>> example
>>>>>
>>>>> Not that I know and handle>1000 requests/second.
>>>>>
>>>>> - How are the issues of caching (say rendered pages) handled? I have
>>>>> done a few Drupal sites and can see the performance effects of caching
>>>>> very clearly. IIRC only Django has caching in the python world?
>>>>>
>>>>> If you use multiple installations behind a load balancer I suggest you
>>>>> use the "pound" load balancer to keep sessions sticky. In that case
>>>>> the different processes do not need to share any data.
>>>>>
>>>>> - Has anyone done any work with web2py in a cluster (similar to a
>>>>> Tomcat cluster behind mod_jk)? (multiple machines running web2py, the
>>>>> session data sync'd etc. I can put the session info in a shared FS
>>>>> though)
>>>>>
>>>>> If you need sessions and you need sessions synced, I suggest you share
>>>>> the sessions folder.
>>>>>
>>>>> Massimo
>>>>>
>>>>> On Jul 21, 12:20 am, Bottiger<[email protected]> wrote:
>>>>>
>>>>>> If it is truly not computationally intensive, and does not even use a
>>>>>> database, it should not be a problem.
>>>>>>
>>>>>> I have benchmarked Web2Py on the static welcome page to 700 requests/
>>>>>> second with a concurrency level of 50.
>>>>>>
>>>>>> To increase the level of concurrency (if you have additional CPU
>>>>>> cores), you should increase the number of Web2Py processes.
>>>>>>
>>>>>> "~ 8000 users, and atleast 1000-2000 simultaneous users."
>>>>>>
>>>>>> This is not really a large installation if it doesn't use a database.
>>>>>>
>>>>>> "How are the issues of caching (say rendered pages) handled? I have
>>>>>> done a few Drupal sites and can see the performance effects of caching
>>>>>> very clearly. IIRC only Django has caching in the python world?"
>>>>>>
>>>>>> Drupal, Django, and Web2Py have equivalent caching mechanisms. Any
>>>>>> external caching mechanism you have seen with Drupal should also be
>>>>>> usable with Web2Py or Django.
>>>>>>
>>>>>> "I can put the session info in a shared FS though"
>>>>>>
>>>>>> You can either do that or use a database for sessions.
>>>>>>
>>>>>> On Jul 20, 8:54 pm, Anand Vaidya<[email protected]> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> After a couple of web2py projects, I am confident of coding a fairly
>>>>>>> big app in web2py.
>>>>>>>
>>>>>>> My previous projects did not need any database (we had to use
>>>>>>> flatfiles), the new project is also similar. I intend to bypass the
>>>>>>> models etc completely.
>>>>>>>
>>>>>>> The app is likely to be used in a corporate setting with ~ 8000 users,
>>>>>>> and atleast 1000-2000 simultaneous users.
>>>>>>>
>>>>>>> The users authenticate to an LDAP server.
>>>>>>>
>>>>>>> The app is not computationally intensive
>>>>>>>
>>>>>>> It queries another service and displays results
>>>>>>>
>>>>>>> No SQL DB is required
>>>>>>>
>>>>>>> Most likely behind a few Apache 2.x front server
>>>>>>>
>>>>>>> I'd like to know:
>>>>>>>
>>>>>>> - Are there any large web2py installations that I can quote as an
>>>>>>> example
>>>>>>>
>>>>>>> - How are the issues of caching (say rendered pages) handled? I have
>>>>>>> done a few Drupal sites and can see the performance effects of caching
>>>>>>> very clearly. IIRC only Django has caching in the python world?
>>>>>>>
>>>>>>> - Has anyone done any work with web2py in a cluster (similar to a
>>>>>>> Tomcat cluster behind mod_jk)? (multiple machines running web2py, the
>>>>>>> session data sync'd etc. I can put the session info in a shared FS
>>>>>>> though)
>>>>>>>
>>>>>>> Regards
>>>>>>> Anand
>>>>>>>
>>>> --
>>>> Alejandro Fanjul Fdez.
>>>> [email protected]
>>>>
> >
>
>
--
Alejandro Fanjul Fdez.
[email protected]
www.mhproject.org
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
