[web2py] Re: web2py performance issue or rather my fault?

2010-07-22 Thread mdipierro
One more thing... in your tests, you may want to save the httpserver.log. If you still have problems, it will be useful to compare cherokee logs vs httpserver logs. Massimo On Jul 21, 4:33 pm, Thadeus Burgess thade...@thadeusb.com wrote: I try. I have several copies of web2py core that I have

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
Hi Thadeus, I looked at the code you used for comparing web2py and Flask. This is NOT A FAIR COMPARISON Here is why: 1) you did not set migrate=False in web2py. This means at every request you lock the entire database table metadata to check whether the table has migrated. You do this

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
One more thing because of design flask with generate and reuse pyc file. In the case of web2py python code is executed. You have to explicitly bytecode compile via admin. I am not sure if you have done that. Massimo On Jul 21, 3:36 am, mdipierro mdipie...@cs.depaul.edu wrote: Hi Thadeus,

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Iceberg
: web2py@googlegroups.com To: web2py-users web2py@googlegroups.com Date:Wed, 21 Jul 2010 01:55:27 -0700 (PDT) Subject: [web2py] Re: web2py performance issue or rather my fault? --- One more thing because of design flask with generate and reuse pyc file. In the case

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
@googlegroups.com Date:    Wed, 21 Jul 2010 01:55:27 -0700 (PDT) Subject: [web2py] Re: web2py performance issue or rather my fault? --- One more thing because of design flask with generate and reuse pyc file. In the case of web2py python code is executed. You have

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
On the other hand, Kuba, who started this thread, later said I moved to apache/WSGI from Rocket. For me the problem disappears.  So I guess there is a standalone rule #5: Use apache/WSGI etc. instead of the built-in Rocket ? I do not think it is a web server issue as much as an issue with

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Kuba Kucharski
On Wed, Jul 21, 2010 at 1:54 PM, mdipierro mdipie...@cs.depaul.edu wrote: On the other hand, Kuba, who started this thread, later said I moved to apache/WSGI from Rocket. For me the problem disappears.  So I guess there is a standalone rule #5: Use apache/WSGI etc. instead of the built-in

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
I am not ruling it out. I believe some people are having the problem but not everybody has having this problem. I am trying help isolate the possible causes. You said: I moved to apache/WSGI from Rocket. For me the problem disappears. therefore your problem is not the same that some other

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Kuba Kucharski
You said: I moved to apache/WSGI from Rocket. For me the problem disappears. therefore your problem is not the same that some other problems experienced with apache/WSGI. look at Iceberg, Rahul, MikeEllis in this thread You also said you cache.ram all requests. Can you try remove the caching?

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Kuba Kucharski
therefore your problem is not the same that some other problems experienced with apache/WSGI. exactly, look at Iceberg, Rahul, MikeEllis in this thread I was just pointing out that I am not the only one seeing rocket-specific-problem

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
I agree with your assessment. The problem you, Iceberg, Rahul, and MikeEllis are having with Rocket appears distinct from the problem Thadeus is having uwsgi+cherokee. In my previous email I tried to suggest changes in Thadeus code to isolate the cause of his problem. Going back to your

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Timbo
Can you get me a debug log of the problem as I described earlier? On Jul 21, 8:49 am, Kuba Kucharski kuba.kuchar...@gmail.com wrote: therefore your problem is not the same that some other problems experienced with apache/WSGI. exactly, look at Iceberg, Rahul, MikeEllis in this thread I

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Thadeus Burgess
I will make said changes, and add my Flask-DAL extension to the flask app, so at least the database layer will be the exact same. Flask handles sessions too. I shouldn't disable web2py sessions while letting flask use sessions, that would be an unfair test too. The good news is, that even with

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
Thank you Thadeus for your help with these tests. I did not see a open_session(request) in your Flask code. Where does Flask save sessions? filesystem or client? If they are saved on the client that is closer to session.forget() in web2py since filling the filesystem with one question file for

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
I propose closing this thread and opening two different threads: Rocket performance issues (Tim/Kuba please open it) uWSGI performance issues (Thadeus please open it) Otherwise this is getting more confused that needs to be. Massimo On Jul 21, 9:44 am, Timbo tfarr...@owassobible.org wrote:

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Vidul Petrov
I am in doubt that web2py is the bottleneck. My recent tests on a modest cloud server (2GHz CPU, 1G RAM): 1_000, 5_000, and 10_000 requests in total, 100 concurrent requests - 16 milliseconds per request with no HTTP server errors. The RDBMS and the HTTP server in question were Postgres and

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
Which os? fcgi? On Jul 21, 4:02 pm, Vidul Petrov vidul.r...@gmail.com wrote: I am in doubt that web2py is the bottleneck. My recent tests on a modest cloud server (2GHz CPU, 1G RAM): 1_000, 5_000, and 10_000 requests in total, 100 concurrent requests - 16 milliseconds per request with no HTTP

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread Thadeus Burgess
I try. I have several copies of web2py core that I have modified in attempt to fix these issues, attempting to narrow down the cause, to no avail yet. I want to do two more tests. The first one, make flask use the web2py DAL so we can compare core to core. Make sure web2py has all of the fancy

[web2py] Re: web2py performance issue or rather my fault?

2010-07-21 Thread mdipierro
Thank you Thadeus. This seems a good approach. Massimo On Jul 21, 4:33 pm, Thadeus Burgess thade...@thadeusb.com wrote: I try. I have several copies of web2py core that I have modified in attempt to fix these issues, attempting to narrow down the cause, to no avail yet. I want to do two

[web2py] Re: web2py performance issue or rather my fault?

2010-07-19 Thread David Marko
I can see the same problems with failed requests on many my machines. (WinXP, Debian 5) . The simple test using default web2py examples shows me, that template rendering is OK e.g. http://localhost:8000/examples/template_examples/test_for but when using form example like in

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-19 Thread Thadeus Burgess
I am taking http://techspot.zzzeek.org/?p=17 and making a benchmark_dal.py and testing the results vs sqlalchemy. So far it looks promising, the DAL seems to be much more efficient than SQLAlchemy, and brick tons faster too. This is good news for me, as it proves that the DAL is valuable... if

[web2py] Re: web2py performance issue or rather my fault?

2010-07-19 Thread David Marko
One can even user examples hosted on web2py.com with same results as I tested on my local machines. simple temple rendering - no failed requests ## c:\apache\binab -n 100 -c 5 http://web2py.com/examples/template_examples/test_def This is ApacheBench, Version 2.3 $Revision:

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-19 Thread Thadeus Burgess
Now the question is why? I don't see how it can just be the generation of the form, I am willing to bet it is the session and CSRF tokens. If someone has time, we need to test a couple of scenarios. A) display a simple FORM B) display a simple FORM, but do NOT include a form.accepts in the

[web2py] Re: web2py performance issue or rather my fault?

2010-07-18 Thread mdipierro
Last week I run some tests. I can confirm I have NO dropped requests on web2py.com running with the recommended configuration (apache +mod_wsgi 3.1). Here is the httpserver.log for 3 days of running (I removed the IPs from the logs and requests for static files served directly by apache).

[web2py] Re: web2py performance issue or rather my fault?

2010-07-18 Thread MikeEllis
I was just about to create new topic, but thought I'd start by reporting here since it could be related. The short description is: 1. Serve web2py (rocket.py, sqlite3, etc) over a lan connection from a laptop 2. Open a page from my app on another laptop. 3. Also open the same page on the server

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-18 Thread Thadeus Burgess
I am wondering if this is a web2py issue or a DAL issue. I would like to re-run my tests just rendering a simple template with no dbio and see if I still get the same results. I love the DAL and there is nothing else quite like it, and would hate for it to be the cause. I would like to compare

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-11 Thread Michele Comitini
:Thu, 8 Jul 2010 02:23:56 -0700 (PDT) Subject: [web2py] Re: web2py performance issue or rather my fault? --- Whatever the problem is it must be resolved. The strange thing is I do not experience this on my production system (~5 hits/day and distinct 1600

[web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Rahul
Hi All, I have experienced a similar issue with web2py. Issue: [sometimes, once every few hits, it loads signifacantly slower weirdest thing is when you re-click the link it loads instantly, when you left it working to load on itself, it is slow.. like 4 to 8seconds ] Earlier I reported

[web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Rahul
All, sorry forgot to mention previously we test this in our own network (office intranet) and not the internet. The number of users testing is max 2 to 3. Thanks

[web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread mdipierro
Whatever the problem is it must be resolved. The strange thing is I do not experience this on my production system (~5 hits/day and distinct 1600 visitors/day). What could it be? - cron when it runs spikes CPU usage. DO NOT USE. In production run web2py cron in a separate process, not the one

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Kuba Kucharski
What could it be? let's rule things out - cron when it runs spikes CPU usage. DO NOT USE. In production run not the case. no cron, no cpu spikes - web2py session lock (the the same user can only access one page at I have session.forget.. - sqlite lock (depending on the query and the

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Iceberg
Sender: web2py@googlegroups.com To: web2py-users web2py@googlegroups.com Date:Thu, 8 Jul 2010 02:23:56 -0700 (PDT) Subject: [web2py] Re: web2py performance issue or rather my fault? --- Whatever the problem is it must be resolved. The strange thing is I do not experience

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Thadeus Burgess
Setup: Two applications that perform a simple db insert, db select, render a template and return it. All in all this is a very small application. The application has two views, one insert/select view and one select view. Each of these views are tested separately using apache ab test. The insert

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Kuba Kucharski
- Both my develop machine (windows xp home edition with latest ServicePack), and my production machine (a Linux), have such issue occasionally. Both environments uses web2py's built-in server, rocket or cherrypy. Both serve no more than 10 people or so. @Massimo: You can check your

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Kuba Kucharski
The web server was restarted and the database reset in between each ab test but AFAIK ab tends to be unreliable.. it is also my conclusion from benchmarking rocket vs cherrypy try httperf?

[web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread mdipierro
This is very interesting. I have never used uwsgi and cherokee with web2py. I will try reproduce this. Massimo On 8 Lug, 09:26, Thadeus Burgess thade...@thadeusb.com wrote: Setup: Two applications that perform a simple db insert, db select, render a template and return it. All in all this is

Re: [web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread Thadeus Burgess
I get similar results with httperf. Web2py has over 15% 500 internal server errors, flask has none. -- Thadeus On Thu, Jul 8, 2010 at 10:17 AM, Kuba Kucharski kuba.kuchar...@gmail.com wrote: The web server was restarted and the database reset in between each ab test but AFAIK ab tends to

[web2py] Re: web2py performance issue or rather my fault?

2010-07-08 Thread mdipierro
) Subject: [web2py] Re: web2py performance issue or rather my fault? --- Whatever the problem is it must be resolved. The strange thing is I do not experience this on my production system (~5 hits/day and distinct 1600 visitors/day). What could it be? - cron when it runs

[web2py] Re: web2py performance issue or rather my fault?

2010-07-07 Thread mdipierro
Can you tell us more about the setting of this test. Did you use apache or the built-in server? Did you use the same server in all cases? Massimo On 7 Lug, 16:06, Thadeus Burgess thade...@thadeusb.com wrote: I have experienced this under high capacity on my web2py sites. I have not found a