I am sorry if I failed to put it clearly or I am asking something absurd.
I am not sure why are you relating it to the Mysql server. I am summarizing
my observations below.
*Context:*
I am testing using ab. The HTTP request does not make any database query.
Number of Daemon processes: 10
Max Number of Mysql connections allowed: 1000
DAL pool size: 100
*Case 1:*
Number of Tables in db.py: ~100 (all with migrate=False)
No. of concurrent requests: 1
Time taken per request: 900 ms
*Case 2:*
Number of Tables in db.py: ~100 (all with migrate=False)
No. of concurrent requests: 25
Time taken per request: 4440 ms
*Case 3:*
Number of Tables in db.py: =2 (all with migrate=False)
No. of concurrent requests: 25
Time taken per request: 1075 ms
In case 2, extra time gets consumed in loading the models in main.py web2py
file.
Logs for case 2:
16217 function calls (16199 primitive calls) in 1.558 seconds
Ordered by: internal time, call count
List reduced from 188 to 50 due to restriction <50>
ncalls tottime percall cumtime percall filename:lineno(function)
1710 0.342 0.000 0.530 0.000
/home/www-data/lambda/gluon/dal.py:8247(__getitem__)
14 0.188 0.013 0.188 0.013
/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py:97(nextset)
183 0.152 0.001 0.152 0.001
/usr/lib/python2.7/threading.py:146(acquire)
366 0.131 0.000 0.132 0.000
/usr/lib/python2.7/genericpath.py:38(isdir)
102 0.107 0.001 0.241 0.002
/home/www-data/lambda/gluon/dal.py:8025(__init__)
595 0.102 0.000 0.556 0.001
/home/www-data/lambda/gluon/dal.py:9100(__init__)
2242 0.086 0.000 0.100 0.000
/home/www-data/lambda/gluon/dal.py:8317(__setattr__)
596 0.055 0.000 0.055 0.000
/home/www-data/lambda/gluon/dal.py:5060(cleanup)
379 0.052 0.000 0.052 0.000
/usr/lib/python2.7/posixpath.py:68(join)
3 0.048 0.016 0.048 0.016
/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py:277(_do_query)
101 0.038 0.000 0.681 0.007
/home/www-data/lambda/gluon/dal.py:7658(lazy_define_table)
101 0.036 0.000 0.073 0.001
/home/www-data/lambda/gluon/dal.py:8198(_create_references)
520 0.035 0.000 0.214 0.000
/home/www-data/lambda/gluon/dal.py:6620(sqlhtml_validators)
101 0.035 0.000 0.717 0.007
/home/www-data/lambda/gluon/dal.py:7628(define_table)
1 0.029 0.029 1.369 1.369
/home/www-data/lambda/applications/chat/compiled/models/db.py:1(<module>)
158 0.024 0.000 0.024 0.000
/home/www-data/lambda/gluon/validators.py:2564(__init__)
1384 0.015 0.000 0.138 0.000
/home/www-data/lambda/gluon/dal.py:8339(__iter__)
102 0.014 0.000 0.014 0.000
/home/www-data/lambda/gluon/dal.py:457(pluralize)
695 0.013 0.000 0.013 0.000
/home/www-data/lambda/gluon/dal.py:8288(__setitem__)
414 0.013 0.000 0.013 0.000
/home/www-data/lambda/gluon/dal.py:7762(__getattr__)
89 0.012 0.000 0.014 0.000
/home/www-data/lambda/gluon/validators.py:450(__init__)
6 0.005 0.001 0.005 0.001
/usr/lib/python2.7/socket.py:223(meth)
Logs for case 3:
14877 function calls (14855 primitive calls) in 0.117 seconds
Ordered by: internal time, call count
List reduced from 220 to 50 due to restriction <50>
ncalls tottime percall cumtime percall filename:lineno(function)
8 0.025 0.003 0.025 0.003
/home/www-data/lambda/gluon/restricted.py:190(compile2)
102 0.017 0.000 0.031 0.000
/home/www-data/lambda/gluon/dal.py:8025(__init__)
596 0.012 0.000 0.015 0.000
/home/www-data/lambda/gluon/dal.py:9100(__init__)
1726 0.006 0.000 0.006 0.000
/home/www-data/lambda/gluon/dal.py:8247(__getitem__)
2242 0.005 0.000 0.007 0.000
/home/www-data/lambda/gluon/dal.py:8317(__setattr__)
520 0.005 0.000 0.015 0.000
/home/www-data/lambda/gluon/dal.py:6620(sqlhtml_validators)
101 0.004 0.000 0.063 0.001
/home/www-data/lambda/gluon/dal.py:7658(lazy_define_table)
1392 0.004 0.000 0.007 0.000
/home/www-data/lambda/gluon/dal.py:8339(__iter__)
1 0.003 0.003 0.091 0.091
/home/www-data/lambda/applications/chat/models/db.py:1(<module>)
101 0.003 0.000 0.066 0.001
/home/www-data/lambda/gluon/dal.py:7628(define_table)
695 0.003 0.000 0.003 0.000
/home/www-data/lambda/gluon/dal.py:8288(__setitem__)
101 0.003 0.000 0.007 0.000
/home/www-data/lambda/gluon/dal.py:8198(_create_references)
596 0.002 0.000 0.002 0.000
/home/www-data/lambda/gluon/dal.py:5060(cleanup)
151 0.002 0.000 0.002 0.000
/home/www-data/lambda/gluon/custom_import.py:37(custom_importer)
89 0.002 0.000 0.003 0.000
/home/www-data/lambda/gluon/validators.py:450(__init__)
102 0.002 0.000 0.002 0.000
/home/www-data/lambda/gluon/dal.py:457(pluralize)
181 0.001 0.000 0.004 0.000
/home/www-data/lambda/gluon/languages.py:867(translate)
160 0.001 0.000 0.007 0.000
/home/www-data/lambda/gluon/validators.py:77(translate)
82 0.001 0.000 0.004 0.000
/home/www-data/lambda/gluon/validators.py:689(__init__)
184 0.001 0.000 0.001 0.000
/home/www-data/lambda/gluon/languages.py:338(__init__)
181 0.001 0.000 0.005 0.000
/home/www-data/lambda/gluon/languages.py:367(__str__)
*Doubt 1:*
If migrate is set to False, then ideally, I would expect the models to load
in same amount of time irrespective of the number of concurrent requests.
*Doubt 2:*
For a given number of concurrent requests (25 here) and migrate set to
False, reducing the number of table definitions should only marginally
improve the response time, not reduce it by 80%.
I have not set lazy_tables=True anywhere yet. So it is taking the default
settings. I will try it and update the thread.
Thanks for your time,
Saurabh
On Tue, Nov 26, 2013 at 2:27 AM, Niphlod <[email protected]> wrote:
> sorry..... but are you **really** asking why your mysql instance suffers
> from 100 concurrent connections ?
> and why web2py takes some time to parse 100 table definitions ?
> and .... you're sure that even with migrate=False and lazy_tables=True
> those timings are comparable ?
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/rH1C7iXMPNA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
