To me it seems that it comes from : *psycopg2._psycopg*
I use Postgres 8.4 under ubuntu 11.04, python 2.6.5, web2py 1.97.1, with the
dev http server...
In [3]: import cProfile
In [4]: import pstats
In [5]: cProfile.run('SQLTABLE(db().select(db.test_ph.ALL))')
650629 function calls (580822 primitive calls) in 14.921 CPU
seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 14.921 14.921 <string>:1(<module>)
421 0.004 0.000 0.004 0.000
copy.py:112(_copy_with_constructor)
421 0.009 0.000 0.016 0.000 copy.py:65(copy)
677 0.222 0.000 1.243 0.002 dal.py:1046(_select)
677 0.012 0.000 0.012 0.000 dal.py:1155(select_limitby)
677 0.044 0.000 13.706 0.020 dal.py:1161(select)
677 0.017 0.000 4.745 0.007 dal.py:1165(response)
2136/784 0.078 0.000 0.161 0.000 dal.py:1202(tables)
677 0.056 0.000 4.687 0.007 dal.py:1243(log_execute)
677 0.012 0.000 4.699 0.007 dal.py:1250(execute)
676 0.039 0.000 0.068 0.000 dal.py:1253(represent)
676 0.004 0.000 0.004 0.000
dal.py:1320(represent_exceptions)
677 0.007 0.000 0.007 0.000 dal.py:1329(rowslice)
677 2.285 0.003 7.652 0.011 dal.py:1333(parse)
677 0.019 0.000 0.045 0.000 dal.py:1458(filter_tenant)
85 0.007 0.000 0.434 0.005 dal.py:3603(ff)
85 0.007 0.000 0.723 0.009 dal.py:3649(list_ref_repr)
170 0.004 0.000 0.455 0.003 dal.py:3653(<genexpr>)
85 0.008 0.000 0.010 0.000
dal.py:3683(bar_decode_integer)
9310 0.153 0.000 0.214 0.000 dal.py:3697(__getitem__)
27725 0.264 0.000 0.264 0.000 dal.py:3706(__setitem__)
85 0.001 0.000 0.003 0.000 dal.py:3709(__getattr__)
1522 0.022 0.000 0.035 0.000 dal.py:3712(__setattr__)
93512 0.803 0.000 0.803 0.000 dal.py:4248(__getitem__)
677 0.006 0.000 0.006 0.000 dal.py:4251(__setitem__)
25694 0.389 0.000 0.601 0.000 dal.py:4254(__getattr__)
677 0.011 0.000 0.017 0.000 dal.py:4257(__setattr__)
677 0.024 0.000 0.039 0.000 dal.py:4266(__call__)
1522 0.023 0.000 0.033 0.000 dal.py:4381(__setattr__)
73242/6480 1.729 0.000 12.928 0.002 dal.py:4579(__getitem__)
85 0.008 0.000 0.406 0.005 dal.py:4594(__call__)
4989 0.072 0.000 0.255 0.000 dal.py:4647(__getattr__)
4492 0.066 0.000 0.239 0.000 dal.py:4655(__iter__)
19439 0.495 0.000 1.167 0.000 dal.py:4950(__eq__)
676 0.018 0.000 0.039 0.000 dal.py:4953(__ne__)
85 0.003 0.000 0.005 0.000 dal.py:4971(belongs)
425 0.024 0.000 0.060 0.000 dal.py:5247(formatter)
1870 0.013 0.000 0.013 0.000 dal.py:5277(__nonzero__)
4661 0.039 0.000 0.039 0.000 dal.py:5280(__str__)
20200 0.226 0.000 0.226 0.000 dal.py:5301(__init__)
19525 0.155 0.000 0.155 0.000 dal.py:5357(__init__)
677 0.025 0.000 13.746 0.020 dal.py:5393(select)
677 0.006 0.000 0.006 0.000 dal.py:5474(__init__)
86 0.001 0.000 0.002 0.000 dal.py:5518(__nonzero__)
86 0.001 0.000 0.002 0.000 dal.py:5523(__len__)
761 0.033 0.000 0.066 0.000 dal.py:5529(__getitem__)
256 0.005 0.000 0.022 0.000 dal.py:5536(__iter__)
591 0.010 0.000 0.063 0.000 dal.py:5553(first)
85 0.003 0.000 0.023 0.000 dal.py:861(BELONGS)
170 0.002 0.000 0.014 0.000 dal.py:865(<genexpr>)
591 0.013 0.000 0.120 0.000 dal.py:883(EQ)
6013/4661 0.149 0.000 0.326 0.000 dal.py:932(expand)
85 0.007 0.000 0.013 0.000 decimal.py:512(__new__)
86 0.001 0.000 0.080 0.001 html.py:1403(_fixup)
1 0.000 0.000 0.000 0.000 html.py:1410(_fixup)
1 0.000 0.000 0.003 0.003 html.py:1418(_fixup)
1 0.000 0.000 0.000 0.000 html.py:1452(_fixup)
1981 0.110 0.000 0.326 0.000 html.py:551(__init__)
1892 0.013 0.000 0.013 0.000 html.py:670(_fixup)
89 0.043 0.000 0.082 0.001 html.py:679(_wrap_components)
1981 0.012 0.000 0.012 0.000 html.py:705(_postprocessing)
3870 0.053 0.000 0.080 0.000 html.py:750(_setnode)
1 0.182 0.182 14.267 14.267 sqlhtml.py:1305(__init__)
85 0.004 0.000 4.493 0.053 test.py:1126(<lambda>)
85 0.003 0.000 1.016 0.012 test.py:1128(<lambda>)
85 0.003 0.000 1.827 0.021 test.py:1131(<lambda>)
85 0.003 0.000 1.863 0.022 test.py:1133(<lambda>)
85 0.003 0.000 1.721 0.020 test.py:1135(<lambda>)
85 0.003 0.000 1.822 0.021 test.py:1137(<lambda>)
170 0.003 0.000 0.003 0.000 tz.py:37(__init__)
3058 0.044 0.000 0.073 0.000 utf_8.py:15(decode)
506 0.003 0.000 0.003 0.000 validators.py:106(formatter)
3058 0.029 0.000 0.029 0.000 {_codecs.utf_8_decode}
85 0.001 0.000 0.001 0.000 {built-in method __new__ of
type object at 0x82e5e0}
340 0.002 0.000 0.002 0.000 {built-in method group}
7788 0.083 0.000 0.083 0.000 {built-in method match}
1182 0.016 0.000 0.052 0.000 {hasattr}
130035 0.925 0.000 0.925 0.000 {isinstance}
4187/4101 0.029 0.000 0.030 0.000 {len}
677 0.035 0.000 0.188 0.000 {map}
783 0.005 0.000 0.005 0.000 {method 'add' of 'set'
objects}
10120 0.066 0.000 0.066 0.000 {method 'append' of 'list'
objects}
2956 0.046 0.000 0.117 0.000 {method 'decode' of 'str'
objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
3052 0.027 0.000 0.027 0.000 {method 'encode' of 'unicode'
objects}
* 677 4.572 0.007 4.572 0.007 {method 'execute' of
'psycopg2._psycopg.cursor' objects}*
677 0.013 0.000 0.029 0.000 {method 'fetchall' of
'psycopg2._psycopg.cursor' objects}
15868 0.107 0.000 0.107 0.000 {method 'get' of 'dict'
objects}
73327 0.481 0.000 0.481 0.000 {method 'isdigit' of 'str'
objects}
85 0.001 0.000 0.001 0.000 {method 'items' of 'dict'
objects}
2115/1860 0.019 0.000 0.486 0.000 {method 'join' of 'str'
objects}
2199 0.016 0.000 0.016 0.000 {method 'keys' of 'dict'
objects}
1 0.000 0.000 0.000 0.000 {method 'lower' of 'str'
objects}
421 0.007 0.000 0.007 0.000 {method 'reverse' of 'list'
objects}
8550 0.064 0.000 0.064 0.000 {method 'split' of 'str'
objects}
28489 0.197 0.000 0.197 0.000 {method 'startswith' of 'str'
objects}
5086 0.032 0.000 0.032 0.000 {method 'strip' of 'str'
objects}
1352 0.010 0.000 0.010 0.000 {method 'union' of 'set'
objects}
1354 0.018 0.000 0.018 0.000 {time.time}
What do you think?
Richard
On Tue, Aug 2, 2011 at 9:01 AM, Anthony <[email protected]> wrote:
> On Tuesday, August 2, 2011 8:54:57 AM UTC-4, David Marko wrote:
>>
>> In the first example, web2py converts SQL query results into DAL row
>> objects which takes time for 31K records. Using executesql is always faster
>> ...
>
>
> Good point -- executesql isn't returning the same object:
> http://web2py.com/book/default/chapter/06#executesql
>
> Anthony
>
