I understand what you say below but I think it is important to measure the total speedup from the original/stable sql.py and the one in trunk on the same machine and same complex model that you have.
Massimo On Jun 8, 9:54 am, Alexey Nezhdanov <[email protected]> wrote: > On Monday 08 June 2009 17:57:36 mdipierro wrote:> Thanks Alexey, since these > tests are on a different architecture, can > > you give the total relative speedup on the same architecture for > > executing your original model (code in trunk vs original code)? > > You see, I switched to another architecture because testing it on my laptop > become unreliable and obtaining more-or-less stable values required a lot of > time. Powersave and CPU frequency scaling features kick in and introduce a > great deal of noise. So I switched to powerful desktop to run tests at high > speed in greater volume to gain at least one more digit of preciseness. > > And may be you missed it - I actually provided comparison numbers. See below. > If you want me to measure also old code (2 days old) on same box - I'll do > that. > > > > First of all, let me remind you about is_integer bug. It doesn't allow > > > negative numbers. Correct code is: > > > is_integer=re.compile('-?[\d]+).match > > > This is not a bug. Since ID values cannot be negative. > > Then it is a bug in name. It should be is_id_int or is_non_negative_int. > > Here are the figures for the code in trunk > > > > 1) Upstream sql.py > > > model 0.01646 > > > min 0.02979 > > > avg 0.03267 > > > max 0.03597 > > Here are the figures for the first two patches. They give about 8% speedup. > > > > 2) Optimised SQLTable.__init__ and SQLField.__init__ (patches > > > attached, backwards compartible, though you may want to remove my > > > testing marks) > > > model 0.01380 > > > min 0.02697 > > > avg 0.03003 > > > max 0.03189 > > > So - about 8% speedup (by avg time) here. > > Here are the figures for the last patch. Uncertain 1-5% speedup. > > > > > > 3) Now lazy tables (patch attached). > > > Since I expect that you will not find any objections to my patches for > > > (2) here are cumulative results, i.e. optimised __init__s + lazy > > > tables > > > ... > > > hmmm. It doesn't improves results significantly anymore. It looks like > > > these patches are interchangeable. I've run my test three times, > > > results are: > > > > model 0.01009 > > > min 0.02555 > > > avg 0.0297105 > > > max 0.033360 > > > > model 0.01012 > > > min 0.02369 > > > avg 0.02983 > > > max 0.03340 > > > > model 0.00966 > > > min 0.02415 > > > avg 0.02850 > > > max 0.03208 > > > > So it makes some improvement about 1%-5% but somehow it also makes > > > testing results more disperse. Do not know if it worths including. > > > Massimo, your opinion? > > > > 4) Also I've tried to further speedup validator generation w/o losing > > > backwards compartibility, but improvement was about 1% and below the > > > margin of error (even less noticeable than with lazy tables). > > > > On Sun, Jun 7, 2009 at 11:45 PM, Alexey Nezhdanov<[email protected]> > wrote: > > > > On Sunday 07 June 2009 18:41:30 mdipierro wrote: > > > >> I implemented a backward compatible speedup of the default validators. > > > >> I also realized that those many calls to is_integer could be avoided. > > > >> I think I fixed it. > > > > > 0.0429/0.0844 > > > > Somehow consistency is not kept across reboots :( > > > > ... about a hour later: and actually now I get about 10% noise so it > > > > become very hard to make any good measurements. I'll try tomorrow on > > > > better hardware. In the meanwhile, I'll try to profile it once again. > > > > ... > > > > ok, there are still two perfomance hogs left: SQLTable.__init__ (about > > > > 11%) and SQLField.__init__ (about 6,5%). These percentages are 'self' > > > > times - i.e. it was spent directly in lines 1099-1149, 1693-1752. > > > > Python profiling doesn't allow to attribute CPU consumption to > > > > individual lines so for now I don't have any more precise pointers. But > > > > that could be split into functional blocks and narrowed further down. > > > > > In any case - we collected almost all low-hanging fruits. Any further > > > > optimisation will give gains of range 2-5%, not times anymore. > > > > > Except lazy tables which still may give tens of percents. BTW - > > > > Massimo, you said something about SQLLazyTable class. You should > > > > implement SQLLazyField too then. Or may be just wait a day or two until > > > > I do deeper profiling. > > > > >> Please check out the latest trunk. > > > > >> Massimo > > > > > P.S. Here is small fix to is_integer. We forgot about negative numbers: > > > > Also this version is tiny bit faster > > > > is_integer=re.compile('-?[\d]+).match > > > > > -- > > > > Sincerely yours > > > > Alexey Nezhdanov > > > > x_table.diff > > > 6KViewDownload > > > > x_field.diff > > > 2KViewDownload > > > > lazy_define_table2.diff > > > 2KViewDownload > > -- > Sincerely yours > Alexey Nezhdanov --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py Web Framework" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
