On Monday 08 June 2009 19:18:26 mdipierro wrote: > I understand what you say below but I think it is important to measure > the total speedup from the original/stable sql.py and the one in trunk > on the same machine and same complex model that you have. NP, it is running right now. Quite slow, will finish in ~10minutes.
Preliminary results (10 sets:) min 0.05579 avg 0.055944 max 0.05611 model time unknown That is against revision 875. > Massimo > > On Jun 8, 9:54 am, Alexey Nezhdanov <[email protected]> wrote: > > On Monday 08 June 2009 17:57:36 mdipierro wrote:> Thanks Alexey, since > > these tests are on a different architecture, can > > > > > you give the total relative speedup on the same architecture for > > > executing your original model (code in trunk vs original code)? > > > > You see, I switched to another architecture because testing it on my > > laptop become unreliable and obtaining more-or-less stable values > > required a lot of time. Powersave and CPU frequency scaling features kick > > in and introduce a great deal of noise. So I switched to powerful desktop > > to run tests at high speed in greater volume to gain at least one more > > digit of preciseness. > > > > And may be you missed it - I actually provided comparison numbers. See > > below. If you want me to measure also old code (2 days old) on same box - > > I'll do that. > > > > > > First of all, let me remind you about is_integer bug. It doesn't > > > > allow negative numbers. Correct code is: > > > > is_integer=re.compile('-?[\d]+).match > > > > > > This is not a bug. Since ID values cannot be negative. > > > > Then it is a bug in name. It should be is_id_int or is_non_negative_int. > > > > Here are the figures for the code in trunk > > > > > > 1) Upstream sql.py > > > > model 0.01646 > > > > min 0.02979 > > > > avg 0.03267 > > > > max 0.03597 > > > > Here are the figures for the first two patches. They give about 8% > > speedup. > > > > > > 2) Optimised SQLTable.__init__ and SQLField.__init__ (patches > > > > attached, backwards compartible, though you may want to remove my > > > > testing marks) > > > > model 0.01380 > > > > min 0.02697 > > > > avg 0.03003 > > > > max 0.03189 > > > > So - about 8% speedup (by avg time) here. > > > > Here are the figures for the last patch. Uncertain 1-5% speedup. > > > > > > 3) Now lazy tables (patch attached). > > > > Since I expect that you will not find any objections to my patches > > > > for (2) here are cumulative results, i.e. optimised __init__s + lazy > > > > tables > > > > ... > > > > hmmm. It doesn't improves results significantly anymore. It looks > > > > like these patches are interchangeable. I've run my test three times, > > > > results are: > > > > > > > > model 0.01009 > > > > min 0.02555 > > > > avg 0.0297105 > > > > max 0.033360 > > > > > > > > model 0.01012 > > > > min 0.02369 > > > > avg 0.02983 > > > > max 0.03340 > > > > > > > > model 0.00966 > > > > min 0.02415 > > > > avg 0.02850 > > > > max 0.03208 > > > > > > > > So it makes some improvement about 1%-5% but somehow it also makes > > > > testing results more disperse. Do not know if it worths including. > > > > Massimo, your opinion? > > > > > > > > 4) Also I've tried to further speedup validator generation w/o losing > > > > backwards compartibility, but improvement was about 1% and below the > > > > margin of error (even less noticeable than with lazy tables). > > > > > > > > On Sun, Jun 7, 2009 at 11:45 PM, Alexey Nezhdanov<[email protected]> > > > > wrote: > > > > > On Sunday 07 June 2009 18:41:30 mdipierro wrote: > > > > >> I implemented a backward compatible speedup of the default > > > > >> validators. I also realized that those many calls to is_integer > > > > >> could be avoided. I think I fixed it. > > > > > > > > > > 0.0429/0.0844 > > > > > Somehow consistency is not kept across reboots :( > > > > > ... about a hour later: and actually now I get about 10% noise so > > > > > it become very hard to make any good measurements. I'll try > > > > > tomorrow on better hardware. In the meanwhile, I'll try to profile > > > > > it once again. ... > > > > > ok, there are still two perfomance hogs left: SQLTable.__init__ > > > > > (about 11%) and SQLField.__init__ (about 6,5%). These percentages > > > > > are 'self' times - i.e. it was spent directly in lines 1099-1149, > > > > > 1693-1752. Python profiling doesn't allow to attribute CPU > > > > > consumption to individual lines so for now I don't have any more > > > > > precise pointers. But that could be split into functional blocks > > > > > and narrowed further down. > > > > > > > > > > In any case - we collected almost all low-hanging fruits. Any > > > > > further optimisation will give gains of range 2-5%, not times > > > > > anymore. > > > > > > > > > > Except lazy tables which still may give tens of percents. BTW - > > > > > Massimo, you said something about SQLLazyTable class. You should > > > > > implement SQLLazyField too then. Or may be just wait a day or two > > > > > until I do deeper profiling. > > > > > > > > > >> Please check out the latest trunk. > > > > >> > > > > >> Massimo > > > > > > > > > > P.S. Here is small fix to is_integer. We forgot about negative > > > > > numbers: Also this version is tiny bit faster > > > > > is_integer=re.compile('-?[\d]+).match > > > > > > > > > > -- > > > > > Sincerely yours > > > > > Alexey Nezhdanov > > > > > > > > x_table.diff > > > > 6KViewDownload > > > > > > > > x_field.diff > > > > 2KViewDownload > > > > > > > > lazy_define_table2.diff > > > > 2KViewDownload > > > > -- > > Sincerely yours > > Alexey Nezhdanov > > -- Sincerely yours Alexey Nezhdanov --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py Web Framework" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
