Thanks for your replies.
I've continued investigating my test case, by profiling a part of my test case
(processing only one date, means one extract/build/store cycle). I've put the
results as attachements (I've only kept the first significative lines of
profiling stats files).
While migrating to SA 0.3.1, I had to modify some of my code to integrate the
changes due to the integrated logging facility of SA (not present in the SA
2.8 revision I used before). This gives some clues to investigate in this
way.
The profiling results shows (if I understand correctly) there's a lot of time
spent formatting string which is not displayed (DEBUG, INFO levels), as I
disabled logging message < level WARNING. Particularly, the function
instance_str from the sqlalchemy.orm.util module appears at first place,
while sorting by total time or ncalls. The same for id and hex, which are
called by instance_str. This func is called in several log message (ncalls is
huge).
Thus I tried to remove some of the logs in SA, so no log formatting is done.
Running/timing my whole test gives the following results:
Using 0.2.8
real 2m19.755s
user 1m45.403s
sys 0m2.104s
Using 0.3.1
real 14m28.188s
user 13m3.053s
sys 0m9.161s
=> 6-7 times slower
Using 0.3.1, no log
real 9m8.644s
user 7m50.517s
sys 0m5.824s
=> 3-4 times slower, that's better
Disabling seems to speed up SA, and seems to be part of the "bottleneck". The
other is... hum... I don't have any idea... Maybe you'll have some watching
the profiling results.
Hope it helps. Let me known if you want some other benchmark/profiling
results.
As attachements:
- sprof28: sorted profiling results, using SA 0.2.8
- sprof31: sorted profiling results, using SA 0.3.1
- sprof31_log: sorted profiling results, using SA 0.3.1 after removing
log
(mainly in orm.{util,attributes,mapper,strategies}).
> also, try out the trunk as I just made a major improvement to cascade
> performance yesterday.
OK, I try this now.
> oh, i forgot - theres one thing that could produce significant speed
> changes, which is if youre using any PickleTypes anywhere. the
No PickleTypes used...
Cheers,
--
Sébastien LELONG
[EMAIL PROTECTED]
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
