ah good point ! robert, can you do me a favor .... line 37 and line 53 of objectstore.py is where its creating the identity key, including a call to repr(table). can we change the repr(table) into:
"Table(%d)" % id(table) and also in schema.py, change line 43, inside of SchemaItem.hash_key from repr(self) to: return "%s(%d)" % (self.__class__.__name__, id(self)) then restore the verbose __repr__() change on the Table class and see if your application runs OK ....I dont think the repr() is being used anywhere else. If it all works, and also the three big unit tests select.py, mapper.py, and objectstore.py run OK (theres a few failures in objectstore.py with postgres, thats OK) you can commit that change. thanks ! - mike Robert Leftwich wrote: > My nightly processing failed on the latest version and it appears that the > new > repr() functionality is causing some significant memory/processing > overhead on > large data sets. I know I'm pushing things a little(!!! - its a darn good > stress > test) but in one routine I'm loading all the records in a table (>189,000) > and > SQLAlchemy is choking at this point, maxing out the memory (1GB) before > getting > all the way through mapping these records. This has worked up until now > (taking > only around 3min on my laptop) so I rolled back to the pre-repr() changes > (v800), added in the Postgres version code from 806 (apologies for the > tabs btw) > and everything is working again. The repr() code is being called a lot, as > it > looks to be used in generating the identity key, so it is obviously a > performance sensitive area. Can we retain the repr() benefits w/o > adversely > affecting SQLAlchemy's performance? > > Robert > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > Sqlalchemy-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users > ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click _______________________________________________ Sqlalchemy-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sqlalchemy-users
