Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-29 Thread Hans-Peter Jansen
Dear Jonathan, thank you for your cool remix of recommendations. Very appreciated. On Donnerstag, 26. Juni 2014 15:09:03 Jonathan Vanasco wrote: In case this helps... This reminds me slightly of some RFID work I did years ago. We had a lot of reads coming in from different units, several

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Mike Bayer
On 6/26/14, 11:49 AM, Hans-Peter Jansen wrote: Dear SQLAchemistas, I'm suffering from a performance problem from a simple sequence like this: rec is a sensor record coming redundantly from the network, enos is a sensor device, ev is a sensor value record: def store_enos_rec(self,

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Mike Bayer
On 6/26/14, 12:24 PM, Mike Bayer wrote: I'm not sure what kind of application this is but I would not be mixing transactional control, that is the commit/rollback, inside of a business method that only seeks to create some new objects. There'd be a containing pattern within which

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Mike Bayer
On 6/26/14, 3:07 PM, Hans-Peter Jansen wrote: Okay, attached is a profile dump with .commit() out of the way. Here's the head of it: -- p = pstats.Stats('srelay.pstats') -- p.strip_dirs().sort_stats('cumulative').print_stats(100) Thu Jun 26 20:41:50 2014 srelay.pstats 55993702 function calls

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Hans-Peter Jansen
Dear Mike, sorry for not coping with preferred reply behavior.. On Donnerstag, 26. Juni 2014 15:26:02 Mike Bayer wrote: On 6/26/14, 3:07 PM, Hans-Peter Jansen wrote: Obviously, some operation triggers the flush method with about the same consequences.. OK, turn off autoflush - either

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Mike Bayer
On 6/26/14, 5:18 PM, Hans-Peter Jansen wrote: Dear Mike, sorry for not coping with preferred reply behavior.. On Donnerstag, 26. Juni 2014 15:26:02 Mike Bayer wrote: On 6/26/14, 3:07 PM, Hans-Peter Jansen wrote: Obviously, some operation triggers the flush method with about the same

Re: [sqlalchemy] Performance problem of simple orm usage

2014-06-26 Thread Jonathan Vanasco
In case this helps... This reminds me slightly of some RFID work I did years ago. We had a lot of reads coming in from different units, several reads per unit per second. I found the best way to handle writing was to just access the db directly, but kept the ORM on the read side. I recall a