Good morning!
I hope I'm not tipping any sacred cows here, but for simple SqlA
objects, it seems to be a heckuva lot faster to just insert into the
table directly (using SqlA expression language) than to insert the
objects via session.flush(). In the attached tests, I'm observing a
10x -20x speedup. I'm still new to SqlA, so I was hoping the list
here would be kind enough to verify my observation.
My questions:
1. If so, why? I assume it's because session_flush() does seperate
insert statments (as verified when echo = True is on).
2. In test 3, is this a reasonable away to "convert" from "session"
to direct table insert? Is there a simpler way than the Thing.to_dict
method I hacked together.
3. Are these valid tests? I don't want to have all the embarrassment
of some others who have 'slammed' SqlA without a proper grounding.
I'm no expert, and I want to make sure what I have is something
approximating idiomatic SqlA. I tried to be generous about what to
include in the timed section of each test. I do have autoflush off,
and I'm using Sqlite (in memory), which might affect things.
4. If there is a faster way to flush out a session, I'm all ears! I
understand the Big Win (tm) of the ORM is programmer simplicity and
power, but if I can get that without major hits to performance, I'd
like to be able to Be Greedy (tm) and have it all.
-----------------
#!/usr/bin/env python2
import sys
import itertools
import time
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, Column, Integer, String, MetaData,
ForeignKey, Boolean
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class Thing(Base):
__tablename__ = 'asn_change'
id = Column(Integer, primary_key=True, nullable=False)
datum = Column(String, primary_key=True, nullable=False)
start = Column(Integer, primary_key=True, nullable=False)
stop = Column(Integer, primary_key=True, nullable=False)
def to_dict(self):
return dict(id=self.id, datum=self.datum, start=self.start,
stop=self.start)
def dummy_setup(connstring='sqlite:///:memory:'):
engine = create_engine(connstring, echo=False)
Session = sessionmaker(bind=engine, autoflush=False, autocommit=False)
session = Session()
Base.metadata.bind = engine
Base.metadata.create_all()
return session, engine
def clean_up(session):
session.expunge_all()
session.query(Thing).delete()
session.commit()
## Tests
class Tests(object):
def setUp(self):
self.session, self.engine = dummy_setup()
self.R = 10000
def test_01_orm(self):
session = self.session
clean_up(session)
for ii in xrange(self.R):
session.add(Thing(id=ii,datum="some data %i" %ii,
start=0,stop=999999999))
now = time.time()
session.flush()
session.commit()
t = time.time() - now
print "timing: %f2.2" % t
assert True
def test_02_direct_insert(self):
session = self.session
clean_up(session)
stm = Thing.__table__.insert().compile()
ipfx = [dict(id=ii,datum="some data %i"
%ii,start=0,stop=999999999) for ii in xrange(self.R)]
now = time.time()
stm.execute(ipfx)
t = time.time() - now
print "timing: %f2.2" % t
assert True
def test_03_convert_from_session(self):
session = self.session
clean_up(session)
stm = Thing.__table__.insert().compile()
for ii in xrange(self.R):
session.add(Thing(id=ii,datum="some data %i" %ii,
start=0,stop=999999999))
# count the conversion time as part of the test
now = time.time()
ipfx = [x.to_dict() for x in session.new]
stm.execute( ipfx)
session.new.clear()
t = time.time() - now
print "timing: %f2.2" % t
assert True
-----
$ nosetests -v -s insertion_test.py
insertion_test.Tests.test_01_orm ...
timing: 8.2513552.2
ok
insertion_test.Tests.test_02_direct_insert ... timing: 0.5210562.2
ok
insertion_test.Tests.test_03_convert_from_session ... timing: 0.8730292.2
ok
----------------------------------------------------------------------
Ran 3 tests in 14.247s
OK
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
