Hi,
I've been trying to profile why loading a single table from remote
ORACLE DB takes more then 5 seconds.
Here is some numbers:
create engine 0.0740728378296
engine.connect 2.05604815483
SELECT table_name FROM all_tables WHERE owner='ZZZ'
get tables 0.18466091156
Loading 'triggerpathdescription' table
5.85890698433
Load tables manually ['triggerpathdescription']
load table 5.85950708389
It looks like SQLAlchemy takes 25 fetchone calls in order to autoload
table, but I tweak a code for ORACLE engine to use fetchmany and just
slighly reduced the total time. To me it's WAY to long and as far as I
can tell it internals of building foreign-key relationships. So I
wonder if there is a way to speed up such auto load.
I'm attaching a simple test code below.
Thanks,
Valentin.
#!/usr/bin/env python
import sqlalchemy,types,sys,os,time
def loadTables(tableNames):
for tName in tableNames:
t = tName[0].lower().split(".")[-1]
print "Loading '%s' table"%t
t1=time.time()
tables[t]=sqlalchemy.Table(t,dbMeta,autoload=True,schema=schema,oracle_renyms=True,useexisting=True)
print time.time()-t1
break
print "Load tables manually",tables.keys()
schema = 'ZZZ'
t1 = time.time()
engine = sqlalchemy.create_engine('oracle://
XXX:[EMAIL PROTECTED]',strategy='threadlocal',threaded=True)
print "create engine",time.time()-t1
t1 = time.time()
con = engine.connect()
print "engine.connect",time.time()-t1
t1 = time.time()
tables={}
dbMeta = sqlalchemy.MetaData()
dbMeta.bind = engine
idx = 0
query="SELECT table_name FROM all_tables WHERE owner='%s'"%schema
print query
tableNames=con.execute(query)
print "get tables",time.time()-t1
t1 = time.time()
loadTables(tableNames)
print "load table",time.time()-t1
t1 = time.time()
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
