I have never observed 5 seconds to reflect a single table even with
Oracle which tends to be the slowest for schema-level operations.
The numbers you posted are too coarse grained to indicate where the
time is being spent, and if it were within SA reflection code then
these kinds of speed issues would be apparent across all dialects.
Can you post some "profile" results for your tests ?
On Aug 27, 2008, at 3:57 PM, vkuznet wrote:
>
> Hi,
> I've been trying to profile why loading a single table from remote
> ORACLE DB takes more then 5 seconds.
> Here is some numbers:
>
> create engine 0.0740728378296
> engine.connect 2.05604815483
> SELECT table_name FROM all_tables WHERE owner='ZZZ'
> get tables 0.18466091156
> Loading 'triggerpathdescription' table
> 5.85890698433
> Load tables manually ['triggerpathdescription']
> load table 5.85950708389
>
> It looks like SQLAlchemy takes 25 fetchone calls in order to autoload
> table, but I tweak a code for ORACLE engine to use fetchmany and just
> slighly reduced the total time. To me it's WAY to long and as far as I
> can tell it internals of building foreign-key relationships. So I
> wonder if there is a way to speed up such auto load.
>
> I'm attaching a simple test code below.
> Thanks,
> Valentin.
>
> #!/usr/bin/env python
>
> import sqlalchemy,types,sys,os,time
>
> def loadTables(tableNames):
> for tName in tableNames:
> t = tName[0].lower().split(".")[-1]
> print "Loading '%s' table"%t
> t1=time.time()
>
> tables
> [t
> ]=
> sqlalchemy
> .Table
> (t
> ,dbMeta
> ,autoload=True,schema=schema,oracle_renyms=True,useexisting=True)
> print time.time()-t1
> break
>
> print "Load tables manually",tables.keys()
>
> schema = 'ZZZ'
> t1 = time.time()
> engine = sqlalchemy.create_engine('oracle://
> XXX:[EMAIL PROTECTED]',strategy='threadlocal',threaded=True)
> print "create engine",time.time()-t1
> t1 = time.time()
>
> con = engine.connect()
>
> print "engine.connect",time.time()-t1
> t1 = time.time()
> tables={}
> dbMeta = sqlalchemy.MetaData()
> dbMeta.bind = engine
>
> idx = 0
> query="SELECT table_name FROM all_tables WHERE owner='%s'"%schema
> print query
> tableNames=con.execute(query)
> print "get tables",time.time()-t1
> t1 = time.time()
>
> loadTables(tableNames)
>
> print "load table",time.time()-t1
> t1 = time.time()
>
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
