From: [email protected] [mailto:[email protected]] On
Behalf Of Michael Bayer
Sent: Wednesday, February 10, 2010 2:38 PM
To: [email protected]
Subject: Re: [sqlalchemy] Warnings take a really long time / NotImplementedError
On Feb 10, 2010, at 3:28 PM, Jeff Peterson wrote:
First spin though, I get these errors/warnings:
/home/zope/.buildout/eggs/SQLAlchemy-0.6-py2.5.egg/sqlalchemy/engine/reflection.py:40:
SAWarning: Did not recognize type 'ROWID' of column 'objid'
ret = fn(self, con, *args, **kw)
/home/zope/.buildout/eggs/SQLAlchemy-0.6-py2.5.egg/sqlalchemy/engine/reflection.py:40:
SAWarning: Did not recognize type 'LONG RAW' of column 'data'
ret = fn(self, con, *args, **kw)
/home/zope/.buildout/eggs/SQLAlchemy-0.6-py2.5.egg/sqlalchemy/engine/reflection.py:40:
SAWarning: Did not recognize type 'ROWID' of column 'source_rowid'
ret = fn(self, con, *args, **kw)
these are oracle column types that aren't present in the reflected types list.
this error is harmless (assuming you don't issue CREATE TABLE like you're
doing later).
File
"/home/zope/.buildout/eggs/megrok.rdb-0.10-py2.5.egg/megrok/rdb/setup.py", line
93, in createTables
metadata.create_all(engine)
NotImplementedError: Can't generate DDL for the null type
this is more of a problem. you're reflecting views (and I assume table
objects) from your database, and then emitting metadata.create_all() - the
views you've reflected are assumed to be tables, which don't exist, and it
attempts to issue CREATE TABLE for them, and fails due to the types missing
above (but luckily, else it would generate a new table for every view).
You shouldn't be calling create_all(). Especially not in a web application
when it starts up, thats kind of crazy, and also not when your application
receives its table metadata through reflection.
The module calling create_all() is a third party lib for Grok, which clearly
doesn't handle reflection very well as it makes that create call regardless.
It may be I need to step back and try and handle the SQLA stuff on my own.
Is there any way to skip/speed up the unrecognized column warnings?
what evidence do you have that the warnings themselves are slow ? just
because thats what you see actually dumped in your output has no relevance to
the work that is actually going on, in this case, the vast majority of columns
being reflected that do *not* generate any warning, since you would appear to
be reflecting at least 12 views. Reflecting a whole database is not a quick
operation.
I am, in fact, only reflecting one view, but you got me to thinking, that view
has 50+ columns and joins in two other views, does it attempt to reflects all
the tables/view that make up that view?
However, I had made some changes to the lib to allow the reflected views to be
"skipped" basically removing them from the metadata, which worked, the views
were reflected but it still took 30-40 seconds for it to reflect this one view.
The code for this:
class CrarySalesPart(rdb.Model):
rdb.reflected()
rdb.tablename('crary_sales_part')
rdb.tableargs(schema='crar1app', useexisting=True)
contract = Column('contract', String, nullable=False, primary_key=True)
catalog_no = Column('catalog_no', String, nullable=False, primary_key=True)
class Index(grok.View):
grok.context(Portal)
grok.name('testsa.html')
def render(self):
session = rdb.Session()
sp =
session.query(CrarySalesPart).filter(CrarySalesPart.contract=='20').limit(10)
msg = ''.join(['<p style="padding:0;margin:0">%s: %s, %s</p>' %
(o.catalog_no, o.catalog_desc, o.part_product_code) for o in sp])
return """<html><head></head><body>%s</body></html>""" % msg
The first time I render that view, the reflection takes place and it takes the
30-40 seconds to load the page (during which time the warnings are being
generated), once it's mapped it is very fast.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
