You're on the right track. The reflection methods are always called with a Connection that is not shared among any other thread (connections aren't considered to be threadsafe in any case) so threadsafety is not a concern.
I think you should look at the mysql dialect sooner rather than later, as its going to present a challenge to the overall notion here. It has a much more elaborate design already and you're going to want to cache the results of SHOW CREATE TABLE somewhere, and that would be better to be *not* cached on the Connection itself, since that is something which can change over the lifespan of a single connection. I think for the PG _get_table_oid method that needs similar treatment, i.e. that its called only once per reflection operation. We have a nice decorator that can be used in some cases called @connection_memoize(), but for per-table information I think caching info on the Connection is too long-lived. I had the notion that the Inspector() object itself might be passed to the more granular reflection methods, where each method could store whatever state it needed into a dictionary like inspector.info_cache - or, perhaps we just pass a dictionary to each method that isn't necessarily tied to anything. This because I think each dialect has a very different way of piecing together the various elements of Table and Column objects, we'd like to issue as little SQL as possible, and we are looking to build a fine-grained interface. The methods can therefore do whatever queries they need and store any kind of structure per table in the cache, and then retrieve the information when the new Dialect methods are called. This means the call for foreign_keys, primary_keys, and columns when using the mysql dialect would issue one SHOW CREATE TABLE, cache all the data, and return what's requested for each call. With the caching approach, multiple queries with the same reflection.Inspector object can be made to work nearly as efficiently as the coarse-grained reflecttable() methods do now. On Nov 8, 2008, at 6:31 PM, Randall Smith wrote: > Michael Bayer wrote: > >> The structure of the API would drive the current reflection API to >> become more componentized. What we see as a need on the "public >> refleciton API" side would drive the currently monolithic >> "reflection" >> methods to be changed. The current reflecttable() methods in turn >> would ideally move *out* of the dialects, and the >> Table(autoload=True) >> functionality would simply call up an Inspector and use that. So the >> ultimate outcome is that the individual "reflecttable" methods go >> away, and dialects implement a larger set of simpler methods. >> > > I started by factoring out components for reflecttable. So now there > are methods get_columns, get_primary_keys, get_foreign_keys, > get_indexes > and get_views. get_columns, get_primary_keys, and get_foreign_keys > are > the methods used by reflecttable (and the ones I actually > implemented). > > There is one thing I did that will slightly degrade performance. I > created a method in postgres.py __get_table_oid that several other > methods use. This value could be cached at the beginning of > reflecttable, but I didn't won't to address threading issues. > > I made a local branch called reflection and have attached the files > and > diffs for engine/base.py and databases/postgres.py at revision > 5262. I > did a few real world tests, but not the automated tests. I don't know > how difficult they are to set up. That'll be the next thing I do. > Mainly I just want to get feedback on the API and if I'm going about > the > implementation right. > > --Randall > > > > <base.py.diff.gz><base.py.gz><postgres.py.diff.gz><postgres.py.gz> --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
