Hi,
I have completed a big reorganization of the pygr.Data code that
addresses the concerns we've discussed in this thread. I am proposing
to include this in the forthcoming pygr 0.8 release, and would like to
get some feedback on this reorganization in our developer conference
call Thursday.
This is a reorganization, not a rewrite. While many pieces of code
have been moved around or renamed to organize them better, the
important functional pieces are largely unchanged, and I'm hoping that
relatively few bugs have been introduced in the process. I had no
trouble getting this working quickly, and all the tests pass (plus new
tests that I've added). This has also eliminated the many annoying
problems we used to have with testing pygr.Data.
Summary of this reorganization:
- the core functionality is now called a "metabase", to emphasize that
pygr.Data is not a *database* but a *metadata database*, i.e. that it
stores metadata (such as schema) describing data that are stored
*elsewhere*. I felt that we needed a new term to clarify what this is
all about; the various terms I found myself using previously
("resource database", "pygr.Data" etc.) seemed inadequate.
- Almost all of the code is now in a new module pygr.metabase, which
acts like a normal Python module (importing it does nothing but gain
access to its classes and functions, without loading any data).
- PYGRDATAPATH is not actually loaded until you try to use data from
it (or save data to it).
- you can change the PYGRDATAPATH at any time using the new method
update(). You can clear the cache of loaded resources at any time
using the new method clear_cache().
- you can create new metabase connections whenever you want, via
mdb = MetabaseList() # use the default PYGRDATAPATH
or
mdb = MetabaseList(somePath) # use somePath instead
Each such connection is independent of any others.
To see how this works in practice, look at the modified test file
metabase_test.py
- The module pygr.Data now contains only a few stubs for continuing to
support the old (pygr.Data 0.7) access methods, which are now
deprecated. You can still "import pygr.Data", but that is no longer
the recommended access method.
- I propose "from pygr import pygrData" as the new easy-access
method. This is an object instead of a module, so there are no
limitations on top-level names (pygrData.Physics,
pygrData.Architecture, etc.), which are searched for only when an
actual user request is received (rather than during module import as
in pygr 0.7). To see how this works in practice, see the modified
test file pygrdata2_test.py
- the not-very-useful concept of "layers" has been replaced by
"zones", a dictionary of zone names with their associated metabases.
The new code is in a new branch called metabase, which you can get
from my github repository, e.g.
git check -b metabase # create a new branch called metabase in your
repository
git pull git://github.com/cjlee112/pygr.git metabase # pull from my
repository branch
Some details:
- the very large class ResourceFinder has been split into a series of
distinct classes:
* Metabase: a single metabase, associated with a single "storage"
backend
* MetabaseList: a list of metabases (e.g. initialized from a path
string like PYGRDATAPATH). This is what users will mainly be
interacting with, at least in beginner-usage patterns.
Both follow the same interface for reading / writing, mostly inherited
from their base class, MetabaseBase.
* ResourceSaver: associated with a single Metabase; stores
resources queued for saving to that Metabase prior to commit or
rollback. Formalizes the principle that any write operation is tied
to a single, specific metabase target.
* ResourceCache: central cache of all currently loaded resources.
* ResourceServer: for creating a new XMLRPC server (replaces method
newServer)
- our three available flavors of backend storage have been renamed
ShelveMetabase, MySQLMetabase and XMLRPCMetabase. Each metabase
instance connects to an instance of one of these.
- there are now three flavors of pygrdata_test.py
* pygrdata_test.py: tests the old-style pygr.Data access methods
(i.e. using import pygr.Data)
* metabase_test.py: performs the same tests but using the metabase
module directly
* pygrdata2_test.py: performs the same tests but using the proposed
new easy-access method (from pygr import pygrData)
-- Chris
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pygr-dev" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/pygr-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
