Hi,
no news seem to be good news, let's do it this way then?
robert
Am Donnerstag, den 13.11.2008, 15:05 + schrieb Jens W. Klein:
I would like to request comments on our idea how to use different
storages for our new model-driven approch with the name Devilstick. You
dont need to know devilstick or its ideas in depth to give valuable
input. More it would helps us to get input from people knowing zopes
persistency layer in depth.
Devilstick is model driven framework to describe and manage data inside
and outside ZODB. some more information at http://devilstickproject.net
At Blackforest sprint in august we researched how the goal to support
different storages than ZODB can be achieved. After first thinking about
an own layer we got there the idea of using the usal persistence and
transaction API of zope. IIRC it was a result of a conversation between
Florian Friesdorf and Roger Ineichen and probably others.
Today is the last day of Bolzano sprint. We researched a lot how it
currently works with ZODB and discussed about how to use all this
framework for devilstick.
Our outcome is a document describing what we found and how we want to use
all this. It follows here.
for the devilstick-team
Jens Klein
=
---
Introduction to Devilstick Storages
---
This document describes the future.
One of Devilsticks power is to support different storages than ZODB
easily.
The storage layer uses 100% zopes persistency implementation. At some
entry point we enter the model driven world of devilstick: We hit 'Cage'.
The Cage itself is not a data-access-object (DAO). But its the bridge to
the otherstorage layer. Inside Devilstick DAOs are still persistent
objects. They may still live in ZODB. But they can live complete outside
if it is needed. They may live in SQL-databases, in LDAP, filesystem or
fetched over a webservice.
For more about DAOs and its API please read API.txt.
Excursus: Zopes Persistence Framework
-
Classic zope objects are derived from 'persistence.Persistent'. Those
objects are tracking themselfes for modifications. Once a modification is
detected it joins it's data-manager to the current transaction-manager.
All this happens in zope fully transparent.
The data-manager is the key to the storage layer. Zope is designed to use
different data-managers. Datamanagers are described well in
'transaction.interfaces.IDataManager'. They care about storing all data
in a 2-phase commit. There is usally one data-manager for all modified
object of one database.
Transaction-manager collects all datamanagers (which are called resources
inside the transaction-manager) with modifications. Once the transaction
is committed the 2-phase commit is started: 1st 'tpc_begin' is called on
each data-manager, 2nd the 'commit' is called for each, then 'tpc_vote'
and finally 'tpc_finish'.
After creation of a persistent object it has an attribute called '_p_jar'
set to 'None'. _p_jar gets a datamanager set - almost magically - after
it was added to a container. The datamanager taken there is copied over
from the containers _p_jar attribute. Container and new object are
marked as modifed and the datamanager joins the transaction. On commit
both are written to the database.
Devilstick persistency
--
To provide other storages we alreay have a powerful framework: the
persistent api and transaction api. Devilstick uses both. To use a
different storage simply a new data-manager is needed. Anyway, for
several uses-cases its fine to stay in the ZODB.
Such a alternative datamanager might work different inside than the
current ZODB one. Since we deal with SQL or LDAP we want to update a
database with one query for several objects involved. So on commit we may
need to look at the modified objects and build one sql-query from a bunch
of modifications. Frameworks like SQLAlchemy may help us here for SQL and
others are probably available for different use-cases.
Entry-Points: Cages
---
We need one point where the datamanager is switched to a different
storage.A model is assigned and there the world of generic DAOs is
entered. This entry point is called 'Cage'. A cage is still persistent in
the ZODB and uses the zopes default data-manager. A cage has the root
container DAO (which is a generic molecule DAO) set as an attribute. Here
some example code how it looks like:
cage = Cage()
cage._p_jar
None
somezodbcontainer._p_jar
Connection at ...
somezodbcontainer['data'] = cage
cage._p_jar
Connection at ...
cage._root
None
cage.model = 'examplemodel'
cage._root
Molecule
