On Feb 26, 2006, at 4:33 PM, Shaun Cutts wrote:
Thanks Gary!
Ah -- very nice: so Data.fs *is* a transaction log. In theory an RDBMS
with write ahead logging is still more secure because the transaction
log is only backup, and the rest of the database is another copy of
the
current state (though not with undo capability).
But with replication, this issue is taken care of. (Too bad
replication
isn't part of the core functionality....)
Some people use ZRS, some people have strategies with
DirectoryStorage, some people use repozo as described in the first
link I sent, some are exploring other options like PGStorage.
Also nice is
http://www.python.org/workshops/2000-01/proceedings/papers/fulton/
zodb3.
html#pgfId=294502
Section 3.1... so ZODB is effectively doing MVCCS and with per-object
locks to resolve conflicts.
That paper is old: the ZODB is doing MVCC now with full views of the
database at the time of transaction start. There's a doc in the wiki
describing it.
(Question: can one explicitly lock an object
without changing it? I guess just setting _p_changed?)
That will mark it as changed whether or not it was, yes. I'm pretty
sure (but notice caveat) that this will "dirty" the object, as far as
write conflicts are concerned, whether or not the object actually
changed.
Are there any benchmarks available?
I believe there is a ZODB bench somewhere. I don't know much about it.
We can't abandon Postgres entirely:
1) we have custom aggregate statistical functions in C
2) we have to allow third-party ODBC access to certain views
3) general lack of query language potentially problematic for
datamining
Especially for third parties (non-Zope/ZODB experts). Two
"howevers": first, I'm led to believe that datamining generally
happens externally from apps anyway, so the Postgres slave idea (that
you have below) would work quite well. Second, even in a ZODB app, as
with an SQL app, if you know where the data resides, know the
available indexes, know how to build and populate new indexes, and
know how to intersect and union results, you can do just about
anything you want. The difference is just that "everyone" knows SQL
spelling for that stuff, and much fewer know the nitty-gritty of
spelling that with Zope 3 indexes.
But 1)-2)-3) for us are "read-only" needs, so in theory, with
replication, we could use Postgres as a slave to ZODB master.
Yes, I've considered an architecture like that recently myself for
some projects. Other approaches are to use the Zope database
adapters (which handle the transaction machinery), and then write
simple wrappers that produce throw-away, non-persistent objects that
persist the data in Postgres. Another would be to monetarily support
someone like Shane to see if a solution like the PG storage will help.
I would not encourage (and, perhaps too gently, have not encouraged)
someone without either a lot of ZODB knowledge or a lot of time and
energy to become a very deep ZODB expert to pursue the __getstate__
__setstate__ approach you showed. It's an interesting idea, but you
are really bypassing huge chunks of the ZODB machinery, probably to
your loss. Much safer to deal with the Zope DBA stuff (persistent
data in transient objects) or, if you are an expert or want to be
one, with an approach like Shane's.
Again, benchmarks would be nice. We haven't yet speced out, let alone
bought, the hardware for our production system, so I couldn't yet say
how high the bar is.
I don't have these, and I'm not even sure exactly what you want.
Gary
_______________________________________________
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users
