On Feb 26, 2006, at 4:33 PM, Shaun Cutts wrote:

Thanks Gary!

Ah -- very nice: so Data.fs *is* a transaction log. In theory an RDBMS
with write ahead logging is still more secure because the transaction
log is only backup, and the rest of the database is another copy of the
current state (though not with undo capability).

But with replication, this issue is taken care of. (Too bad replication
isn't part of the core functionality....)

Some people use ZRS, some people have strategies with DirectoryStorage, some people use repozo as described in the first link I sent, some are exploring other options like PGStorage.

Also nice is
http://www.python.org/workshops/2000-01/proceedings/papers/fulton/ zodb3.
Section 3.1... so ZODB is effectively doing MVCCS and with per-object
locks to resolve conflicts.

That paper is old: the ZODB is doing MVCC now with full views of the database at the time of transaction start. There's a doc in the wiki describing it.

(Question: can one explicitly lock an object
without changing it? I guess just setting _p_changed?)

That will mark it as changed whether or not it was, yes. I'm pretty sure (but notice caveat) that this will "dirty" the object, as far as write conflicts are concerned, whether or not the object actually changed.

Are there any benchmarks available?

I believe there is a ZODB bench somewhere.  I don't know much about it.

We can't abandon Postgres entirely:
  1) we have custom aggregate statistical functions in C
  2) we have to allow third-party ODBC access to certain views
  3) general lack of query language potentially problematic for

Especially for third parties (non-Zope/ZODB experts). Two "howevers": first, I'm led to believe that datamining generally happens externally from apps anyway, so the Postgres slave idea (that you have below) would work quite well. Second, even in a ZODB app, as with an SQL app, if you know where the data resides, know the available indexes, know how to build and populate new indexes, and know how to intersect and union results, you can do just about anything you want. The difference is just that "everyone" knows SQL spelling for that stuff, and much fewer know the nitty-gritty of spelling that with Zope 3 indexes.

But 1)-2)-3) for us are "read-only" needs, so in theory, with
replication, we could use Postgres as a slave to ZODB master.

Yes, I've considered an architecture like that recently myself for some projects. Other approaches are to use the Zope database adapters (which handle the transaction machinery), and then write simple wrappers that produce throw-away, non-persistent objects that persist the data in Postgres. Another would be to monetarily support someone like Shane to see if a solution like the PG storage will help.

I would not encourage (and, perhaps too gently, have not encouraged) someone without either a lot of ZODB knowledge or a lot of time and energy to become a very deep ZODB expert to pursue the __getstate__ __setstate__ approach you showed. It's an interesting idea, but you are really bypassing huge chunks of the ZODB machinery, probably to your loss. Much safer to deal with the Zope DBA stuff (persistent data in transient objects) or, if you are an expert or want to be one, with an approach like Shane's.

Again, benchmarks would be nice. We haven't yet speced out, let alone
bought, the hardware for our production system, so I couldn't yet say
how high the bar is.

I don't have these, and I'm not even sure exactly what you want.

Zope3-users mailing list

Reply via email to