+1. This description is right on the money on all counts, IMO.
--steve
On Sep 8, 2006, at 3:15 PM, Rafael Schloming wrote:
I think a number of people have suggested that we need a thin
implementation neutral abstraction around the persistence APIs.
This is in fact already there. Have a look at the MessageStore
interface at the following URL:
https://etp.108.redhat.com/source/browse/etp/trunk/blaze/java/
broker/src/org/apache/qpid/server/store/
You can also look at the BDB implementation here:
https://etp.108.redhat.com/source/browse/etp/trunk/blaze/java/
bdbstore/src/org/apache/qpid/server/store/berkeleydb/
If you browse the BDB implementation you can get a good idea of
what's required. At the moment it's quite simple. I think all we
need are the following three tables:
create table queues (
queue_id integer primary key,
queue_name varchar
);
create table messages (
message_id integer primary key,
headers varchar,
body BLOB
);
create table queue_entries (
queue_id integer references queues(queue_id),
message_id integer references messages(message_id),
sort_key integer,
primary key (queue_id, message_id)
);
The only reason I'm tempted to avoid using an O/R layer in this
case is because a) the data model is dirt simple and b) the way
we'll be accessing the database isn't particularly suited to O/R
mapping. We don't have a complicated object model, we don't want to
query objects out of the database or build up object graphs and
then persist them. In fact our data is pretty much read only,
perhaps with the exception of changing queue names.
As I understand it the key thing our RDB backed data store
implemenation will do is to write rows into and read/delete rows
out of the messages and queue_entries tables as fast as possible,
so I suspect what we really want is a row oriented API that is well
suited to batch operations, and IMHO JDBC is probably a fine
starting point for this. I do agree 100% that no SQL statements
should appear in the Java code, but I don't think that requires
ruling out using JDBC directly.
--Rafael
Joyce, Sean (Sam) wrote:
Hi Folks,
I think we do need a *thin* layer around the persistence API's and I
don't think that layer should be JDBC (for Java). My reasoning is
that
while most people will likely be using a real database for the
persistence of messages I can think of several use-cases where
there is
no actual database in use. E.g. an N-safe replicated in-memory
database
with async spooling to hard storage. Or, in the presence of a
reliable,
high-speed, distributed shared file system, a check-pointed local
file
would be sufficient.
I would prefer that we not force people (us) to use JDBC directly
from
within the Qpid code base. I have a very string aversion to hard
coding
SQL statements in code. I've had quite a bit of experience (past
life)
accessing different databases from code and while the standard SQL is
most common databases if almost exactly the same it is still only
*almost* exactly the same :)
So I would propose that the persistence API not be database specific.
I do believe we can achieve this without having to sacrifice
performance
too much, but *any* message persistence is going to affect
performance
is some way wrt transient messages.
Note also, that when we are designing this persistence layer we
need to
be mindful of reason for having a persistence layer at all i.e.
reliability and recoverability in the face of apparent disaster :)
therefore having to take a hit (performance wise) to do a full
recovery
vs. not affecting performance (too much) for the normal error-free
code
path.
Whether we use an o/r mapping layer or not is not something I have
strong feelings about as long as can achieve reliability (via
persistence) without affecting performance on the error-free code
paths.
Given that the data model for the kind of persistence we are
looking at
is not large, I thing we should be able to achieve it with some
ease :)
Regards,
Sam.
-----Original Message-----
From: Alan Conway [mailto:[EMAIL PROTECTED]
Sent: 08 September 2006 15:28
To: [email protected]
Subject: Re: persistence
On Fri, 2006-09-08 at 09:56 -0400, Rajith Attapattu wrote:
I agree with Rafeal, and we need to have a very simple persistant
API.
But then again O/R mappers are in style and end users may prefer
things
like
hibernate, ibatis etc..
And they maybe convinent to work with.
People will definitely want to change the DB backend, but I predict
that
no Qpid user will ever care *how* we get the data into the database,
they will only care:
- Can I use my favorite database X?
- Is it fast?
I'm guessing JDBC is all the pluggability we need, so the only
reason
to
use an O/R mapper is if we think it's a productivity win for us
and it
doesn't have significant performance impact. I don't think the Qpid
persistence schemas are going to be very complex so I'm not sure
a O/R
mapper gives us much, but I don't know the Java ones well enough to
comment further.
'nuff said, back to C++ :)
