As a user of Elephant, you really shouldn't have to worry too much
about threading so long as you follow the simple rules laid out in
the manual under "multi-threading". I think you are trying to
understand how we make this possible since it seems harder from your
read of the acache interface.
A simple conceptual model is that each thread has its own
transaction. If these transactions are executing concurrently, the
BDB library or SQL server logs the side read dependencies and the
side effecting writes until a commit or abort. On abort, you throw
away the log. On a commit, one transaction at a time writes their
side effects and cancels any transaction that has read or written one
of the entries written by the committing transaction.
Thread isolation is guaranteed by a policy for using the BDB library
or SQL server. Calls to these libraries are re-entrant. The current
transaction ID (only used by one thread) determines where the reads
and writes are logged (this is a little more complex for SQL, but
handled by the data store and transparent to the user).
The only other thing we need to do is make sure that the parts of
elephant shared across threads are themselves protected by Lisp
locks. Most of this is the store-controller and some data structures
used to initialize a new store controller.
If you stick to the user contract in the documentation, you shouldn't
have to worry further about interactions of multiple threads (other
than the usual considerations if you have shared lisp variables
instead of shared persistent objects).
Please let me know what would need to be clarified in the
documentation to make this clearer, or ask pointed questions if the
above was not a sufficient explanation. After all, the user is
always right!
Thank you,
Ian
On May 24, 2007, at 7:46 PM, Robert L. Read wrote:
I would have to disagree about the documentation for Elephant not
being abundant---Ian has written a 118 page manual.
Nonetheless, you are correct that the use of Elephant in a multi-
threaded webserver environment is not heavily documented. Ian and
I have discussed the need for a killer "example app" and eagerly
await someone contributing one.
I'm a little rusty on some of this, but if you will agree to take
my opinion about skeptically, I will describe my thoughts on the
subject. My own application, Konsenti, is a multi-threaded web
application.
In the first place, one has to go back to basics a little bit.
Whenever you have concurrency, you have to have concurrency
control. Personally, I think to think of this at the object level,
but I know it is now common to think of it at the "database
level". You are generally correct that if you are using SQL (or
BDB, for that matter) as a database and you keep ALL of your state
in the database, then you can generally rely on the concurrency
control of those systems to serialize everything and not allow two
threads to interfere with each other. However, almost ANY
application will have to think about concurrency; if your are SQL-
orieneted, you will do this by defining "transactions". If you
define your transactions wrong, you will have concurrency errors,
even though the database serializes transactions perfectly.
For example, generally, since the Web forces us into "page based"
or "form based" interaction (which javascript, CSS and Ajax
promptly complicate), one can generally think of a web applications
as "one-page turn == one transaction". But even that might not be
true---you could take 10 page turns to submit an order, and the
order must be atomic---that is, you either order the thing with 10
pages of specifications, or you don't order it. A half-order is a
corrupt order.
Elephant has the "with-tranaction" macro. This is really the best
way to use Elephant in most circumstances --- but even then, if you
are keeping ANYTHING in memory, whether to cache it for speed or
because you don't want to put it in the database (session-based
information would be a typical example), you may still have to
"serialize" access to that state. That unavoidable means that you
have to understand some sort of mutual exclusion lock;
unfortunately, these are not 100% standard across different lisps.
However, most lisps do provide a basic "with-mutex" facility. I
use this in the DCM code (in the contrib directory) to serialize
access to a "director", which is very similar to a version of Ian's
persistent classes, but based on a "keep it all in memory and push
writes to the DB" model (that is, Prevalence.)
So before we can really answer you question deeply, I think we
would have to understand more about what you are doing.
If you will forgive me over-simplifying things, if:
1) You can legitimately think of every page turn as a transaction, and
2) You keep all of the meaningful state in the Elephant DB, and
3) You wrap your basic "process-a-page" function in "with-
transaction",
then you won't have a concurrency control problem.
That is a completely appropriate style for certain relatively
simple web-applications; for other kinds of web-applications it
would be very constraining and slow --- but one should never
optimize for speed before it is necessary.
I don't know if that's a useful answer --- if you can refine your
question, I will try again.
On Thu, 2007-05-24 at 18:08 -0400, [EMAIL PROTECTED] wrote:
Hi all, I'm still on my quest to learn to effectively use
Elephant. Although documentation is not so abundant, I've gotten a
pretty good start with the available documentation. However, I
don't have such a strong background on using ODBs and mainly come
from the SQL world. So, just for curiousity's sake, I read the
tutorial for AllegroCache which tries to show the "proper" way to
use AllegroCache in real-world systems (http://www.franz.com/
products/allegrocache/docs/acachetutorial.pdf). Well, conceptwise,
I followed everything in that tutorial and could pretty much
relate everything to Elephant. However, the truth is that I
thought there was a tremendous "overhead" in using AllegroCache
when dealing with multiple threads. The reality is that the
applications we are thinking on using Elephant for are web-based
applications served by multiple web servers, so the practicality
of the tutorial was more inline to our objective use of Elephant.
So, I figured that both systems (Elephant and AllegroCache) have
more or less the same usage for practical purposes. But, none of
the Elephant documentation I've read on Elephant has even referred
to the things mentioned in the tutorial when dealing with multiple
threads. I have read on the list that Elephant is thread safe, but
am wondering if anyone could help me understand how Elephant would
be different from AllegroCache. For example, do I still need to
handle connection pools and thread locks and all of that in order
to do simple multi-threaded requests? Not that I'm looking for an
easy way out, but coming from the world of simply "using" a SQL-
database and being presented with the needs of having to
incorporate a lot of "logic" that the database server would've
handled for you into your application seems a bit overwhelming at
first. I don't know if it has anything to do with the way
AllegroCache is architected (e.g. things like isolation between
connections, etc) or other factors I'm not aware of. Again, I only
read the tutorial in trying to learn more how to "apply the
concepts" of ODB so I could understand them better. Any comments
would be greatly appreciated. Thanks, Daniel
_______________________________________________ elephant-devel
site list elephant-devel@common-lisp.net http://common-lisp.net/
mailman/listinfo/elephant-devel
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
