Another thought on this topic is that with a sufficiently efficient
query system, some of the indices you describe should become
unnecessary. If you have messages indexed by time than you can simply
walk the index and filter by user until you have a web page worth of
messages. A scan, even of an index, may not be fast enough so you
could do an intersection of user-to values with an ordered list of
recent messages. This should perform similarly to a SQL engine which
many ORM systems use for queries like this.
Nothing prevents you from making custom indices when you have to speed
things up, of course!
Ian
On Mar 23, 2008, at 11:12 AM, Alex Mizrahi wrote:
IE> :index t is not necessary - in fact it is ignored. :slot-
deps
are
also not required, but the derived index is updated on any slot
write
if that slot is not transient, set-valued or an association.
We can
add those last three slot types into the mix if necessary, but
I'm
trying to avoid too much complex computation taking place
during slot
writes (self-deadlock, etc) for the time being.
seems to be fine..
actually we are using derived indices in quite special way -- to get
index
that is ordered in special way, to do efficient lookups of some kind.
suppose we have a messaging system with two persistent classes --
user and
message:
(defpclass user ()
((username :index t)
...)
(defpclass message ()
((from-user :accessor from-user :index t)
(to-user :accessor to-user :index t)
(text :accessor text-of :index t)
(modification-time :accessor modification-time-of :index t)))
suppose we'd like to get inbox and outbox views for user, i.e. list
of 10
latest messages to user or from user.
with considerable amounts of users and messages it is not efficient
to get
these latest messages from any of normal indices, as it requires
scanning
potentially large number of messages.
it's possible to make efficient lookups via derived indices --
messages
ordered first by user, then my modification time. iirc cons sorting in
elephant has desired characteristics, but unfortunately postmodern
backend
does not support complex type sorting.
but we happily can reduce problem to sorting strings, e.g.
"13_31321433"
where 13 is oid of from-user and 31321433 is modification time
(universal
time). (actually we have user-id field instead of oid to preserve
identity
across multiple stores etc).
thus, via two derived indices we can efficiently list messages for
inbox and
outbox. (via cursor operations, that's not very easy, but works fine).
as i understant you're going to make elephant more high level, and
possibly
such low level index operations can be replaced with some high level
concept.
are you planning something like these dual indices?
if there will be no special functionality for these, it would make
sense to
make derived indices somewhat more flexible. for example, i suspect
some
uses might require "foreign slot dependency".
like if user definition above had group field, and we'd like to build
messages-by-group index. when user-group changes, all messages of
this user
should update derived index.
probably there's no need in this stuff right now, but it would be
nice if it
will be possible to add it in future, so advanced indices can be built
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
