Gianugo Rabellino wrote:
> Murray Altheim wrote:
>
>> A good reason to not include a metadata implementation in the main
>> code base is that there are quite a lot of approaches to metadata
>> storage, and to the types and complexities of metadata itself, and
>> these vary depending on application. I developed the XNode API as
>> an attempt at creating a *very* lightweight API. If there are three
>> approaches, ordered in terms of complexity, we'd have
>>
>> 1. XNode API
>> 2. (Dave Viner's) Metadata implementation
>> 3. JMI-in-Xindice
>
> I think that these are all way of *accessing* metadata, where I agree
> that we have a pluggability spot. My concern, instead, is about low end
> metadata storage which is totally missing and badly needed. Once we have
> a flexible storage (and IMHO Dave's solution is KISS and effective on
> that point, but it might evolve into something more powerful), we might
> be able to talk about different (and yes, pluggable) ways of accessing
> the metadata.
No, I think all three are potentially a mix of both how metadata is
accessed and how it is stored. There seems to be two methods for
storage:
1. within the database "record" itself, ie., as part of the stored
content of an XML node,
2. as a separate XML document
We previously discussed the pros and cons of each, but I'll reiterate
briefly at least the designs:
1. The XNode API is essentially like a SOAP wrapper on an XML node, a
"super-document" wrapper in its own namespace, with the ability to
then store simple name-value metadata as child elements of the wrapper's
document element. It looks basically like this:
<xnode:Envelope>
<xnode:Header>
[stored metadata]
</xnode:Header>
<xnode:Body>
[stored XML node]
</xnode:Body>
</xnode:Envelope>
(there's an API method to retrieve the original DOM node, so the
API is effectively transparent.)
2. From my understanding of Dave's Metadata implementation, the metadata
is stored in a separate record. For each database record, one or more
metadata records can be associated via a linking mechanism.
3. As for JMI-in-Xindice, since to my knowledge nobody has attempted
an implementation of it within Xindice, it could in theory be
implemented either way (ie., as #1 or #2).
While you in another message state that databases are all about both
data and metadata, that's not necessarily the case. Xindice could be
released out the door without metadata built-in. Given a choice between
an inappropriate metadata solution and none, some may choose none. If
the solution provided is a series of optional plugins, appropriate to
different applications, people may then choose the solution that best
fits their needs. For example, if *neither* #1 nor #2 solutions are
appropriate, say, somebody *requires* a JMI-compliant solution, then
even if we don't provide it they could build it themselves (so much
the better of course if we provided it). Someone else may want an
RDF-based solution, for use with RDF systems. Et cetera.
>> If we make the Xindice addons truly modular, then any of the proposed
>> metadata solutions as well as other non-essential features can be
>> easily added to a Xindice application without enlarging the core code
>> and complicating the lives of those who don't want or need metadata.
>> I like the idea of a trim Xindice rather than including everything,
>> and given this is a public process I'd hate to see it bloat out like
>> some things.
>
> There is a dark side in everything. In this case, I think that there is
> a balance between flexibility and performance, and a risk of exchanging
> flexibility and Flexibility Syndrome. As I said in a previous post, I
> can hardly imagine a common "internal" API that allows for pluggable
> stuff deeply inside Xindice core. We might hit a very hard wall when we
> want to push performance while being stuck to a particular API (as it is
> now, where all the pluggable parts have to go trough DOMs floating all
> over the place which is, to say the least, suboptimal).
>
> Finding that balance is hard: I completely agree on modular
> architectures and the like, but there is a point where you have to stop.
> The risk is coming out whith the dreaded "Object doSomething(Object
> withThis)".
I agree that extremes are to be avoided. But I don't see that wall
being hit very soon. I was (for example) able to create a simple
metadata API without messing *at all* with the internals of Xindice,
and I don't class myself as any sort of remarkable programmer.
>> In your design, Dave, couldn't the entire metadata tree be under
>> its own package rather than having MetaSystemCollection be in
>> package org.apache.xindice.core? (I do understand you've probably
>> designed it as part of core.)
>
> Not until we agree on how to manage an internal API. The only way I can
> see that is working on a set of Listener interfaces, that react to
> changes to the database (maybe the DBObserver can be a good place to
> start). But it won't be an easy task, even if definitely worth the
> effort, since it would allow for triggers, monitors, and any other
> dynamic behaviour on database changes.
My fear about things such as that is performance. Listeners take
processing time, and the larger the database, the more complex the
metadata, the more complex the documents, well, you get the picture.
> But we are already talking about Xindice 2.0, I guess. And we need (or
> at least I do, and all the people I talked to about Xindice do) some
> metadata support now. So what is the best solution?
Is there a way to build it in as a module, or at least not as core?
As I said, I'm no genius and XNode doesn't mess with core, ie.,
people can ignore it and it has no ramifications upon processing
speed, etc. If *at this point* we made that same requirement on
any other metadata solution, could that work? I'd hate to commit
this early in the communications/designs about an internal API
that we'd then be stuck supporting. Look at some of the really bad
decisions made in the DOM design that should be changed now, but
can't due to installed base. Yuck -- something to avoid.
And I suppose I should say that I don't consider myself an expert
on this subject, nor one with a huge feeling of ownership of this
project. These are just my opinions. I prefer Xindice as a lighter
rather than heavier weight application. It's easy to bloat, harder
to trim (ask any writer...).
Murray
......................................................................
Murray Altheim <http://kmi.open.ac.uk/people/murray/>
Knowledge Media Institute
The Open University, Milton Keynes, Bucks, MK7 6AA, UK
If you're the first person in a new territory,
you're likely to get shot at.
-- ma
