On Monday, February 18, 2002, at 04:49 AM, Gianugo Rabellino wrote:
Yes and no. Think about RDBMS, where you have a poor set of system-level metadata (basically all you get is the column type). The solution for developers has always been to add fields (growing data complexity) or tables (growing logic complexity with JOINs) just for metadata. Yet on a RDBMS that was and still is the way to go, since it's trivial to extend the system in this way.
Now what can a developer do on Xindice without a generic metadata support? I see just two solutions:
What if it was something that was stupid simple like automatically creating two collections, one for documents and one for metadata? Each document would have a corresponding metadata document. The metadata doc would have database managed portions and user managed portions, or maybe two separate docs, or even two separate collections for user and system. All updates would be within a transaction across both docs so they can't get out of sync.
That really isn't much different then the data dictionary in an RDBMS, except it's semi-structured and user extensible. I guess you could view it as a data fork, but I'd really prefer not to (Resource forks on the mac are annoying to me, so I'm hiding from the word. :-) ).
The meta-data collection name would just have a reserved prefix/suffix and could be ignored in collection listings.
Maybe It could be an option so that if you don't want the overhead you could create a collection without metadata, probably not really necessary though.
On top of this you can build whatever type of API you want, or simply access the data like any other collection. It wouldn't even require any changes to the XML:DB APi if you layered on top.
It's kind of hacky, but it's simple and in reality it might work decently well. Performance might not be stellar, but there would be room to optimize once the mechanism is worked out. You also leverage all the existing indexing mechanics to enable meta-data queries.
I don't know, just a thought.
1. Create a parallel document with a sensible name (if you have mydoc.xml then you might want to create .mydoc.xml or mydoc.mxml or whatever you want). This is a trick and a sordid hack :) subject to name collision to say the least;
2. add metadata inside the document, as you suggested. Yet this would increase complexity *a lot* from an application point of view.
I'm not saying that metadata shouldn't belong to the document, actually I was preparing a first draft where documents had a "xindice" namespace added automatically and an <xindice:metadata/> section when stored on the filer. The difference is that I'd rather have a generic API to work specifically with metadata and be neutral about physical metadata location. This means that Xindice might:
1. when storing a document (Collection.storeResource()): add the metadata section, if not provided in the document, or update/fill it if otherwise.
2. when retrieving a document (Resource.getContent()): send back the document alone, stripping the metadata section;
2,1/2. have the possibility to ask the full document, including metadata (ugly: Resource.getFullContent());
3. when asked for metadata have two ways of accessing them: as the full set (say Resource.getMetadata()) or as an XPath result (Resource.getMetadata("/last-modified")). Hmmm... to make it even easier I wonder if it might be the case to steal some ideas from Avalon's Configuration framework to give some kind of "direct access" to medata elements.
In this scenario it's just an implementation specific decision to understand where should the metadata be physically placed in the storage.
Yet there are at least two possible problems with storing them inside the document:
1. Collection metadata (I think we badly need them too).
User defined?
2. Binary resources. (currently unsupported). True, you can wrap any binary content in an XML structure, but I don't really know if this is the best solution.
Binary isn't even on the roadmap so I really don't think we should worry about this.
This makes me wonder if the real place for document metadata can be the collection itself (well, this is how the Unix filesystem behaves after all). How about it?
Ciao,
-- Gianugo Rabellino
Kimbro Staken XML Database Software, Consulting and Writing http://www.xmldatabases.org/
