To me, it seems that right now it's really a more specialized application level concept. I think the database should track metadata associated with the system, but application metadata should be layered on top. After all we're talking about an XML database here, metadata is supposed to be inherent in the format.
Yes and no. Think about RDBMS, where you have a poor set of system-level metadata (basically all you get is the column type). The solution for developers has always been to add fields (growing data complexity) or tables (growing logic complexity with JOINs) just for metadata. Yet on a RDBMS that was and still is the way to go, since it's trivial to extend the system in this way.
Now what can a developer do on Xindice without a generic metadata support? I see just two solutions:
1. Create a parallel document with a sensible name (if you have mydoc.xml then you might want to create .mydoc.xml or mydoc.mxml or whatever you want). This is a trick and a sordid hack :) subject to name collision to say the least;
2. add metadata inside the document, as you suggested. Yet this would increase complexity *a lot* from an application point of view.
I'm not saying that metadata shouldn't belong to the document, actually I was preparing a first draft where documents had a "xindice" namespace added automatically and an <xindice:metadata/> section when stored on the filer. The difference is that I'd rather have a generic API to work specifically with metadata and be neutral about physical metadata location. This means that Xindice might:
1. when storing a document (Collection.storeResource()): add the metadata section, if not provided in the document, or update/fill it if otherwise.
2. when retrieving a document (Resource.getContent()): send back the document alone, stripping the metadata section;
2,1/2. have the possibility to ask the full document, including metadata (ugly: Resource.getFullContent());
3. when asked for metadata have two ways of accessing them: as the full set (say Resource.getMetadata()) or as an XPath result (Resource.getMetadata("/last-modified")). Hmmm... to make it even easier I wonder if it might be the case to steal some ideas from Avalon's Configuration framework to give some kind of "direct access" to medata elements.
In this scenario it's just an implementation specific decision to understand where should the metadata be physically placed in the storage. Yet there are at least two possible problems with storing them inside the document:
1. Collection metadata (I think we badly need them too).
2. Binary resources. (currently unsupported). True, you can wrap any binary content in an XML structure, but I don't really know if this is the best solution.
This makes me wonder if the real place for document metadata can be the collection itself (well, this is how the Unix filesystem behaves after all). How about it?
Ciao,
-- Gianugo Rabellino
