We need to get new development kick started here. To start I'm trying to collect all the bits of information/features/ideas/questions into a document for everyone to see. I've posted the start here
http://www.xindice.org/papers/planning.html
3. Database Core 1. Transactions 1. What's the strategy to implement this? 2. Could bootstrap off Berkley DB
We definitely need transactional capabilities. I had some ideas for this early on, but they've oozed out of my head since then. I'll think about this a little more deeply when I get the chance. I share the same concerns regarding Berkley DB, and would prefer to avoid it for now.
2. Better support of different encodings
Absolutely. Once we can get UTF-8 support properly working, then we've taken a good first step. The DTSM stuff that I started to specify would help to address some of these problems as well, but wouldn't be something we should focus on until 2.0.
3. Schema validation in core
I'd still like to define a minimal abstract data model, and then try to fit any of several schema languages onto it, starting with W3C schemas.
4. Focus on roundtripping? 1. How important is this in the real world?
Not sure it's so important... But it depends on how compliant with the W3C XML data model/infoset we want to be.
5. Better support for large documents.
The DTSM spec, combined with streaming capability going in and out of Filers will make this possible, and should be incredibly hard to do. I also want to explore using Java 1.4's new I/O system for more efficient paged IO.
8. Should explore other types of indexing options
We need indexes based on a data model that takes into account that documents may have several different contexts with the same element or attribute name which are actually bound to different types. We can't have an optimally performing schema system without this.
10. We should probably look at including some connectivity to other data sources
1. Maybe a MySQL backend to start.
We should look into possibly merging Ari's DBDOM stuff as a backend.
11. Linking 1. To keep and mature or to eliminate?
I'm for eliminating it.
12. Expanded in database meta-data 1. Need to see exactly what we do and do not have here.
It's something I've wanted. It's hard to implement consistently if we expand out to several backends, but it's workable.
13. Triggers 1. Been planned for a while, probably still worth adding
I agree.
14. XMLObjects 1. Are these really necessary as a separate concept? 2. Especially questionable if we're embedding in Tomcat.
Not so sure. I would still lean toward having them. If we were to incorporate Labrador, XMLObjects would be a free. We'd only have to write enough code to load and instantiate classes, but not much else.
4. Command Line Tools 1. Simpler interface 2. Maybe replace with interactive interpreter and SixDML implementation
I like the idea of an interative SixDML interpreter. Especially for people who might be writing batch scripts and such.
5. Graphical Tools 1. Bring in browser projects as part of the core? 2. Graphical admin tool?
There should be a single admin/browser tool, and I'm leaning toward saying that it should be web-based, rather than a standalone GUI.
4. Should we virtualize things more so that the collection path and any XPath can be treated as one unit?
1. Would allow easy retrieval of bits of XML documents
2. The database would appear logically as one big XML document, while physically being a different structure.
3. Could enable either the collection centric or document centric view to be used.
The problem with this is the same thing that I pointed out in our SixDML discussions. It creates a potential for ambiguity that isn't easily overcome, and will lead to non-deterministic behavior. This type of thing isn't something I'd recommend.
5. Add XML-RPC or SOAP based API
1. Need to determine main API?
1. XML-RPC is very simple and lightweight
2. SOAP provides abilities for alternative encodings to be used. Could result in more efficient in system use.
2. Labrador proposed as possible solution
1. Is this the right way to go?
Of course it is!
3. APIs need to expose more meta data about objects in the database.
6. Client API will evolve with SixDML becoming an XML:DB project and marging portions with the current XML:DB API.
7. Schemas should also be considered within the context of the API.
1. Need to make sure documents with associated schemas and DTDs can be stored cleanly in the database.
Absolutely... also necessary for proper indexing and querying.
8. WebDAV support 1. Either built in or a simple integration with an existing WebDAV impl? 1. Current best route seems existing impl.
Are there any servlet based WebDAV impls that are Apache licensed... maybe something included in Tomcat?
2. Add XQuery
1. Lots'o work - especially with typing
1. Very important regardless
2. Shouldn't be a Xindice only effort.
3. Queries should be able to be applied to a single collection or a collection and all child collections.
4. Deprecate XUpdate?
XQuery is something I spoke to Scott Boag about before he got ill and was unavilable for a month. We should assume that this will be a joint effort between the Xalan and Xindice projects (along with any other projects who need to be involved)
I'm for deprecating XUpdate and possibly implementing one of the proposed update syntaxes for XQuery. We can't expose a query language that only allows you to update using static values (XUpdate). Otherwise, we need to extend XUpdate to support calculated expressions.
9. Server Framework 1. Move to Avalon for most framework services?
Yes... In my experimental Xindice stuff, I've almost completely eliminated all of the Juggernaut code except for Services. These can easily be translated to Avalon blocks.
3. Overall stronger focus on embedding and less on standalone server?
We should try to strike a nice balance between the two. I think this shouldn't be too hard to do based on the work that I've done.
-- Tom Bradford - http://www.tbradford.org Architect - XQRL (XQuery Engine) - http://www.xqrl.com Apache Xindice (Native XML Database) - http://xml.apache.org Project Labrador (Web Services Framework) - http://notdotnet.org
