Gentlemen, While maintaining my bookmarks I ran into this: "Case Study: Enabling Low-Cost XML-Aware Searching Capable of Complex Querying": http://www.idealliance.org/papers/xmle02/dx_xmle02/papers/03-02-08/03-02-08.html
Some loose thoughts: In the system described there a Lucene document is used for each low level xml construct, even when it contains very few characters of text. The resulting Lucene indexes are at least 2.5 times the size of the original document, which is not a surprise given this document structure. Normal index size is about one third of the indexed text. I don't know about the XQuery standard, but I was wondering whether this unusual document structure and the non straightforward fit between Lucene queries and XQuery queries are related. As for the joines and iterations over items from the stream of XML results: iteration over matching XML constructs should be no problem in Lucene. Joins in Lucene are normally done via boolean filters, so I was wondering how XQuery joins fit these. The case study above has a note a the end of par 5.3: "The Search Result list that comes back could then be organized by document id to group together all the results for a single XML document. This is not provided by default, but has been done with extension to this code." Regards, Paul Elschot On Friday 16 December 2005 03:45, Wolfgang Hoschek wrote: > I think implementing an XQuery Full-Text engine is far beyond the > scope of Lucene. > > Implementing a building block for the fulltext aspect of it would be > more manageable. Unfortunately The W3C fulltext drafts > indiscriminately mix and mingle two completely different languages > into a single language, without clear boundaries. That's why most > practical folks implement XQuery fulltext search via extension > functions rather than within XQuery itself. This also allows for much > more detailed tokenization, configuration and extensibility than what > would be possible with the W3C draft. > > Wolfgang. > > On Dec 15, 2005, at 4:20 PM, [EMAIL PROTECTED] wrote: > > > Mark, > > > > This is very cool. When I was at TripleHop we did something very > > similar where both query and results conformed to an XML Schema and > > we used XML over HTTP as our main vehicle to do remote/federated > > searches with quick rendering with stylesheets. > > > > That however is the first piece of the puzzle. If you really want > > to go beyond search (in the traditional sense) and be able to > > perform more complex operations such as joines and iterations over > > items from the stream of XML results you are getting you should > > consider implementing an XQuery Full-Text engine with Lucene > > adopting the now standard XQuery language. > > > > Here is the pointer to the working draft on the W3C working draft > > on XQuery 1.0 and XPath 2.0 Full-Text: > > http://www.w3.org/TR/xquery-full-text/ > > > > Now I'm part of the task force editing this draft so your comments > > are very much welcomed. > > > > -- J.D. > > > > > > http://www.inperspective.com/lucene/LXQueryV0_1.zip > > > > I've implemented just a few queries (Boolean, Term, FilteredQuery, > > BoostingQuery ...) but other queries are fairly trivial to add. > > At this stage I am more interested in feedback on parser design/ > > approach > > rather than trying to achieve complete coverage of all the Lucene > > Query > > types or debating the choice of tag names. > > > > Please see the readme.txt in the package for more details. > > > > Cheers > > Mark --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
