Grant Ingersoll <[EMAIL PROTECTED]> wrote on 28/05/2006 06:22:06 PM: > I have added http://wiki.apache.org/jakarta-lucene/LucenePlanning to the > Wiki. Currently there are two items of interest in it. A start of some > documentation related to a Java 1.5 migration and a start of some > documentation concerning how to add more flexible indexing options and > how to store metadata at the index level. The former conversation was > started by Karl on the developer's list and the latter was kicked off by > an email from me to Doug on how to implement #11 of > http://wiki.apache.org/jakarta-lucene/Lucene2Whiteboard
I think the suggestion for position-specific boost is not enough, and what is really be needed is a more general "payload" mechanism, that allows storing with each position a variable length payload (byte[]) which the application can use for its purposes. Such payloads are essential for many applications - including XML search, faceted search (if you don't want to cache stuff in memory, like people suggested on a thread from last week), fast numeric search, and more. Adding payloads is actually not difficult, but would require a change to the index file format (probably the positions file) and some changes to the basic indexing API (such as a new Field constructor with a payload, adding payloads to tokens coming out of an analyzer, and getting payloads from a TermPositions), so we better do this after a bit of thought, and do it now - when it's natural to start thinking about changes to the index file format. Another, related, improvement, I think, should be to make positions optional for certain fields. For some fields, positions are useless because phrase search will never be used. For example, a field that keeps a list of "categories" that a document is in. A document can either be, or not be, in a category, but there is no significance in the order of these categories in a document's list. -- Nadav Har'El --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
