I guess that's the big wish we all have! :) One other big problem with doing inline updates is that we have to write-once model in Lucene, so we never touch a file after it was written.
My goal is to be able to achieve what you have in mind with CSF and parallel incremental indexing. When we have CSFs we will be able to define a fixed-length CSF. With parallel indexing you will be able to create a parallel index containing only that field and you can write a new version of it containing the update. This works very efficiently if you want to change a large number of field values. However, if you often change only a single value, then we should work on a solution optimized for that use case, something like Yonik pointed out recently on java-dev. The other possible performance problem is that when you end up with multiple files (parallel indexes) on disk each containing only a few updates, that a read of such a CSF is not sequential then anymore, because we need to seek to each of those file in case of updates. With two-dimensional merge policies (mentioned on the parallel indexing wiki page) we should be able to balance merge and read overhead by controlling the number of parallel indexes we have, similar to what is done with "normal" segment merges today. Also the perf problems of frequent small updates can probably be improved by have a buffer and writing the new parallel index in the background, while serving read requests from the buffer (for NRT readers). Michael On Wed, Sep 23, 2009 at 3:40 PM, Grant Ingersoll <gsing...@apache.org>wrote: > Has anyone done any work on modifying payloads "inline" in the index? The > idea being that if you know the length of the payload isn't changing, you > can modify it w/o reindexing. Some concerns that come to mind are > thread-safety, etc. > > -Grant > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > >
