Hi, I've about 14M of document. My index is about 11G. For the moment I update every 20mn about 30 000 documents. Lucene alwarys merge data, What would you reckon? My replication cost too much for the slave, they always bring back new index directories and no segment.
Is there a way to get around this issue ? what would you reckon to people who need fresh update on the slave with a big amount of data ?? Thanks a lot, http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr/ : "When the time and bandwidth needed for replication is less of a concern, and high query throughput is more important, it can be wise to abandon the advantage of transferring changed segments and only replicate fully optimized indexes. It costs a bit more in terms of resources, but the master will eat the cost of optimizing (so that users don't see the standard machine slowdown affect that performing an optimize brings), and the slaves will always get a fully optimized index to issue queries against, allowing for maximum query performance. Generally, bandwidth for replication is not much of a concern now, but keep in mind that optimizing on a large index can be quite time consuming, so this strategy is not for every situation." -- View this message in context: http://www.nabble.com/commit-often-and-lot-of-data-cost-too-much--tp22804941p22804941.html Sent from the Lucene - General mailing list archive at Nabble.com.
