At Veoh, we pulled machines from the pool during updates, but we did the updates by copying entirely new indexes to the machines. We might have been able to avoid pulling the machines from the pool if we had used better I/O scheduling, but we were very risk averse by that time.
My current system uses elastic computing instead based on Katta. When doing a full update, we just launch an entire new search cluster and then flip traffic to it. Nodes download shards autonomously from an HDFS file system. Updates in the form of additions (our dominant form) are propagated by simply adding a new shard and letting Katta distribute it. This seems to work *much* better than the Solr based system we had a Veoh. On Fri, Apr 10, 2009 at 3:18 AM, sunnyfr <[email protected]> wrote: > How people do when they have frequent update to do ? to they turn off their > servers during the warmup ??? > what is the solution, should I make it lower ?? > -- Ted Dunning, CTO DeepDyve
