On Feb 28, 2013, at 7:55 PM, Walter Underwood <wun...@wunderwood.org> wrote:
> 100 shards on a node will almost certainly be slow I think it depends on some things - with one of the largest of those things being your hardware. Many have found that you can get much better performance out of super concurrent, beefy hardware using more cores on a single node. So there will be some give and take that are tough jump to conclusions about. Slower at 100, I would assume yes, slow, depends. One thing that will happen is that you will require a lot more threads… You would want some pretty beefy hardware. But you don't have to do 100 either. That should just be a rough starting number. At some point you have to reindex into a new cluster if you keep growing. Or consider shard splitting if its feasible (and becomes available). You can only over shard so much. So perhaps you do 50 or whatever. It will be faster than you think I imagine. My main concern is the number of threads - might want to mess with Xss to minimize their ram usage at least. - Mark
