Shawn,
I managed to create 8 new cores and the Solr Admin cloud page showed
them wonderfully as active replicas.
The only issue I have is what goes into solr.xml (I'm using tomcat)?
Putting
<core name="..." />
for each of the new cores I created seemed like the reasonable approach
but when I tested a tomcat restart, the distribution was messed up ...
for one thing, the cores on the new machine showed up as collections!
And tomcat never even made it to accept connections for some reason.
I cleaned everything up with zookepper so my graph looks like it should
and I removed that new machine from the distribution (by removing zk
attributes) and restarted .. all is well again.
Any idea what could have went wrong on tomcat restart?
thanks.
On 12/22/2013 09:48 PM, Shawn Heisey wrote:
On 12/22/2013 2:10 PM, David Santamauro wrote:
My goal is to have a redundant copy of all 8 currently running, but
non-redundant shards. This setup (8 nodes with no replicas) was a test
and it has proven quite functional from a performance perspective.
Loading, though, takes almost 3 weeks so I'm really not in a position to
redesign the distribution, though I can add nodes.
I have acquired another resource, a very large machine that I'd like to
use to hold the replicas of the currently deployed 8-nodes.
I realize I can run 8 jetty/tomcats and accomplish my goal but that is a
maintenance headache and is really a last resort. I really would just
like to be able to deploy this big machine with 'numShards=8'.
Is that possible or do I really need to have 8 other nodes running?
You don't want to run more than one container or Solr instance per
machine. Things can get very confused, and it's too much overhead.
Also, you shouldn't start Solr with the numShards parameter on the
commandline. That should be given when you create each collection.
With existing collections, you can simply run the CoreAdmin CREATE
action on the new node with more resources.
http://wiki.apache.org/solr/SolrCloud#Creating_cores_via_CoreAdmin
So you'd do something like this, once for each of the 8 existing parts:
http://newnode:port/solr/admin/cores?action=CREATE&name=collname_shard1_replica2&collection=collname&shard=shard1
It will automatically replicate the shard from its current leader.
One thing to be aware of: With 1.4TB of index data, it might be
impossible to keep enough of the index in RAM for good performance,
unless the machine has a terabyte or more of RAM.
http://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache
Thanks,
Shawn
