On 5/9/2013 7:31 AM, heaven wrote: > Can confirm this lead to data loss. I have 1217427 records in database and > only 1217216 indexed. Which does mean that Solr gave a successful response > and then did not added some documents to the index. > > Seems like SolrCloud is not a production-ready solution, would be good if > there was a warning in the Solr wiki about that.
You've got some kind of underlying problem here. Here are my guesses about what that might be: - An improperly configured Linux firewall and/or SELinux is enabled. - The hardware is already overtaxed by other software. - Your zkClientTimeout value is extremely small. - Your GC pauses are large. - You're running into an open file limit. Here's what you could do to resolve each of these: - Disable the firewall and selinux, reboot. - Stop other software. - The example zkClientTimeout is 15 seconds. Try 30-60. - See http://wiki.apache.org/solr/SolrPerformanceProblems for some GC ideas. - Increase the file and process limits. For most versions of Linux, in /etc/security/limits.conf: solr hard nproc 6144 solr soft nproc 4096 solr hard nofile 65536 solr soft nofile 49152 These numbers should be sufficient for deployments considerably larger than yours. SolrCloud is not only production ready, it's being used by many many people for extremely large indexes. My own SolrCloud deployment is fairly small with only 1.5 million docs, but it's extremely stable. I also have a somewhat large (77 million docs) non-cloud deployment. Are you running 4.2.1? I feel fairly certain based on your screenshots that you are not running 4.3, but I can't tell which version you are running. There are some bugs in the 4.3 release, a 4.3.1 will be released soon. If you had planned to upgrade, you should wait for 4.3.1 or 4.4. NB, and something you might already know: When talking about production-ready, you can't run everything on the same server. You need at least three - two of them can run Solr and zookeeper, and the third runs zookeeper. This single-server setup is fine for a proof-of-concept. Thanks, Shawn
