Re: zkcli issues
I agree that it *shouldn't* be OS specific. :) Anyway, thanks for the suggestion, but that's not it. I get the same error with the script right out of the box: Error: Could not find or load main class C:\sw\apache-solr-4.0.0\example\cloud-scripts\..\solr-webapp\webapp\WEB-INF\lib\apache-solr-solrj-4.0.0.jar And anyway, it's a weird error, referencing a jar as a class, isn't it? Start up a JIRA? - Nick On 11/16/2012 10:42 AM, Mark Miller wrote: I *think* I tested the script on windows once way back. Anyway, the code itself should not be OS specific. One thing you might want to check if you are copying unix cmd line stuff - I think windows separates classpath entries with ; rather than : - so you likely to need to change that. You'd think java could have been smart enough to accept either/or at worst, but meh. For example: .:/Users/jeeva/dc-1/solr-cli-lib/* should be .;/Users/jeeva/dc-1/solr-cli-lib/* - Mark On Thu, Nov 15, 2012 at 8:53 PM, Nick Chase nch...@earthlink.net wrote: Unfortunately, this doesn't seem to solve the issue; now I'm beginning to wonder if maybe it's because I'm on Windows. Has anyone successfully run ZkCLI on Windows? Nick On 11/12/2012 2:27 AM, Jeevanandam Madanagopal wrote: Nick - Sorry, embedded links are not shown in previous email. I'm mentioning below. Handy SolrCloud ZkCLI Commands (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#handy-solrcloud-cli-commands) Uploading Solr Configuration into ZooKeeper ensemble (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#uploading-solrconfig-to-zookeeper) Cheers, Jeeva On Nov 12, 2012, at 12:48 PM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Nick - I believe you're experiencing a difficulties with SolrCloud CLI commands for interacting ZooKeeper. Please have a look on below links, it will provide you direction. Handy SolrCloud ZkCLI Commands Uploading Solr Configuration into ZooKeeper ensemble Cheers, Jeeva On Nov 12, 2012, at 4:45 AM, Mark Miller markrmil...@gmail.com wrote: On 11/11/2012 04:47 PM, Yonik Seeley wrote: On Sun, Nov 11, 2012 at 10:39 PM, Nick Chase nch...@earthlink.net wrote: So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ.
Re: zkcli issues
Unfortunately, this doesn't seem to solve the issue; now I'm beginning to wonder if maybe it's because I'm on Windows. Has anyone successfully run ZkCLI on Windows? Nick On 11/12/2012 2:27 AM, Jeevanandam Madanagopal wrote: Nick - Sorry, embedded links are not shown in previous email. I'm mentioning below. Handy SolrCloud ZkCLI Commands (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#handy-solrcloud-cli-commands) Uploading Solr Configuration into ZooKeeper ensemble (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#uploading-solrconfig-to-zookeeper) Cheers, Jeeva On Nov 12, 2012, at 12:48 PM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Nick - I believe you're experiencing a difficulties with SolrCloud CLI commands for interacting ZooKeeper. Please have a look on below links, it will provide you direction. Handy SolrCloud ZkCLI Commands Uploading Solr Configuration into ZooKeeper ensemble Cheers, Jeeva On Nov 12, 2012, at 4:45 AM, Mark Miller markrmil...@gmail.com wrote: On 11/11/2012 04:47 PM, Yonik Seeley wrote: On Sun, Nov 11, 2012 at 10:39 PM, Nick Chase nch...@earthlink.net wrote: So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ.
Internal Vs. External ZooKeeper
OK, I can't find a definitive answer on this. The wiki says not to use the embedded ZooKeeper servers for production. But my question is: why not? Basically, what are the reasons and circumstances that make you better off using an external ZooKeeper ensemble? Thanks... Nick
Re: Internal Vs. External ZooKeeper
Thanks, Jack, this is a great explanation! And since a greater number of ZK nodes tends to degrade write performance, that would be a factor in making every Solr node a ZK node as well. Much obliged! Nick On 11/11/2012 10:45 AM, Jack Krupansky wrote: Production typically implies high availability and in a distributed system the goal is that the overall cluster integrity and performance should not be compromised just because a few worker nodes go down. Solr nodes do a lot of complex operations and are quite prone to running into issues that compromise their integrity and require that they be taken down, restarted, etc. In fact, taking down a bunch of Solr worker nodes should not be a big deal (unless they are all of the nodes/replicas from a single shard/slice), while taking down a bunch of zookeepers could be catastrophic to maintaining the integrity of the zookeeper ensemble. (OTOH, if every Solr node is also a zookeeper node, a bunch of Solr nodes would generally be less than a quorum, so maybe that is not an absolute issue per se.) Zookeeper nodes are categorically distinct in terms of their importance to maintaining the integrity and availability of the overall cluster. They are special in that sense. And they are special because they are maintaining the integrity of the cluster's configuration information. Even for large clusters their number will be relatively few compared to the many of worker nodes (replicas), so zookeeper nodes need to be protected from the vagaries that can disrupt and take Solr nodes down, not the least of which is incoming traffic. I'm not sure what the implications would be if you had a large cluster and because Zookeeper was embedded you had a large number of zookeepers. Any of the inter-zookeeper operations would take longer and could be compromised by even a single busy/overloaded/dead Solr node. OTOH, the Zookeeper ensemble design is supposed to be able to handle a far number of missing zookeeper nodes. OTOH, if high availability is not a requirement for a production cluster (use case?), then non-embedded zookeepers are certainly an annoyance. Maybe you could think of embedded zookeeper like every employee having their manager sitting right next to them all the time. How could that be anything but a bad idea in terms of maximizing worker output - and distracting/preventing managers from focusing on their own work? -- Jack Krupansky -Original Message- From: Nick Chase Sent: Sunday, November 11, 2012 7:12 AM To: solr-user@lucene.apache.org Subject: Internal Vs. External ZooKeeper OK, I can't find a definitive answer on this. The wiki says not to use the embedded ZooKeeper servers for production. But my question is: why not? Basically, what are the reasons and circumstances that make you better off using an external ZooKeeper ensemble? Thanks... Nick
zkcli issues
OK, so this is my ZooKeeper week, sorry. :) So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ. I even tried to run it from the provided script (in cloud-scripts) with no success. Here's what I've got: cd my-solr-install .\example\cloud-scripts\zkcli.bat -cmd upconfig -zkhost localhost:9983 -confdir example/solr/collection/conf -confname conf1 -solrhome example/solr set JVM=java set SDIR=C:\sw\apache-solr-4.0.0\example\cloud-scripts\ if \ == \ set SDIR=C:\sw\apache-solr-4.0.0\example\cloud-scripts java -classpath C:\sw\apache-solr-4.0.0\example\cloud-scripts\..\solr-webapp\webapp\WEB-INF\lib\* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost localhost:9983 -confdir example/solr/collection/conf -confname conf1 -solrhome example/solr Error: Could not find or load main class C:\sw\apache-solr-4.0.0\example\cloud-scripts\..\solr-webapp\webapp\WEB-INF\lib\apache-solr-solrj-4.0.0.jar I've verified that C:\sw\apache-solr-4.0.0\example\cloud-scripts\..\solr-webapp\webapp\WEB-INF\lib\apache-solr-solrj-4.0.0.jar exists, so I'm really at a loss here. Thanks... Nick
Re: SolrCloud failover behavior
Thanks a million, Erick! You're right about killing both nodes hosting the shard. I'll get the wiki corrected. Nick On 11/3/2012 10:51 PM, Erick Erickson wrote: SolrCloud doesn't work unless every shard has at least one server that is up and running. I _think_ you might be killing both nodes that host one of the shards. The admin page has a link showing you the state of your cluster. So when this happens, does that page show both nodes for that shard being down? And yeah, SolrCloud requires a quorum of ZK nodes up. So with only one ZK node, killing that will bring down the whole cluster. Which is why the usual recommendation is that ZK be run externally and usually an odd number of ZK nodes (three or more). Anyone can create a login and edit the Wiki, so any clarifications are welcome! Best Erick On Sat, Nov 3, 2012 at 12:17 PM, Nick Chase nch...@earthlink.net wrote: I think there's a change in the behavior of SolrCloud vs. what's in the wiki, but I was hoping someone could confirm for me. I checked JIRA and there were a couple of issues requesting partial results if one server comes down, but that doesn't seem to be the issue here. I also checked CHANGES.txt and don't see anything that seems to apply. I'm running Example B: Simple two shard cluster with shard replicas from the wiki at https://wiki.apache.org/solr/**SolrCloudhttps://wiki.apache.org/solr/SolrCloudand everything starts out as expected. However, when I get to the part about fail over behavior is when things get a little wonky. I added data to the shard running on 7475. If I kill 7500, a query to any of the other servers works fine. But if I kill 7475, rather than getting zero results on a search to 8983 or 8900, I get a 503 error: response lst name=responseHeader int name=status503/int int name=QTime5/int lst name=params str name=q*:*/str /lst /lst lst name=error str name=msgno servers hosting shard:/str int name=code503/int /lst /response I don't see any errors in the consoles. Also, if I kill 8983, which includes the Zookeeper server, everything dies, rather than just staying in a steady state; the other servers continually show: Nov 03, 2012 11:39:34 AM org.apache.zookeeper.**ClientCnxn$SendThread startConnect NFO: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:9983 ov 03, 2012 11:39:35 AM org.apache.zookeeper.**ClientCnxn$SendThread run ARNING: Session 0x13ac6cf87890002 for server null, unexpected error, closing socket connection and attempting reconnect ava.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.**checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.**finishConnect(Unknown Source) at org.apache.zookeeper.**ClientCnxn$SendThread.run(** ClientCnxn.java:1143) ov 03, 2012 11:39:35 AM org.apache.zookeeper.**ClientCnxn$SendThread startConnect over and over again, and a call to any of the servers shows a connection error to 8983. This is the current 4.0.0 release, running on Windows 7. If this is the proper behavior and the wiki needs updating, fine; I just need to know. Otherwise if anybody has any clues as to what I may be missing, I'd be grateful. :) Thanks... --- Nick
SolrCloud failover behavior
I think there's a change in the behavior of SolrCloud vs. what's in the wiki, but I was hoping someone could confirm for me. I checked JIRA and there were a couple of issues requesting partial results if one server comes down, but that doesn't seem to be the issue here. I also checked CHANGES.txt and don't see anything that seems to apply. I'm running Example B: Simple two shard cluster with shard replicas from the wiki at https://wiki.apache.org/solr/SolrCloud and everything starts out as expected. However, when I get to the part about fail over behavior is when things get a little wonky. I added data to the shard running on 7475. If I kill 7500, a query to any of the other servers works fine. But if I kill 7475, rather than getting zero results on a search to 8983 or 8900, I get a 503 error: response lst name=responseHeader int name=status503/int int name=QTime5/int lst name=params str name=q*:*/str /lst /lst lst name=error str name=msgno servers hosting shard:/str int name=code503/int /lst /response I don't see any errors in the consoles. Also, if I kill 8983, which includes the Zookeeper server, everything dies, rather than just staying in a steady state; the other servers continually show: Nov 03, 2012 11:39:34 AM org.apache.zookeeper.ClientCnxn$SendThread startConnect NFO: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:9983 ov 03, 2012 11:39:35 AM org.apache.zookeeper.ClientCnxn$SendThread run ARNING: Session 0x13ac6cf87890002 for server null, unexpected error, closing socket connection and attempting reconnect ava.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143) ov 03, 2012 11:39:35 AM org.apache.zookeeper.ClientCnxn$SendThread startConnect over and over again, and a call to any of the servers shows a connection error to 8983. This is the current 4.0.0 release, running on Windows 7. If this is the proper behavior and the wiki needs updating, fine; I just need to know. Otherwise if anybody has any clues as to what I may be missing, I'd be grateful. :) Thanks... --- Nick
Re: Help! Confused about using Jquery for the Search query - Want to ditch it
On 6/7/2012 1:53 PM, Spadez wrote: Hi Ben, Thank you for the reply. So, If I don't want to use Javascript and I want the entire page to reload each time, is it being done like this? 1. User submits form via GET 2. Solr server queried via GET 3. Solr server completes query 4. Solr server returns XML output 5. XML data put into results page 6. User shown new results page Is this basically how it would work if we wanted Javascript out of the equation? Seems to me that you'd still have to have Javascript turn the XML into HTML -- unless you use the XsltResponseWriter (http://wiki.apache.org/solr/XsltResponseWriter) to use XSLT to turn the raw XML into your actual results HTML. The other option is to create a python page that does the call to Solr and spits out just the HTML for your results, then call THAT rather than calling Solr directly. Nick
Re: Help! Confused about using Jquery for the Search query - Want to ditch it
+1 on that! If you do want to provide direct results, ALWAYS send requests through a proxy that can verify that a) all requests are coming from your web app, and b) only acceptable queries are being passed on. Nick On 6/7/2012 2:50 PM, Michael Della Bitta wrote: On Thu, Jun 7, 2012 at 1:59 PM, Nick Chasench...@earthlink.net wrote: The other option is to create a python page that does the call to Solr and spits out just the HTML for your results, then call THAT rather than calling Solr directly. This is the *only* option if you're listening to Walter and I. Don't give end users direct access to your Solr box!
