Unsubscribing MAIL
HI I want unsubscribe the mailing list od solr and lucene so plz do the same.. Regards, Kishan Parmar Software Developer +91 95 100 77394 Jay Shree Krishnaa !!
Re: solr cloud does not start with many collections
On 2/26/2015 11:14 PM, Damien Kamerman wrote: I've run into an issue with starting my solr cloud with many collections. My setup is: 3 nodes (solr 4.10.3 ; 64GB RAM each ; jdk1.8.0_25) running on a single server (256GB RAM). 5,000 collections (1 x shard ; 2 x replica) = 10,000 cores 1 x Zookeeper 3.4.6 Java arg -Djute.maxbuffer=67108864 added to solr and ZK. Then I stop all nodes, then start all nodes. All replicas are in the down state, some have no leader. At times I have seen some (12 or so) leaders in the active state. In the solr logs I see lots of: org.apache.solr.cloud.ZkController; Still seeing conflicting information about the leader of shard shard1 for collection DD-4351 after 30 seconds; our state says http://ftea1:8001/solr/DD-4351_shard1_replica1/, but ZooKeeper says http://ftea1:8000/solr/DD-4351_shard1_replica2/ snip I've tried staggering the starts (1min) but does not help. I've reproduced with zero documents. Restarts are OK up to around 3,000 cores. Should this work? This is going to push SolrCloud beyond its limits. Is this just an exercise to see how far you can push Solr, or are you looking at setting up a production install with several thousand collections? In Solr 4.x, the clusterstate is one giant JSON structure containing the state of the entire cloud. With 5000 collections, the entire thing would need to be downloaded and uploaded at least 5000 times during the course of a successful full system startup ... and I think with replicationFactor set to 2, that might actually be 1 times. The best-case scenario is that it would take a VERY long time, the worst-case scenario is that concurrency problems would lead to a deadlock. A deadlock might be what is happening here. In Solr 5.x, the clusterstate is broken up so there's a separate state structure for each collection. This setup allows for faster and safer multi-threading and far less data transfer. Assuming I understand the implications correctly, there might not be any need to increase jute.maxbuffer with 5.x ... although I have to assume that I might be wrong about that. I would very much recommend that you set your scenario up from scratch in Solr 5.0.0, to see if the new clusterstate format can eliminate the problem you're seeing. If it doesn't, then we can pursue it as a likely bug in the 5.x branch and you can file an issue in Jira. Thanks, Shawn
Re: Unsubscribing MAIL
On 27 February 2015 at 12:10, Kishan Parmar kishan@gmail.com wrote: HI I want unsubscribe the mailing list od solr and lucene so plz do the same.. Please follow the standard procedure for unsubscribing from most mailing lists, and send a mail to solr-user-unsubscr...@lucene.apache.org . For other lists, you might want to take a look at the addresses listed under http://lucene.apache.org/core/discussion.html Regards, Gora
Re: solr cloud does not start with many collections
Oh, and I was wondering if 'leaderVoteWait' might help in Solr4. On 27 February 2015 at 18:04, Damien Kamerman dami...@gmail.com wrote: This is going to push SolrCloud beyond its limits. Is this just an exercise to see how far you can push Solr, or are you looking at setting up a production install with several thousand collections? I'm looking towards production. In Solr 4.x, the clusterstate is one giant JSON structure containing the state of the entire cloud. With 5000 collections, the entire thing would need to be downloaded and uploaded at least 5000 times during the course of a successful full system startup ... and I think with replicationFactor set to 2, that might actually be 1 times. The best-case scenario is that it would take a VERY long time, the worst-case scenario is that concurrency problems would lead to a deadlock. A deadlock might be what is happening here. Yes, clusterstate.json is 3.3M. At times on startup I think it does deadlock; log shows after 1min: org.apache.solr.cloud.ZkController; Timed out waiting to see all nodes published as DOWN in our cluster state. In Solr 5.x, the clusterstate is broken up so there's a separate state structure for each collection. This setup allows for faster and safer multi-threading and far less data transfer. Assuming I understand the implications correctly, there might not be any need to increase jute.maxbuffer with 5.x ... although I have to assume that I might be wrong about that. I would very much recommend that you set your scenario up from scratch in Solr 5.0.0, to see if the new clusterstate format can eliminate the problem you're seeing. If it doesn't, then we can pursue it as a likely bug in the 5.x branch and you can file an issue in Jira. Thanks, will test in Solr 5.0.0. -- Damien Kamerman
Re: solr cloud does not start with many collections
This is going to push SolrCloud beyond its limits. Is this just an exercise to see how far you can push Solr, or are you looking at setting up a production install with several thousand collections? I'm looking towards production. In Solr 4.x, the clusterstate is one giant JSON structure containing the state of the entire cloud. With 5000 collections, the entire thing would need to be downloaded and uploaded at least 5000 times during the course of a successful full system startup ... and I think with replicationFactor set to 2, that might actually be 1 times. The best-case scenario is that it would take a VERY long time, the worst-case scenario is that concurrency problems would lead to a deadlock. A deadlock might be what is happening here. Yes, clusterstate.json is 3.3M. At times on startup I think it does deadlock; log shows after 1min: org.apache.solr.cloud.ZkController; Timed out waiting to see all nodes published as DOWN in our cluster state. In Solr 5.x, the clusterstate is broken up so there's a separate state structure for each collection. This setup allows for faster and safer multi-threading and far less data transfer. Assuming I understand the implications correctly, there might not be any need to increase jute.maxbuffer with 5.x ... although I have to assume that I might be wrong about that. I would very much recommend that you set your scenario up from scratch in Solr 5.0.0, to see if the new clusterstate format can eliminate the problem you're seeing. If it doesn't, then we can pursue it as a likely bug in the 5.x branch and you can file an issue in Jira. Thanks, will test in Solr 5.0.0.
Re: Dependency Need to include for embedded solr.
On 2/26/2015 11:41 PM, Danesh Kuruppu wrote: My application is a standalone application. I though of embedding solr server, so I can pack it inside my application. In solr 5.0.0, solr is no longer distributed as a war file. how I can find the war file from the distribution. I am glad to see that people are actually reading documentation that is included with the release. With 5.0.0 (and probably the next few 5.x releases), Solr actually still is a war file. You can find it in server/webapps in the binary download. There are two reason we are telling everyone it's not a war file: 1) We now have very capable scripts to start and stop Solr with optimal java options, so there's no longer any need to rely on scripts packaged with a servlet container. 2) In a future 5.x release, Solr actually will become a standalone application, not a war ... preparing users in advance is a good idea. I need some advanced features like synonyms search, stop words, wild card search etc. It would be great, if you can provide some references to get idea which dependencies need to add to get those features. If you don't want to simply add every dependency included in the war, then you can use the tried and true method for finding the minimum set of jars: Try to get it running. If it fails, look at the log and see which class it was unable to find. Add the relevant jar to the classpath and try again. Thanks, Shawn
Re: Dependency Need to include for embedded solr.
Thanks shawn, I am doing some feasibility studies for moving directly to solr 5.0.0. One more thing, It is related to standalone server. How security handle in solr standalone server. lets say, I configured my application to use remote solr standalone server. 1. How I would enable secure communication between my application and solr server. 2. How solr server authenticate user. Thanks Danesh On Fri, Feb 27, 2015 at 12:35 PM, Shawn Heisey apa...@elyograg.org wrote: On 2/26/2015 11:41 PM, Danesh Kuruppu wrote: My application is a standalone application. I though of embedding solr server, so I can pack it inside my application. In solr 5.0.0, solr is no longer distributed as a war file. how I can find the war file from the distribution. I am glad to see that people are actually reading documentation that is included with the release. With 5.0.0 (and probably the next few 5.x releases), Solr actually still is a war file. You can find it in server/webapps in the binary download. There are two reason we are telling everyone it's not a war file: 1) We now have very capable scripts to start and stop Solr with optimal java options, so there's no longer any need to rely on scripts packaged with a servlet container. 2) In a future 5.x release, Solr actually will become a standalone application, not a war ... preparing users in advance is a good idea. I need some advanced features like synonyms search, stop words, wild card search etc. It would be great, if you can provide some references to get idea which dependencies need to add to get those features. If you don't want to simply add every dependency included in the war, then you can use the tried and true method for finding the minimum set of jars: Try to get it running. If it fails, look at the log and see which class it was unable to find. Add the relevant jar to the classpath and try again. Thanks, Shawn
Dependency Need to include for embedded solr.
Hi all, I need to include embed solr server into my maven project. I am going to use latest solr 5.0.0. Need to know which dependencies I need to include in my project. As I understand, I need to have solr-core[1] and solr-solrj[2]. Do I need to include lucene dependency in my project. If so, which dependencies we need to include to enable all indexing capabilities. 1. http://mvnrepository.com/artifact/org.apache.solr/solr-core/5.0.0 2. http://mvnrepository.com/artifact/org.apache.solr/solr-solrj/5.0.0 Please help Thanks Danesh
Re: Dependency Need to include for embedded solr.
On 2/26/2015 10:07 PM, Danesh Kuruppu wrote: I need to include embed solr server into my maven project. I am going to use latest solr 5.0.0. Need to know which dependencies I need to include in my project. As I understand, I need to have solr-core[1] and solr-solrj[2]. Do I need to include lucene dependency in my project. If so, which dependencies we need to include to enable all indexing capabilities. 1. http://mvnrepository.com/artifact/org.apache.solr/solr-core/5.0.0 2. http://mvnrepository.com/artifact/org.apache.solr/solr-solrj/5.0.0 Using the embedded server may not be the best idea. A lot of Solr functionality is not available in the embedded server. You can't use SolrCloud, which is a relatively easy way to provide high availability. The legacy method for redundancy, master-slave replication, also does not work in the embedded server. The admin UI is not available. If you choose to go ahead with the embedded server ... for complete safety, you should probably extract the war file and copy all the jars from WEB-INF/lib. If you want to take a more minimalistic approach, I think these are the Lucene jars you will need for minimum functionality: lucene-analyzers-common-5.0.0.jar lucene-codecs-5.0.0.jar lucene-core-5.0.0.jar lucene-expressions-5.0.0.jar lucene-queries-5.0.0.jar lucene-queryparser-5.0.0.jar There are quite a few Lucene jars, and I'm not overly familiar with everything that Solr uses, so I might have left some out that would be required for very basic functionality. For more advanced functionality, additional Lucene jars will definitely be required. There are also third-party jars that are required, such as slf4j jars for logging. The codebase as a whole has dependencies on things like google guava, several apache commons jars, and other pieces ... I have no idea which of those can be left out when using the embedded server. I tried to find a definitive list of required jars, and was not able to locate one. Thanks, Shawn
Get suggestion for each term in the query
Hi, I want to get suggestion of each term/word in query. Condition: i) Either word/term is correct or incorrect. ii) Either word/term has high frequency or has low frequency. Whatever the condition of term/word, I need to suggestion all time.
solr cloud does not start with many collections
I've run into an issue with starting my solr cloud with many collections. My setup is: 3 nodes (solr 4.10.3 ; 64GB RAM each ; jdk1.8.0_25) running on a single server (256GB RAM). 5,000 collections (1 x shard ; 2 x replica) = 10,000 cores 1 x Zookeeper 3.4.6 Java arg -Djute.maxbuffer=67108864 added to solr and ZK. Then I stop all nodes, then start all nodes. All replicas are in the down state, some have no leader. At times I have seen some (12 or so) leaders in the active state. In the solr logs I see lots of: org.apache.solr.cloud.ZkController; Still seeing conflicting information about the leader of shard shard1 for collection DD-4351 after 30 seconds; our state says http://ftea1:8001/solr/DD-4351_shard1_replica1/, but ZooKeeper says http://ftea1:8000/solr/DD-4351_shard1_replica2/ org.apache.solr.common.SolrException; :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1 at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:910) at org.apache.solr.cloud.ZkController.register(ZkController.java:822) at org.apache.solr.cloud.ZkController.register(ZkController.java:770) at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:221) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.common.SolrException: There is conflicting information about the leader of shard: shard1 our state says: http://ftea1:8001/solr/DD-1564_shard1_replica2/ but zookeeper says: http://ftea1:8000/solr/DD-1564_shard1_replica1/ at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:889) ... 6 more I've tried staggering the starts (1min) but does not help. I've reproduced with zero documents. Restarts are OK up to around 3,000 cores. Should this work? Damien.
Confusion in making true or false in spellcheck.onlymorepopular
HI, Only return suggestions that result in more hits for the query than the existing query What does it means the existing query in above sentence for spellcheck.onlymorepopular? what happens when I make true to spellcheck.onlymorepopular or false to spellcheck.onlymorepopular? Any difference in it?
Re: Dependency Need to include for embedded solr.
Thanks Shawn, My application is a standalone application. I though of embedding solr server, so I can pack it inside my application. In solr 5.0.0, solr is no longer distributed as a war file. how I can find the war file from the distribution. I need some advanced features like synonyms search, stop words, wild card search etc. It would be great, if you can provide some references to get idea which dependencies need to add to get those features. Thanks Danesh On Fri, Feb 27, 2015 at 11:32 AM, Shawn Heisey apa...@elyograg.org wrote: On 2/26/2015 10:07 PM, Danesh Kuruppu wrote: I need to include embed solr server into my maven project. I am going to use latest solr 5.0.0. Need to know which dependencies I need to include in my project. As I understand, I need to have solr-core[1] and solr-solrj[2]. Do I need to include lucene dependency in my project. If so, which dependencies we need to include to enable all indexing capabilities. 1. http://mvnrepository.com/artifact/org.apache.solr/solr-core/5.0.0 2. http://mvnrepository.com/artifact/org.apache.solr/solr-solrj/5.0.0 Using the embedded server may not be the best idea. A lot of Solr functionality is not available in the embedded server. You can't use SolrCloud, which is a relatively easy way to provide high availability. The legacy method for redundancy, master-slave replication, also does not work in the embedded server. The admin UI is not available. If you choose to go ahead with the embedded server ... for complete safety, you should probably extract the war file and copy all the jars from WEB-INF/lib. If you want to take a more minimalistic approach, I think these are the Lucene jars you will need for minimum functionality: lucene-analyzers-common-5.0.0.jar lucene-codecs-5.0.0.jar lucene-core-5.0.0.jar lucene-expressions-5.0.0.jar lucene-queries-5.0.0.jar lucene-queryparser-5.0.0.jar There are quite a few Lucene jars, and I'm not overly familiar with everything that Solr uses, so I might have left some out that would be required for very basic functionality. For more advanced functionality, additional Lucene jars will definitely be required. There are also third-party jars that are required, such as slf4j jars for logging. The codebase as a whole has dependencies on things like google guava, several apache commons jars, and other pieces ... I have no idea which of those can be left out when using the embedded server. I tried to find a definitive list of required jars, and was not able to locate one. Thanks, Shawn
Re: Getting started with Solr
I’m sorry, I’m not following exactly. Somehow you no longer have a gettingstarted collection, but it is not clear how that happened. Could you post the exact script steps you used that got you this error? What collections/cores does the Solr admin show you have?What are the results of http://localhost:8983/solr/admin/cores http://localhost:8983/solr/admin/cores ? — Erik Hatcher, Senior Solutions Architect http://www.lucidworks.com http://www.lucidworks.com/ On Feb 26, 2015, at 9:58 AM, Baruch Kogan bar...@sellerpanda.com wrote: Oh, I see. I used the start -e cloud command, then ran through a setup with one core and default options for the rest, then tried to post the json example again, and got another error: buntu@ubuntu-VirtualBox:~/crawler/solr$ bin/post -c gettingstarted example/exampledocs/*.json /usr/lib/jvm/java-7-oracle/bin/java -classpath /home/ubuntu/crawler/solr/dist/solr-core-5.0.0.jar -Dauto=yes -Dc=gettingstarted -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/books.json SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/gettingstarted/update... Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log POSTing file books.json (application/json) to [base] SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: http://localhost:8983/solr/gettingstarted/update SimplePostTool: WARNING: Response: html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 404 Not Found/title /head bodyh2HTTP ERROR 404/h2 pProblem accessing /solr/gettingstarted/update. Reason: preNot Found/pre/phr /ismallPowered by Jetty:///small/ibr/ Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype On Thu, Feb 26, 2015 at 4:07 PM, Erik Hatcher erik.hatc...@gmail.com wrote: How did you start Solr? If you started with `bin/solr start -e cloud` you’ll have a gettingstarted collection created automatically, otherwise you’ll need to create it yourself with `bin/solr create -c gettingstarted` — Erik Hatcher, Senior Solutions Architect http://www.lucidworks.com http://www.lucidworks.com/ On Feb 26, 2015, at 4:53 AM, Baruch Kogan bar...@sellerpanda.com wrote: Hi, I've just installed Solr (will be controlling with Solarium and using to search Nutch queries.) I'm working through the starting tutorials described here: https://cwiki.apache.org/confluence/display/solr/Running+Solr When I try to run $ bin/post -c gettingstarted example/exampledocs/*.json, I get a bunch of errors having to do with there not being a gettingstarted folder in /solr/. Is this normal? Should I create one? Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype
Re: [ANNOUNCE] Luke 4.10.3 released
Sorry, I'm afraid I have not encountered such errors when launch. Seems something wrong around Pivot's, but I have no idea about it. Would you tell me java version you're using ? Tomoko 2015-02-26 21:15 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Thanks, Tomoko, it compiles ok! Now launching produces some errors: $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication Exception in thread main java.lang.ExceptionInInitializerError at org.apache.lucene.luke.ui.LukeApplication.main(Unknown Source) Caused by: java.lang.NumberFormatException: For input string: 3 1644336 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:492) at java.lang.Byte.parseByte(Byte.java:148) at java.lang.Byte.parseByte(Byte.java:174) at org.apache.pivot.util.Version.decode(Version.java:156) at org.apache.pivot.wtk.ApplicationContext.clinit(ApplicationContext.java:1704) ... 1 more On Thu, Feb 26, 2015 at 1:48 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thank you for checking out it! Sorry, I've forgot to note important information... ivy jar is needed to compile. Packaging process needs to be organized, but for now, I'm borrowing it from lucene's tools/lib. In my environment, Fedora 20 and OpenJDK 1.7.0_71, it can be compiled and run as follows. If there are any problems, please let me know. $ svn co http://svn.apache.org/repos/asf/lucene/sandbox/luke/ $ cd luke/ // copy ivy jar to lib/tools $ cp /path/to/lucene_solr_4_10_3/lucene/tools/lib/ivy-2.3.0.jar lib/tools/ $ ls lib/tools/ ivy-2.3.0.jar $ java -version java version 1.7.0_71 OpenJDK Runtime Environment (fedora-2.5.3.3.fc20-x86_64 u71-b14) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) $ ant ivy-resolve ... BUILD SUCCESSFUL // compile and make jars and run $ ant dist ... BUILD SUCCESSFULL $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication ... Thanks, Tomoko 2015-02-26 16:39 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi Tomoko, Thanks for the link. Do you have build instructions somewhere? When I executed ant with no params, I get: BUILD FAILED /home/dmitry/projects/svn/luke/build.xml:40: /home/dmitry/projects/svn/luke/lib-ivy does not exist. On Thu, Feb 26, 2015 at 2:27 AM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thanks! Would you announce at LUCENE-2562 to me and all watchers interested in this issue, when the branch is ready? :) As you know, current pivots's version (that supports Lucene 4.10.3) is here. http://svn.apache.org/repos/asf/lucene/sandbox/luke/ Regards, Tomoko 2015-02-25 18:37 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Ok, sure. The plan is to make the pivot branch in the current github repo and update its structure accordingly. Once it is there, I'll let you know. Thank you, Dmitry On Tue, Feb 24, 2015 at 5:26 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Hi Dmitry, Thank you for the detailed clarification! Recently, I've created a few patches to Pivot version(LUCENE-2562), so I'd like to some more work and keep up to date it. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at least we will have the common plate. :) Yes, I love to the idea about having common code base. I've looked at both codes of github's (thinlet's) and Pivot's, Pivot's version has very different structure from github's (I think that is mainly for UI framework's requirement.) So it seems to be difficult to directly fork github's version to develop Pivot's version..., but I think I (or any other developers) could catch up changes in github's version. There's long way to go for Pivot's version, of course, I'd like to also make pull requests to enhance github's version if I can. Thanks, Tomoko 2015-02-24 23:34 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi, Tomoko! Thanks for being a fan of luke! Current status of github's luke ( https://github.com/DmitryKey/luke ) is that it has releases for all the major lucene versions since 4.3.0, excluding 4.4.0 (luke 4.5.0 should be able open indices of 4.4.0) and the latest -- 5.0.0. Porting the github's luke to ALv2 compliant framework (GWT or Pivot) is a long standing goal. With GWT I had issues related to listing and reading the index directory. So
Re: qt.shards in solrconfig.xml
Hello, Giving http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3c711daae5-c366-4349-b644-8e29e80e2...@gmail.com%3E you can add qt.shards into handler defaults/invariants. On Thu, Feb 26, 2015 at 5:40 PM, Benson Margulies bimargul...@gmail.com wrote: A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: qt.shards in solrconfig.xml
I was hoping that Benson was hinting at adding a qt.shards.auto=true parameter to so that would magically use on the path from the incoming request - and that this would be the default, since that's what most people would expect. Or, maybe just add a commented-out custom handler that has the qt.shards parameter as suggested, to re-emphasize to people that if they want to use a custom handler in distributed mode, then they will most likely need this parameter. -- Jack Krupansky On Thu, Feb 26, 2015 at 11:28 AM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, Giving http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3c711daae5-c366-4349-b644-8e29e80e2...@gmail.com%3E you can add qt.shards into handler defaults/invariants. On Thu, Feb 26, 2015 at 5:40 PM, Benson Margulies bimargul...@gmail.com wrote: A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Collations are not working fine.
Below is the filed definition that we used its just a basic definition :: analyzer type=index tokenizer class=solr.ClassicTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ /analyzer analyzer type=query tokenizer class=solr.ClassicTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=false / filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ /analyzer *Rajesh.* On Thu, Feb 26, 2015 at 2:03 AM, Nitin Solanki nitinml...@gmail.com wrote: Hi Rajesh, What configuration had you set in your schema.xml? On Sat, Feb 14, 2015 at 2:18 AM, Rajesh Hazari rajeshhaz...@gmail.com wrote: Hi Nitin, Can u try with the below config, we have these config seems to be working for us. searchComponent name=spellcheck class=solr.SpellCheckComponent str name=queryAnalyzerFieldTypetext_general/str lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldtextSpell/str str name=combineWordstrue/str str name=breakWordsfalse/str int name=maxChanges5/int /lst lst name=spellchecker str name=namedefault/str str name=fieldtextSpell/str str name=classnamesolr.IndexBasedSpellChecker/str str name=spellcheckIndexDir./spellchecker/str str name=accuracy0.75/str float name=thresholdTokenFrequency0.01/float str name=buildOnCommittrue/str str name=spellcheck.maxResultsForSuggest5/str /lst /searchComponent str name=spellchecktrue/str str name=spellcheck.dictionarydefault/str str name=spellcheck.dictionarywordbreak/str int name=spellcheck.count5/int str name=spellcheck.alternativeTermCount15/str str name=spellcheck.collatetrue/str str name=spellcheck.onlyMorePopularfalse/str str name=spellcheck.extendedResultstrue/str str name =spellcheck.maxCollations100/str str name=spellcheck.collateParam.mm100%/str str name=spellcheck.collateParam.q.opAND/str str name=spellcheck.maxCollationTries1000/str *Rajesh.* On Fri, Feb 13, 2015 at 1:01 PM, Dyer, James james.d...@ingramcontent.com wrote: Nitin, Can you post the full spellcheck response when you query: q=gram_ci:gone wthh thes wintwt=jsonindent=trueshards.qt=/spell James Dyer Ingram Content Group -Original Message- From: Nitin Solanki [mailto:nitinml...@gmail.com] Sent: Friday, February 13, 2015 1:05 AM To: solr-user@lucene.apache.org Subject: Re: Collations are not working fine. Hi James Dyer, I did the same as you told me. Used WordBreakSolrSpellChecker instead of shingles. But still collations are not coming or working. For instance, I tried to get collation of gone with the wind by searching gone wthh thes wint on field=gram_ci but didn't succeed. Even, I am getting the suggestions of wtth as *with*, thes as *the*, wint as *wind*. Also I have documents which contains gone with the wind having 167 times in the documents. I don't know that I am missing something or not. Please check my below solr configuration: *URL: *localhost:8983/solr/wikingram/spell?q=gram_ci:gone wthh thes wintwt=jsonindent=trueshards.qt=/spell *solrconfig.xml:* searchComponent name=spellcheck class=solr.SpellCheckComponent str name=queryAnalyzerFieldTypetextSpellCi/str lst name=spellchecker str name=namedefault/str str name=fieldgram_ci/str str name=classnamesolr.DirectSolrSpellChecker/str str name=distanceMeasureinternal/str float name=accuracy0.5/float int name=maxEdits2/int int name=minPrefix0/int int name=maxInspections5/int int name=minQueryLength2/int float name=maxQueryFrequency0.9/float str name=comparatorClassfreq/str /lst lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldgram/str str name=combineWordstrue/str str name=breakWordstrue/str int name=maxChanges5/int /lst /searchComponent requestHandler name=/spell class=solr.SearchHandler startup=lazy lst name=defaults str name=dfgram_ci/str str name=spellcheck.dictionarydefault/str str name=spellcheckon/str str name=spellcheck.extendedResultstrue/str str name=spellcheck.count25/str str name=spellcheck.onlyMorePopulartrue/str str name=spellcheck.maxResultsForSuggest1/str
Solr Backup Strategy
What is the best backup and restore strategy for Solr 3.6.1?
Re: Getting started with Solr
Oh, I see. I used the start -e cloud command, then ran through a setup with one core and default options for the rest, then tried to post the json example again, and got another error: buntu@ubuntu-VirtualBox:~/crawler/solr$ bin/post -c gettingstarted example/exampledocs/*.json /usr/lib/jvm/java-7-oracle/bin/java -classpath /home/ubuntu/crawler/solr/dist/solr-core-5.0.0.jar -Dauto=yes -Dc=gettingstarted -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/books.json SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/gettingstarted/update... Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log POSTing file books.json (application/json) to [base] SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: http://localhost:8983/solr/gettingstarted/update SimplePostTool: WARNING: Response: html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 404 Not Found/title /head bodyh2HTTP ERROR 404/h2 pProblem accessing /solr/gettingstarted/update. Reason: preNot Found/pre/phr /ismallPowered by Jetty:///small/ibr/ Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype On Thu, Feb 26, 2015 at 4:07 PM, Erik Hatcher erik.hatc...@gmail.com wrote: How did you start Solr? If you started with `bin/solr start -e cloud` you’ll have a gettingstarted collection created automatically, otherwise you’ll need to create it yourself with `bin/solr create -c gettingstarted` — Erik Hatcher, Senior Solutions Architect http://www.lucidworks.com http://www.lucidworks.com/ On Feb 26, 2015, at 4:53 AM, Baruch Kogan bar...@sellerpanda.com wrote: Hi, I've just installed Solr (will be controlling with Solarium and using to search Nutch queries.) I'm working through the starting tutorials described here: https://cwiki.apache.org/confluence/display/solr/Running+Solr When I try to run $ bin/post -c gettingstarted example/exampledocs/*.json, I get a bunch of errors having to do with there not being a gettingstarted folder in /solr/. Is this normal? Should I create one? Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype
Re: Unable to find query result in solr 5.0.0
What data did you have in the 1.3 version? Because the bin/solr -e techproduts process only indexes 30+ docs total. So if your 1.3 installation is returning more docs as your note seems to imply, you somehow have a lot more docs indexed. There is no mention of hotel in any of the sample docs that are indexed by the 'techproducts' target, so I suspect you're comparing apples to oranges. Not to mention that there are many, many changes to both the code and the sample data since 1.3 so whether the results are exactly comparable or not is highly questionable. Best, Erick On Thu, Feb 26, 2015 at 4:30 AM, Jack Krupansky jack.krupan...@gmail.com wrote: Does a query for *:* return all documents? Pick one of those documents and try a query using a field name and the value of that field for one of the documents and see if that document is returned. Maybe you skipped a step in the tutorial process or maybe there was an error that you ignored. Please confirm which doc you were reading for the tutorial steps. -- Jack Krupansky On Thu, Feb 26, 2015 at 6:17 AM, rupak rupak.d2...@gmail.com wrote: Hi, I am new in Solr and using Solr 5.0.0 search server. After installing when I’m going to search any keyword in solr 5.0.0 it dose not give any results back. But when I was using a previous version of Solr (1.3.0)(previously installed) it gives each and every results of the queried Keyword. For Example: In previous version (1.3.0) when I’m searching with any keyword like “Hotel”, “Motel”, “Television” , “i-pod” , “Books”, “cricket” etc in Query String section, it gives all search results with large number of records as a XML output. But in Solr 5.0.0 I start up with techproducts core (bin/solr -e techproducts) and then going to search keywords like “Television” , “i-pod” etc then it gives 2 or 3 results and also if we going to search any others keyword like “Hotel”, “Motel” it dose not return back any results. Also if we start up with cloud by bin/solr start -e cloud -noprompt it dose not gives any results. Also when we are going to use ‘POST’ tools by executing post.jar in command prompt says an error that this is not a valid command. Currently I’m unable to find any keyword. Please help me to query any string keyword from solr 5.0.0. Thanks Regards, Rupak Das -- View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-find-query-result-in-solr-5-0-0-tp4189196.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: New leader/replica solution for HDFS
I’ll be working on this at some point: https://issues.apache.org/jira/browse/SOLR-6237 - Mark http://about.me/markrmiller On Feb 25, 2015, at 2:12 AM, longsan longsan...@sina.com wrote: We used HDFS as our Solr index storage and we really have a heavy update load. We had met much problems with current leader/replica solution. There is duplicate index computing on Replilca side. And the data sync between leader/replica is always a problem. As HDFS already provides data replication on data layer, could Solr provide just service layer replication? My thought is that the leader and the replica all bind to the same data index directory. And the leader will build up index for new request, the replica will just keep update the index version with the leader(such as a soft commit periodically? ). If the leader lost then the replica will take the duty immediately. Thanks for any suggestion of this idea. -- View this message in context: http://lucene.472066.n3.nabble.com/New-leader-replica-solution-for-HDFS-tp4188735.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Document expiration with TTL
Hi
Thanks for your quick reply.
since your time_to_live_s and expire_at_dt fields are both
stored, can you confirm that a expire_at_dt field is getting popularted by
the update processor by doing as simple query for your doc (ie
q=id:10seconds)
No, expire_at_dt field does not get populated when we have added document
with the TTL defined in the TTL field. Like with following query,
curl -X POST -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/update?commit=true' -d
'[{id:10seconds,time_to_live_s:+10SECONDS}]'
and when document retrieved, it gives following result (Can see that
expire_at_dt field is not showing at all).
curl -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/select?q=id:10secondswt=jsonindent=true
'
{
responseHeader:{
status:0,
QTime:19,
params:{
indent:true,
q:id:10seconds,
wt:json}},
response:{numFound:1,start:0,docs:[
{
id:10seconds,
time_to_live_s:+10SECONDS,
_version_:1494171978430414848}]
}}
While if document is added with the TTL value defined explicitly in
expire_at_dt field, like,
curl -X POST -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/update?commit=true' -d
'[{id:10seconds,expire_at_dt:NOW+10SECONDS}]'
We can see the document with expire_at_dt field populated.
curl -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/select?q=id:10secondswt=jsonindent=true
'
{
responseHeader:{
status:0,
QTime:2,
params:{
indent:true,
q:id:10seconds,
wt:json}},
response:{numFound:1,start:0,docs:[
{
id:10seconds,
expire_at_dt:2015-02-26T12:27:31.983Z,
_version_:1494172190095966208}]
}}
Thanks,
Makailol
On Wed, Feb 25, 2015 at 10:00 PM, Chris Hostetter hossman_luc...@fucit.org
wrote:
: Following query posts a document and sets expire_at_dt explicitly. That
: is working perfectly ok and ducument expires at defined time.
so the delete trigge logic is working correctly...
: But when trying to post with TTL (following query), document does not
: expire after given time.
...which suggests that the TTL-expire_at logic is not being applied
properly.
which is weird.
since your time_to_live_s and expire_at_dt fields are both
stored, can you confirm that a expire_at_dt field is getting popularted by
the update processor by doing as simple query for your doc (ie
q=id:10seconds)
(either way: i can't explain why it's not getting deleted, but it would
help narrow down where the problem is)
-Hoss
http://www.lucidworks.com/
Re: Can't index all docs in a local folder with DIH in Solr 5.0.0
Alex, Same results on recursive=true / recursive=false. I also tried importing plain text files instead of epub (still using TikeEntityProcessor though) and get exactly the same result - ie. all files fetched, but only one document indexed in Solr. With verbose output, I get a row for each file in the directory, but only the first one has a non-empty documentImport entity. All subsequent documentImport entities just have an empty document#2 entry. eg: verbose-output: [ entity:files, [ null, --- row #1-, fileSize, 2609004, fileLastModified, 2015-02-25T11:37:25.217Z, fileAbsolutePath, c:\\Users\\gt\\Documents\\epub\\issue018.epub, fileDir, c:\\Users\\gt\\Documents\\epub, file, issue018.epub, null, -, entity:documentImport, [ document#1, [ query, c:\\Users\\gt\\Documents\\epub\\issue018.epub, time-taken, 0:0:0.0, null, --- row #1-, text, ... parsed epub text - snip ... title, Issue 18 title, Author, Author text, null, - ], document#2, [] ], null, --- row #2-, fileSize, 4428804, fileLastModified, 2015-02-25T11:37:36.399Z, fileAbsolutePath, c:\\Users\\gt\\Documents\\epub\\issue019.epub, fileDir, c:\\Users\\gt\\Documents\\epub, file, issue019.epub, null, -, entity:documentImport, [ document#2, [] ], null, --- row #3-, fileSize, 2580266, fileLastModified, 2015-02-25T11:37:41.188Z, fileAbsolutePath, c:\\Users\\gt\\Documents\\epub\\issue020.epub, fileDir, c:\\Users\\gt\\Documents\\epub, file, issue020.epub, null, -, entity:documentImport, [ document#2, [] ],
Re: Solr takes time to start
On 2/26/2015 12:11 AM, Nitin Solanki wrote: Why Solr is taking too much of time to start all nodes/ports? Very slow Solr startup is typically caused by one of two things. Both are described here: https://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup There could be other causes, but one of these two are usually the culprit. Thanks, Shawn
Re: Can't index all docs in a local folder with DIH in Solr 5.0.0
Alex, That's great. Thanks for the pointers. I'll try and get more info on this and file a JIRA issue. Kind regards, Gary. On 26/02/2015 14:16, Alexandre Rafalovitch wrote: On 26 February 2015 at 08:32, Gary Taylor g...@inovem.com wrote: Alex, Same results on recursive=true / recursive=false. I also tried importing plain text files instead of epub (still using TikeEntityProcessor though) and get exactly the same result - ie. all files fetched, but only one document indexed in Solr. To me, this would indicate that something is a problem with the inner DIH entity then. As a next set of steps, I would probably 1) remove both onError statements and see if there is an exception that is being swallowed. 2) run the import under ProcessMonitor and see if the other files are actually being read https://technet.microsoft.com/en-us/library/bb896645.aspx 3) Assume a Windows bug and test this on Mac/Linux 4) File a JIRA with a replication case. If there is a full replication setup, I'll test it machines I have access to with full debugger step-through For example, I wonder if FileBinDataSource is somehow not cleaning up after the first file properly on Windows and fails to open the second one. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ -- Gary Taylor | www.inovem.com | www.kahootz.com INOVEM Ltd is registered in England and Wales No 4228932 Registered Office 1, Weston Court, Weston, Berkshire. RG20 8JE kahootz.com is a trading name of INOVEM Ltd.
qt.shards in solrconfig.xml
A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in?
Re: Solr Document expiration with TTL
If your expire_at_dt field is not populated automatically, let's step back and recheck a sanity setting. You said it is a managed schema? Is it a schemaless as well? With an explicit processor chain? If that's the case, your default chain may not be running AT ALL. So, recheck your solrconfig.xml. Or add another explicit field population inside the chain, just like the example did with TimestampUpdateProcessorFactory : https://lucidworks.com/blog/document-expiration/ Regards, Alex. On 26 February 2015 at 07:52, Makailol Charls 4extrama...@gmail.com wrote: since your time_to_live_s and expire_at_dt fields are both stored, can you confirm that a expire_at_dt field is getting popularted by the update processor by doing as simple query for your doc (ie q=id:10seconds) No, expire_at_dt field does not get populated when we have added document with the TTL defined in the TTL field. Like with following query, Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/
Re: [ANNOUNCE] Luke 4.10.3 released
Sure, it is: java version 1.7.0_76 Java(TM) SE Runtime Environment (build 1.7.0_76-b13) Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode) On Thu, Feb 26, 2015 at 2:39 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Sorry, I'm afraid I have not encountered such errors when launch. Seems something wrong around Pivot's, but I have no idea about it. Would you tell me java version you're using ? Tomoko 2015-02-26 21:15 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Thanks, Tomoko, it compiles ok! Now launching produces some errors: $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication Exception in thread main java.lang.ExceptionInInitializerError at org.apache.lucene.luke.ui.LukeApplication.main(Unknown Source) Caused by: java.lang.NumberFormatException: For input string: 3 1644336 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:492) at java.lang.Byte.parseByte(Byte.java:148) at java.lang.Byte.parseByte(Byte.java:174) at org.apache.pivot.util.Version.decode(Version.java:156) at org.apache.pivot.wtk.ApplicationContext.clinit(ApplicationContext.java:1704) ... 1 more On Thu, Feb 26, 2015 at 1:48 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thank you for checking out it! Sorry, I've forgot to note important information... ivy jar is needed to compile. Packaging process needs to be organized, but for now, I'm borrowing it from lucene's tools/lib. In my environment, Fedora 20 and OpenJDK 1.7.0_71, it can be compiled and run as follows. If there are any problems, please let me know. $ svn co http://svn.apache.org/repos/asf/lucene/sandbox/luke/ $ cd luke/ // copy ivy jar to lib/tools $ cp /path/to/lucene_solr_4_10_3/lucene/tools/lib/ivy-2.3.0.jar lib/tools/ $ ls lib/tools/ ivy-2.3.0.jar $ java -version java version 1.7.0_71 OpenJDK Runtime Environment (fedora-2.5.3.3.fc20-x86_64 u71-b14) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) $ ant ivy-resolve ... BUILD SUCCESSFUL // compile and make jars and run $ ant dist ... BUILD SUCCESSFULL $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication ... Thanks, Tomoko 2015-02-26 16:39 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi Tomoko, Thanks for the link. Do you have build instructions somewhere? When I executed ant with no params, I get: BUILD FAILED /home/dmitry/projects/svn/luke/build.xml:40: /home/dmitry/projects/svn/luke/lib-ivy does not exist. On Thu, Feb 26, 2015 at 2:27 AM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thanks! Would you announce at LUCENE-2562 to me and all watchers interested in this issue, when the branch is ready? :) As you know, current pivots's version (that supports Lucene 4.10.3) is here. http://svn.apache.org/repos/asf/lucene/sandbox/luke/ Regards, Tomoko 2015-02-25 18:37 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Ok, sure. The plan is to make the pivot branch in the current github repo and update its structure accordingly. Once it is there, I'll let you know. Thank you, Dmitry On Tue, Feb 24, 2015 at 5:26 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Hi Dmitry, Thank you for the detailed clarification! Recently, I've created a few patches to Pivot version(LUCENE-2562), so I'd like to some more work and keep up to date it. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at least we will have the common plate. :) Yes, I love to the idea about having common code base. I've looked at both codes of github's (thinlet's) and Pivot's, Pivot's version has very different structure from github's (I think that is mainly for UI framework's requirement.) So it seems to be difficult to directly fork github's version to develop Pivot's version..., but I think I (or any other developers) could catch up changes in github's version. There's long way to go for Pivot's version, of course, I'd like to also make pull requests to enhance github's version if I can. Thanks, Tomoko 2015-02-24 23:34 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi, Tomoko! Thanks for being a fan of luke! Current status of github's luke ( https://github.com/DmitryKey/luke ) is
Re: Getting started with Solr
How did you start Solr? If you started with `bin/solr start -e cloud` you’ll have a gettingstarted collection created automatically, otherwise you’ll need to create it yourself with `bin/solr create -c gettingstarted` — Erik Hatcher, Senior Solutions Architect http://www.lucidworks.com http://www.lucidworks.com/ On Feb 26, 2015, at 4:53 AM, Baruch Kogan bar...@sellerpanda.com wrote: Hi, I've just installed Solr (will be controlling with Solarium and using to search Nutch queries.) I'm working through the starting tutorials described here: https://cwiki.apache.org/confluence/display/solr/Running+Solr When I try to run $ bin/post -c gettingstarted example/exampledocs/*.json, I get a bunch of errors having to do with there not being a gettingstarted folder in /solr/. Is this normal? Should I create one? Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype
Re: Can't index all docs in a local folder with DIH in Solr 5.0.0
On 26 February 2015 at 08:32, Gary Taylor g...@inovem.com wrote: Alex, Same results on recursive=true / recursive=false. I also tried importing plain text files instead of epub (still using TikeEntityProcessor though) and get exactly the same result - ie. all files fetched, but only one document indexed in Solr. To me, this would indicate that something is a problem with the inner DIH entity then. As a next set of steps, I would probably 1) remove both onError statements and see if there is an exception that is being swallowed. 2) run the import under ProcessMonitor and see if the other files are actually being read https://technet.microsoft.com/en-us/library/bb896645.aspx 3) Assume a Windows bug and test this on Mac/Linux 4) File a JIRA with a replication case. If there is a full replication setup, I'll test it machines I have access to with full debugger step-through For example, I wonder if FileBinDataSource is somehow not cleaning up after the first file properly on Windows and fails to open the second one. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/
Re: Solr Document expiration with TTL
Hi Alex,
Thanks for the reply.
Yes, we have already tried to set the autoDeletePeriodSeconds period to
some low value like 5 seconds and tried checking the document expiration
after 30 seconds or minute or even after an hour. But result is same and
document does not get expired automatically.
Thanks,
Makailol
On Thu, Feb 26, 2015 at 6:22 PM, Makailol Charls 4extrama...@gmail.com
wrote:
Hi
Thanks for your quick reply.
since your time_to_live_s and expire_at_dt fields are both
stored, can you confirm that a expire_at_dt field is getting popularted by
the update processor by doing as simple query for your doc (ie
q=id:10seconds)
No, expire_at_dt field does not get populated when we have added document
with the TTL defined in the TTL field. Like with following query,
curl -X POST -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/update?commit=true' -d
'[{id:10seconds,time_to_live_s:+10SECONDS}]'
and when document retrieved, it gives following result (Can see that
expire_at_dt field is not showing at all).
curl -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/select?q=id:10secondswt=jsonindent=true
'
{
responseHeader:{
status:0,
QTime:19,
params:{
indent:true,
q:id:10seconds,
wt:json}},
response:{numFound:1,start:0,docs:[
{
id:10seconds,
time_to_live_s:+10SECONDS,
_version_:1494171978430414848}]
}}
While if document is added with the TTL value defined explicitly in
expire_at_dt field, like,
curl -X POST -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/update?commit=true' -d
'[{id:10seconds,expire_at_dt:NOW+10SECONDS}]'
We can see the document with expire_at_dt field populated.
curl -H 'Content-Type: application/json' '
http://localhost:8983/solr/collection1/select?q=id:10secondswt=jsonindent=true
'
{
responseHeader:{
status:0,
QTime:2,
params:{
indent:true,
q:id:10seconds,
wt:json}},
response:{numFound:1,start:0,docs:[
{
id:10seconds,
expire_at_dt:2015-02-26T12:27:31.983Z,
_version_:1494172190095966208}]
}}
Thanks,
Makailol
On Wed, Feb 25, 2015 at 10:00 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
: Following query posts a document and sets expire_at_dt explicitly.
That
: is working perfectly ok and ducument expires at defined time.
so the delete trigge logic is working correctly...
: But when trying to post with TTL (following query), document does not
: expire after given time.
...which suggests that the TTL-expire_at logic is not being applied
properly.
which is weird.
since your time_to_live_s and expire_at_dt fields are both
stored, can you confirm that a expire_at_dt field is getting popularted by
the update processor by doing as simple query for your doc (ie
q=id:10seconds)
(either way: i can't explain why it's not getting deleted, but it would
help narrow down where the problem is)
-Hoss
http://www.lucidworks.com/
Re: Solr Document expiration with TTL
: If your expire_at_dt field is not populated automatically, let's step : back and recheck a sanity setting. You said it is a managed schema? Is : it a schemaless as well? With an explicit processor chain? If that's : the case, your default chain may not be running AT ALL. yeah ... my only guess here is that even though you posted before that you had this configured in your defaut chain... processor class=solr.processor.DocExpirationUpdateProcessorFactory int name=autoDeletePeriodSeconds30/int str name=ttlFieldNametime_to_live_s/str str name=expirationFieldNameexpire_at_dt/str /processor ...perhaps you have an update.chain=foo type default param configured for your /update handler? * what does your /update requestHandler/ config look like? * are you using the new initParams/ feature of solr? what does it's config look like? : So, recheck your solrconfig.xml. Or add another explicit field : population inside the chain, just like the example did with : TimestampUpdateProcessorFactory : : https://lucidworks.com/blog/document-expiration/ yeah ... that would help as a sanity check as well ... point is: we need to verify which chain you are using when adding the doc. : : : : : Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: : http://www.solr-start.com/ : -Hoss http://www.lucidworks.com/
Re: New leader/replica solution for HDFS
Great! Thank you! I had a 4 shard setup - no replicas. Index size was 2.0TBytes stored in HDFS with each node having approximately 500G of index. I added four more shards on four other machines as replicas. One thing that happened was the 4 replicas all ran out of HDFS cache size (SnapPull failed: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding - XX:MaxDirectMemorySize=sizeg -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating) I was using 160 slabs (20GBytes iof RAM). I dropped the config to 80 slabs and restarted the replicas. Two of the replicas came up OK, but the other 2 have stayed in 'Recovering'. I stopped those two and restarted them - now I have 3 OK, but one is still in Recovering. Given that each replica does indexing as well, I was expecting the amount of HDFS disk usage to double, but that has not happened. Once I get the last replica to come up, I'll run some tests. -Joe On 2/26/2015 10:45 AM, Mark Miller wrote: I’ll be working on this at some point: https://issues.apache.org/jira/browse/SOLR-6237 - Mark http://about.me/markrmiller On Feb 25, 2015, at 2:12 AM, longsan longsan...@sina.com wrote: We used HDFS as our Solr index storage and we really have a heavy update load. We had met much problems with current leader/replica solution. There is duplicate index computing on Replilca side. And the data sync between leader/replica is always a problem. As HDFS already provides data replication on data layer, could Solr provide just service layer replication? My thought is that the leader and the replica all bind to the same data index directory. And the leader will build up index for new request, the replica will just keep update the index version with the leader(such as a soft commit periodically? ). If the leader lost then the replica will take the duty immediately. Thanks for any suggestion of this idea. -- View this message in context: http://lucene.472066.n3.nabble.com/New-leader-replica-solution-for-HDFS-tp4188735.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr collection unavailable after reload
This is very, very strange. How are you indexing the docs? SolrJ? XML? DIH? What happens if you _only_ index the doc? You say nothing comes out in the log file indicating an error, but what _does_ come out? Particularly at the end? And in general note that attachments don't come through the Apache mailing lists, to show them you usually have to post it somewhere and provide a link. You _might_ be hitting the problem where the suggester takes a long time to build on startup, in which case you'll be seeing a message at the end of your log when you can't start things about building. If you wait _very_ patiently it'll eventually complete. This was commented out of solrconfig.xml stock distros in 4.10.3, see: https://issues.apache.org/jira/browse/SOLR-6679. And if your suggester is configured with anything like build on commit, it'll happen whenever you index. Anyway, this is consistent with the problems you've reported, so at least there's a chance it's the issue. Best, Erick On Thu, Feb 26, 2015 at 10:40 AM, akira...@libero.it akira...@libero.it wrote: Hi everybody, hi have a very strange issue in my solr (version 4.10.2) installation (on a windows server 2008 java jdk 1.7). I am sure nobody met this problem before (at least googling around i found nothing). I have a simple configuration with a base text_general field. I was able to index any kind of text without any problem. After some days i got a strange behavior. After restarting my server i was not able to load my collection My server started without error messages in my log but, trying to select my collection, web app hangs (as in picture) I googled around to find a way to debug such situation, or a way to fix collection itself (but found nothing). I am able to produce such error trying to index word wasting in my text_general field ( i know that problem is not related to this wordbut is the only way to produce such behavior). The only thing i can do is stop solr, delete index folder content and restart solr. Then i try again to index a document with wasting word, reload collection, and the problem is there again. Please someone could at least tell me a way to investigate
Solr collection unavailable after reload
Hi everybody, hi have a very strange issue in my solr (version 4.10.2) installation (on a windows server 2008 java jdk 1.7). I am sure nobody met this problem before (at least googling around i found nothing). I have a simple configuration with a base text_general field. I was able to index any kind of text without any problem. After some days i got a strange behavior. After restarting my server i was not able to load my collection My server started without error messages in my log but, trying to select my collection, web app hangs (as in picture) I googled around to find a way to debug such situation, or a way to fix collection itself (but found nothing). I am able to produce such error trying to index word wasting in my text_general field ( i know that problem is not related to this wordbut is the only way to produce such behavior). The only thing i can do is stop solr, delete index folder content and restart solr. Then i try again to index a document with wasting word, reload collection, and the problem is there again. Please someone could at least tell me a way to investigate
Leading Wildcard Support (ReversedWildcardFilterFactory)
All, I am currently using 4.10.3 running Solr Cloud. I have configured my index analyzer to leverage the solr.ReversedWildcardFilterFactory with various settings for the maxFractionAsterisk, maxPosAsterisk,etc. Currently I am running with the defaults (ie not configured) Using the Analysis capability in the Solr admin I see the Field Value (Index) fields going in correctly, both normal order and reversed order. However, on the Field Value (Query) side it is not generating a token that is reversed as expected (no matter where I place the * in the leading position of the search term). I also confirmed through the Query capability with debugQuery turned on that the parsed query is not reversed as expected. From my current understanding you do not need to have anything configured on the index analyzer to make leading wildcards work as expected with the reversedwildcardfilterfactory. The default query parser will know to look at the index analyzer and leverage the ReversedWildcardFilterFactory configuration if the term contains a leading wildcard. (This is what I have read) Without uploading my entire configuration to this email I was hoping someone could point me in the right direction because I am at a loss at this point. Thanks!
Re: Leading Wildcard Support (ReversedWildcardFilterFactory)
Please post your field type... or at least confirm a comparison to the example in the javadoc: http://lucene.apache.org/solr/4_10_3/solr-core/org/apache/solr/analysis/ReversedWildcardFilterFactory.html -- Jack Krupansky On Thu, Feb 26, 2015 at 2:38 PM, jaime spicciati jaime.spicci...@gmail.com wrote: All, I am currently using 4.10.3 running Solr Cloud. I have configured my index analyzer to leverage the solr.ReversedWildcardFilterFactory with various settings for the maxFractionAsterisk, maxPosAsterisk,etc. Currently I am running with the defaults (ie not configured) Using the Analysis capability in the Solr admin I see the Field Value (Index) fields going in correctly, both normal order and reversed order. However, on the Field Value (Query) side it is not generating a token that is reversed as expected (no matter where I place the * in the leading position of the search term). I also confirmed through the Query capability with debugQuery turned on that the parsed query is not reversed as expected. From my current understanding you do not need to have anything configured on the index analyzer to make leading wildcards work as expected with the reversedwildcardfilterfactory. The default query parser will know to look at the index analyzer and leverage the ReversedWildcardFilterFactory configuration if the term contains a leading wildcard. (This is what I have read) Without uploading my entire configuration to this email I was hoping someone could point me in the right direction because I am at a loss at this point. Thanks!
Re: Leading Wildcard Support (ReversedWildcardFilterFactory)
Most of the magic is done internal to the query parser which actually inspects the index analyzer chain when a leading wildcard is present. Look at the parsed_query in the debug response, and you should see that special prefix query. -- Jack Krupansky On Thu, Feb 26, 2015 at 3:49 PM, jaime spicciati jaime.spicci...@gmail.com wrote: Thanks for the quick response. The index I am currently testing with has the following configuration which is the default for the text_general_rev The field type is solr.TextField maxFractionAsterisk=.33 maxPosAsterisk=3 maxPosQuestion=2 withOriginal=true Through additional review I think it *might *be working as expected even though the Analysis tab and debugQuery parsed query lead me to think otherwise. If I look at the explain plan from the debugQuery and I actually get a hit, I see word/word(s) that actually come back in reversed order with the \u0001 prefix character, so the actual hit against the inverted index appears to be correct even though the parsed query doesn't reflect this. Is it safe to say that things are in fact working correctly? Thanks again On Thu, Feb 26, 2015 at 3:34 PM, Jack Krupansky jack.krupan...@gmail.com wrote: Please post your field type... or at least confirm a comparison to the example in the javadoc: http://lucene.apache.org/solr/4_10_3/solr-core/org/apache/solr/analysis/ReversedWildcardFilterFactory.html -- Jack Krupansky On Thu, Feb 26, 2015 at 2:38 PM, jaime spicciati jaime.spicci...@gmail.com wrote: All, I am currently using 4.10.3 running Solr Cloud. I have configured my index analyzer to leverage the solr.ReversedWildcardFilterFactory with various settings for the maxFractionAsterisk, maxPosAsterisk,etc. Currently I am running with the defaults (ie not configured) Using the Analysis capability in the Solr admin I see the Field Value (Index) fields going in correctly, both normal order and reversed order. However, on the Field Value (Query) side it is not generating a token that is reversed as expected (no matter where I place the * in the leading position of the search term). I also confirmed through the Query capability with debugQuery turned on that the parsed query is not reversed as expected. From my current understanding you do not need to have anything configured on the index analyzer to make leading wildcards work as expected with the reversedwildcardfilterfactory. The default query parser will know to look at the index analyzer and leverage the ReversedWildcardFilterFactory configuration if the term contains a leading wildcard. (This is what I have read) Without uploading my entire configuration to this email I was hoping someone could point me in the right direction because I am at a loss at this point. Thanks!
Re: Leading Wildcard Support (ReversedWildcardFilterFactory)
Thanks for the quick response. The index I am currently testing with has the following configuration which is the default for the text_general_rev The field type is solr.TextField maxFractionAsterisk=.33 maxPosAsterisk=3 maxPosQuestion=2 withOriginal=true Through additional review I think it *might *be working as expected even though the Analysis tab and debugQuery parsed query lead me to think otherwise. If I look at the explain plan from the debugQuery and I actually get a hit, I see word/word(s) that actually come back in reversed order with the \u0001 prefix character, so the actual hit against the inverted index appears to be correct even though the parsed query doesn't reflect this. Is it safe to say that things are in fact working correctly? Thanks again On Thu, Feb 26, 2015 at 3:34 PM, Jack Krupansky jack.krupan...@gmail.com wrote: Please post your field type... or at least confirm a comparison to the example in the javadoc: http://lucene.apache.org/solr/4_10_3/solr-core/org/apache/solr/analysis/ReversedWildcardFilterFactory.html -- Jack Krupansky On Thu, Feb 26, 2015 at 2:38 PM, jaime spicciati jaime.spicci...@gmail.com wrote: All, I am currently using 4.10.3 running Solr Cloud. I have configured my index analyzer to leverage the solr.ReversedWildcardFilterFactory with various settings for the maxFractionAsterisk, maxPosAsterisk,etc. Currently I am running with the defaults (ie not configured) Using the Analysis capability in the Solr admin I see the Field Value (Index) fields going in correctly, both normal order and reversed order. However, on the Field Value (Query) side it is not generating a token that is reversed as expected (no matter where I place the * in the leading position of the search term). I also confirmed through the Query capability with debugQuery turned on that the parsed query is not reversed as expected. From my current understanding you do not need to have anything configured on the index analyzer to make leading wildcards work as expected with the reversedwildcardfilterfactory. The default query parser will know to look at the index analyzer and leverage the ReversedWildcardFilterFactory configuration if the term contains a leading wildcard. (This is what I have read) Without uploading my entire configuration to this email I was hoping someone could point me in the right direction because I am at a loss at this point. Thanks!
Re: Basic Multilingual search capability
Hi Tom, Thanks for your inputs. I was planning to use stopword filter, but will definitely make sure they are unique and not to step over each other. I think for our system even going with length of 50-75 should be fine, will definitely up that number after doing some analysis on our input. Just one clarification, when you say ICUFilterFactory am I correct in thinking its ICUFodingFilterFactory. Thanks, Rishi. -Original Message- From: Tom Burton-West tburt...@umich.edu To: solr-user solr-user@lucene.apache.org Sent: Wed, Feb 25, 2015 4:33 pm Subject: Re: Basic Multilingual search capability Hi Rishi, As others have indicated Multilingual search is very difficult to do well. At HathiTrust we've been using the ICUTokenizer and ICUFilterFactory to deal with having materials in 400 languages. We also added the CJKBigramFilter to get better precision on CJK queries. We don't use stop words because stop words in one language are content words in another. For example die in German is a stopword but it is a content word in English. Putting multiple languages in one index can affect word frequency statistics which make relevance ranking less accurate. So for example for the English query Die Hard the word die would get a low idf score because it occurs so frequently in German. We realize that our approach does not produce the best results, but given the 400 languages, and limited resources, we do our best to make search not suck for non-English languages. When we have the resources we are thinking about doing special processing for a small fraction of the top 20 languages. We plan to select those languages that most need special processing and relatively easy to disambiguate from other languages. If you plan on identifying languages (rather than scripts), you should be aware that most language detection libraries don't work well on short texts such as queries. If you know that you have scripts for which you have content in only one language, you can use script detection instead of language detection. If you have German, a filter length of 25 might be too low (Because of compounding). You might want to analyze a sample of your German text to find a good length. Tom http://www.hathitrust.org/blogs/Large-scale-Search On Wed, Feb 25, 2015 at 10:31 AM, Rishi Easwaran rishi.easwa...@aol.com wrote: Hi Alex, Thanks for the suggestions. These steps will definitely help out with our use case. Thanks for the idea about the lengthFilter to protect our system. Thanks, Rishi. -Original Message- From: Alexandre Rafalovitch arafa...@gmail.com To: solr-user solr-user@lucene.apache.org Sent: Tue, Feb 24, 2015 8:50 am Subject: Re: Basic Multilingual search capability Given the limited needs, I would probably do something like this: 1) Put a language identifier in the UpdateRequestProcessor chain during indexing and route out at least known problematic languages, such as Chinese, Japanese, Arabic into individual fields 2) Put everything else together into one field with ICUTokenizer, maybe also ICUFoldingFilter 3) At the very end of that joint filter, stick in LengthFilter with some high number, e.g. 25 characters max. This will ensure that super-long words from non-space languages and edge conditions do not break the rest of your system. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 23 February 2015 at 23:14, Walter Underwood wun...@wunderwood.org wrote: I understand relevancy, stemming etc becomes extremely complicated with multilingual support, but our first goal is to be able to tokenize and provide basic search capability for any language. Ex: When the document contains hello or здравствуйте, the analyzer creates tokens and provides exact match search results.
Re: Problem with queries that includes NOT
Hi,
I thought that we were using the edismax query parser, but it seems that
we had configured the dismax parser.
I have made some tests with the edismax parser and it works fine, so I'll
change it in our production Solr.
Regards,
David Dávila
DIT - 915828763
De: Alvaro Cabrerizo topor...@gmail.com
Para: solr-user@lucene.apache.org solr-user@lucene.apache.org,
Fecha: 25/02/2015 16:41
Asunto: Re: Problem with queries that includes NOT
Hi,
The edismax parser should be able to manage the query you want to ask.
I've
made a test and the next both queries give me the right result (see the
parenthesis):
- {!edismax}(NOT id:7 AND NOT id:8 AND id:9) (gives 1
hit
the id:9)
- {!edismax}((NOT id:7 AND NOT id:8) AND id:9) (gives 1
hit
the id:9)
In general, the issue appears when using the lucene query parser mixing
different boolean clauses (including NOT). Thus, as you commented, the
next
queries gives different result
- NOT id:7 AND NOT id:8 AND id:9 (gives 1 hit the
id:9)
- (NOT id:7 AND NOT id:8) AND id:9 (gives 0 hits when
expecting 1 )
Since I read the chapter Limitations of prohibited clauses in
sub-queries
from the Apache Solr 3 Enterprise Search Server many years ago, I
always
add the *all documents query clause *:** to the negative clauses to avoid
the problem you mentioned. Thus I will recommend to rewrite the query you
showed us as:
- (**:*: AND* NOT Proc:ID01 AND NOT FileType:PDF_TEXT) AND
sys_FileType:PROTOTIPE
- (NOT id:7 AND NOT id:8 *AND *:**) AND id:9 (gives 1 hit
as expected)
The above query can be read then as give me all the documents except those
having ID01 and PDF_TEXT and having PROTOTIPE
Regards.
On Wed, Feb 25, 2015 at 1:23 PM, Shawn Heisey apa...@elyograg.org wrote:
On 2/25/2015 4:04 AM, david.dav...@correo.aeat.es wrote:
We have problems with some queries. All of them include the tag NOT,
and
in my opinion, the results don´t make any sense.
First problem:
This query NOT Proc:ID01returns 95806 results, however this
one
NOT Proc:ID01 OR FileType:PDF_TEXT returns 11484 results. But it's
impossible that adding a tag OR the query has less number of results.
Second problem. Here the problem is because of the brackets and the
NOT
tag:
This query:
(NOT Proc:ID01 AND NOT FileType:PDF_TEXT) AND sys_FileType:PROTOTIPE
returns 0 documents.
But this query:
(NOT Proc:ID01 AND NOT FileType:PDF_TEXT AND sys_FileType:PROTOTIPE)
returns 53 documents, which is correct. So, the problem is the
position
of
the bracket. I have checked the same query without NOTs, and it works
fine
returning the same number of results in both cases. So, I think the
problem is the combination of the bracket positions and the NOT tag.
For the first query, there is a difference between NOT condition1 OR
condition2 and NOT (condition1 OR condition2) ... I can imagine the
first one increasing the document count compared to just NOT
condition1 ... the second one wouldn't increase it.
Boolean queries in Solr (and very likely Lucene as well) do not always
do what people expect.
http://robotlibrarian.billdueber.com/2011/12/solr-and-boolean-operators/
https://lucidworks.com/blog/why-not-and-or-and-not/
As mentioned in the second link above, you'll get better results if you
use the prefix operators with explicit parentheses. One word of
warning, though -- the prefix operators do not work correctly if you
change the default operator to AND.
Thanks,
Shawn
Re: Facet By Distance
Thank you for your replies, added q and it works! I agree the examples are
a bit confusing. It turned out also that points are clustered around the
center and had to increase d as well.
On Wed, Feb 25, 2015 at 11:46 PM, Alexandre Rafalovitch arafa...@gmail.com
wrote:
In the examples it used to default to *:* with default params, which
caused even more confusion.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 25 February 2015 at 15:21, david.w.smi...@gmail.com
david.w.smi...@gmail.com wrote:
If 'q' is absent, then you always match nothing (there may be
exceptions?);
so it's sort of required, in effect. I wish it defaulted to *:*.
~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, Feb 25, 2015 at 2:28 PM, Ahmed Adel ahmed.a...@badrit.com
wrote:
Hi,
Thank you for your reply. I added a filter query to the query in two
ways
as follows:
fq={!geofilt}sfield=start_stationpt=40.71754834,-74.01322069facet.query={!frange
l=0.0 u=0.1}geodist()facet.query={!frange l=0.10001
u=0.2}geodist()d=0.2
-- returns 0 docs
q=*:*fq={!geofilt}sfield=start_stationpt=40.71754834,-74.01322069d=0.2
-- returns 1484 docs
Not sure why the first query with returns 0 documents
On Wed, Feb 25, 2015 at 8:46 PM, david.w.smi...@gmail.com
david.w.smi...@gmail.com wrote:
Hi,
This will return all the documents in the index because you did
nothing
to filter them out. Your query is *:* (everything) and there are no
filter
queries.
~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, Feb 25, 2015 at 12:27 PM, Ahmed Adel ahmed.a...@badrit.com
wrote:
Hello,
I'm trying to get Facet By Distance working on an index with
LatLonType
fields. The schema is as follows:
fields
...
field name=trip_duration type=int indexed=true
stored=true/
field name=start_station type=location indexed=true
stored=true
/
field name=end_station type=location indexed=true
stored=true
/
field name=birth_year type=int stored=true/
field name=gender type=int stored=true /
...
/fields
And the query I'm running is:
q=*:*sfield=start_stationpt=40.71754834,-74.01322069facet.query={!frange
l=0.0 u=0.1}geodist()facet.query={!frange l=0.10001 u=0.2}geodist()
But it returns all the documents in the index so it seems something
is
missing. I'm using Solr 4.9.0.
--
A. Adel
A. Adel
--
A. Adel
Getting started with Solr
Hi, I've just installed Solr (will be controlling with Solarium and using to search Nutch queries.) I'm working through the starting tutorials described here: https://cwiki.apache.org/confluence/display/solr/Running+Solr When I try to run $ bin/post -c gettingstarted example/exampledocs/*.json, I get a bunch of errors having to do with there not being a gettingstarted folder in /solr/. Is this normal? Should I create one? Sincerely, Baruch Kogan Marketing Manager Seller Panda http://sellerpanda.com +972(58)441-3829 baruch.kogan at Skype
Re: [ANNOUNCE] Luke 4.10.3 released
Thank you for checking out it! Sorry, I've forgot to note important information... ivy jar is needed to compile. Packaging process needs to be organized, but for now, I'm borrowing it from lucene's tools/lib. In my environment, Fedora 20 and OpenJDK 1.7.0_71, it can be compiled and run as follows. If there are any problems, please let me know. $ svn co http://svn.apache.org/repos/asf/lucene/sandbox/luke/ $ cd luke/ // copy ivy jar to lib/tools $ cp /path/to/lucene_solr_4_10_3/lucene/tools/lib/ivy-2.3.0.jar lib/tools/ $ ls lib/tools/ ivy-2.3.0.jar $ java -version java version 1.7.0_71 OpenJDK Runtime Environment (fedora-2.5.3.3.fc20-x86_64 u71-b14) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) $ ant ivy-resolve ... BUILD SUCCESSFUL // compile and make jars and run $ ant dist ... BUILD SUCCESSFULL $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication ... Thanks, Tomoko 2015-02-26 16:39 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi Tomoko, Thanks for the link. Do you have build instructions somewhere? When I executed ant with no params, I get: BUILD FAILED /home/dmitry/projects/svn/luke/build.xml:40: /home/dmitry/projects/svn/luke/lib-ivy does not exist. On Thu, Feb 26, 2015 at 2:27 AM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thanks! Would you announce at LUCENE-2562 to me and all watchers interested in this issue, when the branch is ready? :) As you know, current pivots's version (that supports Lucene 4.10.3) is here. http://svn.apache.org/repos/asf/lucene/sandbox/luke/ Regards, Tomoko 2015-02-25 18:37 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Ok, sure. The plan is to make the pivot branch in the current github repo and update its structure accordingly. Once it is there, I'll let you know. Thank you, Dmitry On Tue, Feb 24, 2015 at 5:26 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Hi Dmitry, Thank you for the detailed clarification! Recently, I've created a few patches to Pivot version(LUCENE-2562), so I'd like to some more work and keep up to date it. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at least we will have the common plate. :) Yes, I love to the idea about having common code base. I've looked at both codes of github's (thinlet's) and Pivot's, Pivot's version has very different structure from github's (I think that is mainly for UI framework's requirement.) So it seems to be difficult to directly fork github's version to develop Pivot's version..., but I think I (or any other developers) could catch up changes in github's version. There's long way to go for Pivot's version, of course, I'd like to also make pull requests to enhance github's version if I can. Thanks, Tomoko 2015-02-24 23:34 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi, Tomoko! Thanks for being a fan of luke! Current status of github's luke (https://github.com/DmitryKey/luke ) is that it has releases for all the major lucene versions since 4.3.0, excluding 4.4.0 (luke 4.5.0 should be able open indices of 4.4.0) and the latest -- 5.0.0. Porting the github's luke to ALv2 compliant framework (GWT or Pivot) is a long standing goal. With GWT I had issues related to listing and reading the index directory. So this effort has been parked. Most recently I have been approaching the Pivot. Mark Miller has done an initial port, that I took as the basis. I'm hoping to continue on this track as time permits. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at least we will have the common plate. :) Thanks, Dmitry On Tue, Feb 24, 2015 at 4:02 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Hi, I'm an user / fan of Luke, so deeply appreciate your work. I've carefully read the readme, noticed the (one of) project's goal: To port the thinlet UI to an ASL compliant license framework so that it can be contributed back to Apache Lucene. Current work is done with GWT 2.5.1. There has been GWT based, ASL compliant Luke supporting the latest Lucene ? I've recently got in with LUCENE-2562. Currently, Apache Pivot based port is going. But I do not know so much about Luke's long (and may be slightly complex) history, so I would grateful if anybody clear the association of the Luke project (now on Github) and the Jira issue. Or, they can be independent of each other.
Unable to find query result in solr 5.0.0
Hi, I am new in Solr and using Solr 5.0.0 search server. After installing when I’m going to search any keyword in solr 5.0.0 it dose not give any results back. But when I was using a previous version of Solr (1.3.0)(previously installed) it gives each and every results of the queried Keyword. For Example: In previous version (1.3.0) when I’m searching with any keyword like “Hotel”, “Motel”, “Television” , “i-pod” , “Books”, “cricket” etc in Query String section, it gives all search results with large number of records as a XML output. But in Solr 5.0.0 I start up with techproducts core (bin/solr -e techproducts) and then going to search keywords like “Television” , “i-pod” etc then it gives 2 or 3 results and also if we going to search any others keyword like “Hotel”, “Motel” it dose not return back any results. Also if we start up with cloud by bin/solr start -e cloud -noprompt it dose not gives any results. Also when we are going to use ‘POST’ tools by executing post.jar in command prompt says an error that this is not a valid command. Currently I’m unable to find any keyword. Please help me to query any string keyword from solr 5.0.0. Thanks Regards, Rupak Das -- View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-find-query-result-in-solr-5-0-0-tp4189196.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: [ANNOUNCE] Luke 4.10.3 released
Thanks, Tomoko, it compiles ok! Now launching produces some errors: $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication Exception in thread main java.lang.ExceptionInInitializerError at org.apache.lucene.luke.ui.LukeApplication.main(Unknown Source) Caused by: java.lang.NumberFormatException: For input string: 3 1644336 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:492) at java.lang.Byte.parseByte(Byte.java:148) at java.lang.Byte.parseByte(Byte.java:174) at org.apache.pivot.util.Version.decode(Version.java:156) at org.apache.pivot.wtk.ApplicationContext.clinit(ApplicationContext.java:1704) ... 1 more On Thu, Feb 26, 2015 at 1:48 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thank you for checking out it! Sorry, I've forgot to note important information... ivy jar is needed to compile. Packaging process needs to be organized, but for now, I'm borrowing it from lucene's tools/lib. In my environment, Fedora 20 and OpenJDK 1.7.0_71, it can be compiled and run as follows. If there are any problems, please let me know. $ svn co http://svn.apache.org/repos/asf/lucene/sandbox/luke/ $ cd luke/ // copy ivy jar to lib/tools $ cp /path/to/lucene_solr_4_10_3/lucene/tools/lib/ivy-2.3.0.jar lib/tools/ $ ls lib/tools/ ivy-2.3.0.jar $ java -version java version 1.7.0_71 OpenJDK Runtime Environment (fedora-2.5.3.3.fc20-x86_64 u71-b14) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) $ ant ivy-resolve ... BUILD SUCCESSFUL // compile and make jars and run $ ant dist ... BUILD SUCCESSFULL $ java -cp dist/* org.apache.lucene.luke.ui.LukeApplication ... Thanks, Tomoko 2015-02-26 16:39 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi Tomoko, Thanks for the link. Do you have build instructions somewhere? When I executed ant with no params, I get: BUILD FAILED /home/dmitry/projects/svn/luke/build.xml:40: /home/dmitry/projects/svn/luke/lib-ivy does not exist. On Thu, Feb 26, 2015 at 2:27 AM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Thanks! Would you announce at LUCENE-2562 to me and all watchers interested in this issue, when the branch is ready? :) As you know, current pivots's version (that supports Lucene 4.10.3) is here. http://svn.apache.org/repos/asf/lucene/sandbox/luke/ Regards, Tomoko 2015-02-25 18:37 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Ok, sure. The plan is to make the pivot branch in the current github repo and update its structure accordingly. Once it is there, I'll let you know. Thank you, Dmitry On Tue, Feb 24, 2015 at 5:26 PM, Tomoko Uchida tomoko.uchida.1...@gmail.com wrote: Hi Dmitry, Thank you for the detailed clarification! Recently, I've created a few patches to Pivot version(LUCENE-2562), so I'd like to some more work and keep up to date it. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at least we will have the common plate. :) Yes, I love to the idea about having common code base. I've looked at both codes of github's (thinlet's) and Pivot's, Pivot's version has very different structure from github's (I think that is mainly for UI framework's requirement.) So it seems to be difficult to directly fork github's version to develop Pivot's version..., but I think I (or any other developers) could catch up changes in github's version. There's long way to go for Pivot's version, of course, I'd like to also make pull requests to enhance github's version if I can. Thanks, Tomoko 2015-02-24 23:34 GMT+09:00 Dmitry Kan solrexp...@gmail.com: Hi, Tomoko! Thanks for being a fan of luke! Current status of github's luke ( https://github.com/DmitryKey/luke ) is that it has releases for all the major lucene versions since 4.3.0, excluding 4.4.0 (luke 4.5.0 should be able open indices of 4.4.0) and the latest -- 5.0.0. Porting the github's luke to ALv2 compliant framework (GWT or Pivot) is a long standing goal. With GWT I had issues related to listing and reading the index directory. So this effort has been parked. Most recently I have been approaching the Pivot. Mark Miller has done an initial port, that I took as the basis. I'm hoping to continue on this track as time permits. If you would like to work on the Pivot version, may I suggest you to fork the github's version? The ultimate goal is to donate this to Apache, but at
Re: Unable to find query result in solr 5.0.0
Does a query for *:* return all documents? Pick one of those documents and try a query using a field name and the value of that field for one of the documents and see if that document is returned. Maybe you skipped a step in the tutorial process or maybe there was an error that you ignored. Please confirm which doc you were reading for the tutorial steps. -- Jack Krupansky On Thu, Feb 26, 2015 at 6:17 AM, rupak rupak.d2...@gmail.com wrote: Hi, I am new in Solr and using Solr 5.0.0 search server. After installing when I’m going to search any keyword in solr 5.0.0 it dose not give any results back. But when I was using a previous version of Solr (1.3.0)(previously installed) it gives each and every results of the queried Keyword. For Example: In previous version (1.3.0) when I’m searching with any keyword like “Hotel”, “Motel”, “Television” , “i-pod” , “Books”, “cricket” etc in Query String section, it gives all search results with large number of records as a XML output. But in Solr 5.0.0 I start up with techproducts core (bin/solr -e techproducts) and then going to search keywords like “Television” , “i-pod” etc then it gives 2 or 3 results and also if we going to search any others keyword like “Hotel”, “Motel” it dose not return back any results. Also if we start up with cloud by bin/solr start -e cloud -noprompt it dose not gives any results. Also when we are going to use ‘POST’ tools by executing post.jar in command prompt says an error that this is not a valid command. Currently I’m unable to find any keyword. Please help me to query any string keyword from solr 5.0.0. Thanks Regards, Rupak Das -- View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-find-query-result-in-solr-5-0-0-tp4189196.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr 5.0.0 on Windows Server
Hello, I'm deploying Solr5.0.0. on Windows 2008 server. I'm planning to add a task to the task scheduler to start the Solr server at system boot time. So I call the bin\solr.bat start options from the task scheduler. Is this the preferred method om Windows, because I read that running under Tomcat or Jetty is not supported any more. Regards, John Jenniskens (fairly new to Solr)
Re: qt.shards in solrconfig.xml
I apparently am feeling dense; the following does not worl. requestHandler name=/RNI class=solr.SearchHandler default=false list name=defaults str name=shards.qt/RNI/str /list arr name=components strname-indexing-query/str strname-indexing-rescore/str strfacet/str strmlt/str strhighlight/str strstats/str strdebug/str /arr /requestHandler On Thu, Feb 26, 2015 at 11:33 AM, Jack Krupansky jack.krupan...@gmail.com wrote: I was hoping that Benson was hinting at adding a qt.shards.auto=true parameter to so that would magically use on the path from the incoming request - and that this would be the default, since that's what most people would expect. Or, maybe just add a commented-out custom handler that has the qt.shards parameter as suggested, to re-emphasize to people that if they want to use a custom handler in distributed mode, then they will most likely need this parameter. -- Jack Krupansky On Thu, Feb 26, 2015 at 11:28 AM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, Giving http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3c711daae5-c366-4349-b644-8e29e80e2...@gmail.com%3E you can add qt.shards into handler defaults/invariants. On Thu, Feb 26, 2015 at 5:40 PM, Benson Margulies bimargul...@gmail.com wrote: A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Solr logs encoding
Hi, I've wired situation. Starting yesterday restart I've issue with log encoding. My log looks like: DEBUG - 2015-02-27 10:47:01.432; [0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xc7]8[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x89][0x5][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x97][0x4][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xa4][0x6][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfc]b[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfc]F[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xfb]:[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]a[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]v[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]Y[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]Y[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]V[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]H[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]U[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]\[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xe4][0x96][0x1][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]`[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]j[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]l[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]j[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]X[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]e[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xdd][0xba][0x1][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]h[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xb5][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xee][0x3][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]\[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xe2][0x1d][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xbb][0x1a][0x4][0xfc][0xff][0xff][0xff][0xf][0x4]c[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xd2]%[0x4][0xfc][0xff][0xff][0xff][0xf][0x4]b[0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0x92][0x1a][0x4][0xfc][0xff][0xff][0xff][0xf][0x4][0xa3][0x4][0x4][0xfc][0xff][0xff][0xff] Anyone familiar with this? How to fix it? Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati [KMS2]http://finance.yahoo.com/news/kms-lighthouse-named-gartner-cool-121000184.html More at: www.kmslh.comhttp://www.kmslh.com/ | LinkedInhttp://www.linkedin.com/company/kms-lighthouse | FBhttps://www.facebook.com/pages/KMS-lighthouse/123774257810917
Re: qt.shards in solrconfig.xml
If I'm reading your suggestion right, Tim fixed this for 5.1 with http://issues.apache.org/jira/browse/SOLR-6311 On Thu, Feb 26, 2015 at 10:03 PM, Jack Krupansky jack.krupan...@gmail.com wrote: I was hoping that Benson was hinting at adding a qt.shards.auto=true parameter to so that would magically use on the path from the incoming request - and that this would be the default, since that's what most people would expect. Or, maybe just add a commented-out custom handler that has the qt.shards parameter as suggested, to re-emphasize to people that if they want to use a custom handler in distributed mode, then they will most likely need this parameter. -- Jack Krupansky On Thu, Feb 26, 2015 at 11:28 AM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, Giving http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3c711daae5-c366-4349-b644-8e29e80e2...@gmail.com%3E you can add qt.shards into handler defaults/invariants. On Thu, Feb 26, 2015 at 5:40 PM, Benson Margulies bimargul...@gmail.com wrote: A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com -- Regards, Shalin Shekhar Mangar.
Re: qt.shards in solrconfig.xml
Hi Benson, Do not use shards.qt with a leading '/'. See https://issues.apache.org/jira/browse/SOLR-3161 for details. Also note that shards.qt will not be necessary with 5.1 and beyond because of SOLR-6311 On Fri, Feb 27, 2015 at 8:16 AM, Benson Margulies bimargul...@gmail.com wrote: I apparently am feeling dense; the following does not worl. requestHandler name=/RNI class=solr.SearchHandler default=false list name=defaults str name=shards.qt/RNI/str /list arr name=components strname-indexing-query/str strname-indexing-rescore/str strfacet/str strmlt/str strhighlight/str strstats/str strdebug/str /arr /requestHandler On Thu, Feb 26, 2015 at 11:33 AM, Jack Krupansky jack.krupan...@gmail.com wrote: I was hoping that Benson was hinting at adding a qt.shards.auto=true parameter to so that would magically use on the path from the incoming request - and that this would be the default, since that's what most people would expect. Or, maybe just add a commented-out custom handler that has the qt.shards parameter as suggested, to re-emphasize to people that if they want to use a custom handler in distributed mode, then they will most likely need this parameter. -- Jack Krupansky On Thu, Feb 26, 2015 at 11:28 AM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, Giving http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201301.mbox/%3c711daae5-c366-4349-b644-8e29e80e2...@gmail.com%3E you can add qt.shards into handler defaults/invariants. On Thu, Feb 26, 2015 at 5:40 PM, Benson Margulies bimargul...@gmail.com wrote: A query I posted yesterday amounted to me forgetting that I have to set qt.shards when I use a URL other than plain old '/select' with SolrCloud. Is there any way to configure a query handler to automate this, so that all queries addressed to '/RNI' get that added in? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com -- Regards, Shalin Shekhar Mangar.
