Exception while solr commit
Hello, I am calling commit for every record (document) added/updated to the index. Our number of records size is 50k. Getting the following exception during commit. Is it correct approach to call commit for every insert/update?. Apr 7, 2009 4:41:23 PM org.apache.solr.handler.dataimport.SolrWriter commit SEVERE: Exception while solr commit. java.lang.RuntimeException: after flush: fdx size mismatch: 20096 docs vs 65536 length in bytes of _6.fdx at org.apache.lucene.index.StoredFieldsWriter.closeDocStore(StoredFieldsWri ter.java:94) at org.apache.lucene.index.DocFieldConsumers.closeDocStore(DocFieldConsumer s.java:83) at org.apache.lucene.index.DocFieldProcessor.closeDocStore(DocFieldProcesso r.java:47) at org.apache.lucene.index.DocumentsWriter.closeDocStore(DocumentsWriter.ja va:367) at org.apache.lucene.index.IndexWriter.flushDocStores(IndexWriter.java:1774 ) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3600) at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4151) at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4031) at org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeSc heduler.java:176) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2485) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2332) at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2280) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2. java:355) at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpd ateProcessorFactory.java:77) at org.apache.solr.handler.dataimport.SolrWriter.commit(SolrWriter.java:180 ) at org.apache.solr.handler.dataimport.DocBuilder.commit(DocBuilder.java:168 ) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:15 2) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporte r.java:334) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java :386) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java: 377) Apr 7, 2009 4:41:23 PM org.apache.solr.handler.dataimport.DocBuilder execute Thanks. Karthik
Solr Search Error
Hi All, I am intermittently getting this Exception when I do the search. What could be the reason?. Caused by: org.apache.solr.common.SolrException: 11938 java.lang.ArrayIndexOutOfBoundsException: 11938 at org.apache.lucene.search.TermScorer.score(TermScorer.java:74)at org.apache.lucene.search.TermScorer.score(TermScorer.java:61)at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:137)at org.apache.lucene.search.Searcher.search(Searcher.java:126) at org.apache.lucene.search.Searcher.search(Searcher.java:105) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:619) Thanks. Karthik
RE: Solr Search Error
Hi Otis, Thanks for the recommendation. Will try with latest nightly build.. I did couple of full data import and got this error at few times while searching.. Thanks. Karthik -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Friday, March 27, 2009 12:57 PM To: solr-user@lucene.apache.org Subject: Re: Solr Search Error Hi Karthik, First thing I'd do is get the latest Solr nightly build. If that doesn't fix thing, I'd grab the latest Lucene nightly build and use it to replace Lucene jars that are in your version of Solr. If that doesn't work I'd email the ML with a bit more info about the type of search that causes this (e.g. Do all searches cause this or only some? What do those that trigger this error look like or have in common?) Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Narayanan, Karthikeyan karthikeyan.naraya...@gs.com To: solr-user@lucene.apache.org Sent: Friday, March 27, 2009 11:42:12 AM Subject: Solr Search Error Hi All, I am intermittently getting this Exception when I do the search. What could be the reason?. Caused by: org.apache.solr.common.SolrException: 11938 java.lang.ArrayIndexOutOfBoundsException: 11938 at org.apache.lucene.search.TermScorer.score(TermScorer.java:74) at org.apache.lucene.search.TermScorer.score(TermScorer.java:61) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:137) at org.apache.lucene.search.Searcher.search(Searcher.java:126) at org.apache.lucene.search.Searcher.search(Searcher.java:105) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher. java:966) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.j ava:838) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:2 69) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent. java:160) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(Search Handler.java:169) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerB ase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.ja va:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j ava:232) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica tionFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt erChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv e.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv e.java:174) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticator Base.java:433) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java :127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java :117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve. java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:1 51) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:87 0) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proc essConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint .java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollow erWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool .java:685) at java.lang.Thread.run(Thread.java:619) Thanks. Karthik
RE: spellchecker: returning results even with misspelt words
Shyam, I tried using spellcheck.collate=true, it doesn't return results with correct word. Do I need to make any other settings?. Thanks. Karthik -Original Message- From: Shyamsunder Reddy [mailto:sjh...@yahoo.com] Sent: Tuesday, March 17, 2009 12:06 PM To: solr-user@lucene.apache.org Subject: Re: spellchecker: returning results even with misspelt words I think if you use spellcheck.collate=true, you will still receive the results for correct word and suggestion for wrong word. I have name field (which is first name+last name) configured for spell check. I have name entry: GUY SHUMAKER. I am trying to find out person names where either 'GUY' or 'SHUMAKER' or both are spelled wrong. 1. Last Name spelled wrong as 'SHAMAKER' http://localhost:8090/solr/select?q=NAME:GUY%20SHAMAKERfq=TYPE:PERSONspellcheck=truespellcheck.collate=true It return all results that match 'GUY' and spelling suggestion for 'SHAMAKER' as 'SHUMAKER' 2. First Name spelled wrong as 'GYY' http://localhost:8090/solr/select?q=NAME:GYY SHUMAKERfq=TYPE:PERSONspellcheck=truespellcheck.collate=true It return No results and spelling suggestion for 'GYY' as 'GUY' and collation as str name=collationNAME:guy SHUMAKER/str Note:But here I expected result that match SHUAMKER 3. Both first name and last name spelled wrong as: GYY SHAMAKER http://localhost:8090/solr/select?q=NAME:GYY%20SHAMAKERfq=TYPE:PERSONspellcheck=truespellcheck.collate=true Here no results, but received suggestion for both words and collation. It is similar to your scenario? Also why NO results are returned for case 2. --- On Tue, 3/17/09, Ingo Renner i...@typo3.org wrote: From: Ingo Renner i...@typo3.org Subject: spellchecker: returning results even with misspelt words To: solr-user@lucene.apache.org Date: Tuesday, March 17, 2009, 9:52 AM Hi all, I'd like to achieve the following: When searching for e.g. two words, one of them being spelt correctly the other one misspelt I'd like to receive results for the correct word but would still like to get spelling suggestions for the wrong word. Currently when I search for misspelt words I get suggestions, but no results at all although there would be results when searching for the correct word only. Hope you understand what I want to achieve as it's a little hard to explain. all the best Ingo --Ingo Renner TYPO3 Core Developer, Release Manager TYPO3 4.2
Highlighting wildcard search results.
Does Solr support highlighting for wildcard searches?. Like I search for tele* and all words matching 'tele' should get highlighted ex: telephone, telex etc. I am trying to get the highlighting using ressponse.getHighlighting(). But It returns empty list. Would appreciate any help. Thanks.
Wildcard query search
Hi, I am trying to perform wildcard search using q query. The query results are returned. After getting the results, I trying to get the highlighting using ressponse.getHighlighting(). It returns empty list. But It works fine for non-wildcard searches. Any ideas please?. Thanks. Karthik
Adding authentication Token to the CommonsHttpSolrServer
Hi, We have installed the Solr in Tomcat server and enabled the security constraint at the Tomcat level.. We require to pass the authentication token(cookie) to the search call that is made using CommonsHttpSolrServer. Would like to know how can I add the token to the CommonsHttpSolrServer. Appreciate any idea on this. Thanks. Karthik
RE: Solr search with Auto Spellchecker
Is it possible get the search results from the spell corrected word in a single solr search query?. Like I search for the word globl and the correct spelling is global.. The query should return results matching with the word global. Would appreciate any ideas.. Thanks. Karthik _ From: Narayanan, Karthikeyan Sent: Tuesday, March 10, 2009 9:43 AM To: 'solr-user@lucene.apache.org' Subject: Solr search with Auto Spellchecker Hello All, Am trying to implement spell checker in the Solr search. I am able to get the suggestions by passing 'spellcheck=true'. But I want the search results from the spell corrected word. How do I achieve that?. Thanks in Advance. Thanks. Karthik
RE: Solr search with Auto Spellchecker
Shalin, Thanks for info... Thanks. Karthik -Original Message- From: Shalin Shekhar Mangar [mailto:shalinman...@gmail.com] Sent: Wednesday, March 11, 2009 9:33 AM To: solr-user@lucene.apache.org Subject: Re: Solr search with Auto Spellchecker On Wed, Mar 11, 2009 at 7:00 PM, Narayanan, Karthikeyan karthikeyan.naraya...@gs.com wrote: Is it possible get the search results from the spell corrected word in a single solr search query?. Like I search for the word globl and the correct spelling is global.. The query should return results matching with the word global. Would appreciate any ideas.. No, you'll need to make two queries. -- Regards, Shalin Shekhar Mangar.
Solr search with Auto Spellchecker
Hello All, Am trying to implement spell checker in the Solr search. I am able to get the suggestions by passing 'spellcheck=true'. But I want the search results from the spell corrected word. How do I achieve that?. Thanks in Advance. Thanks. Karthik
Solr search with Auto Spellchecker
Hello All, Am trying to implement spell checker in the Solr search. I am able to get the suggestions by passing 'spellcheck=true'. But I want the search results from the spell corrected word. How do I achieve that?. Thanks in Advance. Thanks. Karthik
