RE: Retrieving a non-indexed but stored field
Sorry for the late follow-up. I am doing this, but get nothing back. Can anyone replicate this problem? Andrew From: Erik Hatcher [EMAIL PROTECTED] Sent: Tuesday, October 14, 2008 12:36 PM To: solr-user@lucene.apache.org Subject: Re: Retrieving a non-indexed but stored field On Oct 14, 2008, at 12:16 PM, Andrew Nagy wrote: Hello - I have the following field: field name=url type=string indexed=false stored=true multiValued=true/ However, when I do a search, the url field does not display. Does the field also need to be indexed in order to retrieve the data? No... but it does need to be requested, either as fl=url, or fl=* Erik
RE: Retrieving a non-indexed but stored field
Sorry for the late follow-up. I am doing this, but get nothing back. Can anyone replicate this problem? Andrew From: Erik Hatcher [EMAIL PROTECTED] Sent: Tuesday, October 14, 2008 12:36 PM To: solr-user@lucene.apache.org Subject: Re: Retrieving a non-indexed but stored field On Oct 14, 2008, at 12:16 PM, Andrew Nagy wrote: Hello - I have the following field: field name=url type=string indexed=false stored=true multiValued=true/ However, when I do a search, the url field does not display. Does the field also need to be indexed in order to retrieve the data? No... but it does need to be requested, either as fl=url, or fl=* Erik
RE: Retrieving a non-indexed but stored field
Nope - I made the schema change and then indexed all of my content. I can confirm that the URL string is included, cause when I change my schema back to have both stored and indexed, it shows the URL data in the search results. When I change it to stored and not indexed, no data is returned. Andrew From: [EMAIL PROTECTED] [EMAIL PROTECTED] On Behalf Of Yonik Seeley [EMAIL PROTECTED] Sent: Wednesday, November 05, 2008 1:41 PM To: solr-user@lucene.apache.org Subject: Re: Retrieving a non-indexed but stored field On Wed, Nov 5, 2008 at 11:47 AM, Andrew Nagy [EMAIL PROTECTED] wrote: Sorry for the late follow-up. I am doing this, but get nothing back. Did you change the field to stored in the schema after you added the document? I've never seen anyone having this problem, so perhaps verify that you are actually sending the url field when indexing. -Yonik
odd 500 error
Hello - I stumbled across an odd error which my intuition is telling me is a bug. Here is my installation: Solr Specification Version: 1.2.2008.08.13.13.05.16 Lucene Implementation Version: 2.4-dev 685576 - 2008-08-13 10:55:25 I did the following query today: author:(r*a* AND fisher) And get the following 500 error: maxClauseCount is set to 1024 org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024 at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:165) at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:156) at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:63) at org.apache.lucene.search.WildcardQuery.rewrite(WildcardQuery.java:54) at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:385) at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:163) at org.apache.lucene.search.Query.weight(Query.java:94) at org.apache.lucene.search.Searcher.createWeight(Searcher.java:175) at org.apache.lucene.search.Searcher.search(Searcher.java:126) at org.apache.lucene.search.Searcher.search(Searcher.java:105) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:167) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1156) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:341) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:272) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1088) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:829) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488) Thanks Andrew
RE: NOTICE: multicore.xml changed to solr.xml, format changes as well
Okay - I found the removal of the default attribute in https://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/core/MultiCore.java? r1=606335r2=602003 I will update the documentation on the multicore changes. Andrew -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 13, 2008 2:27 AM To: solr-user@lucene.apache.org Subject: RE: NOTICE: multicore.xml changed to solr.xml, format changes as well : Chris - thanks for the alert. Can you please clarify the usage of the : default attribute that is documented to be used in the core node. : Solr-545 has a note about this being removed and it is not shown in the : new example solr.xml file. Any attribute that was in the old example multicore.xml has a corrisponding attribute in the example solr.xml ... https://svn.apache.org/viewvc/lucene/solr/trunk/example/multicore/solr. xml?r1=650331r2=685244 ...no functionality was changed at all in this commit, it was just renamed. i don't know anything about a default attribute, other then the fact that the previous commit to that file (r650331) had the message default is no longer a multicore concept https://svn.apache.org/viewvc/lucene/solr/trunk/example/multicore/solr. xml -Hoss
multicore can't find the data directories
I am trying to setup a multicore system implementation. I just upgraded to today's snapshot and have converted my multicore.xml to solr.war and I also changed the xml to match the new schema. However, now that I have done that, Solr is not finding my data directory. With the use of multicore - does each core have its own data directory? I have the following setup: solr/core0/conf solr/core1/conf solr/solr.xml When I look at the admin (http://localhost:8080/solr/core0/admin/stats.jsp) it shows the following: readerDir : org.apache.lucene.store.FSDirectory@/usr/local/solr/data/indexmailto:org.apache.lucene.store.FSDirectory@/usr/local/solr/data/index and numDocs is reporting 0. However I have my index with plenty of records located in solr/core0/data/index Is Solr looking in the wrong directory, or do I have my index in the wrong place? Thanks Andrew
RE: multicore can't find the data directories
Nevermind - sorry. The data directory in my solrconfig.xml was not changed to the correct path. Now it's alive! Andrew -Original Message- From: Andrew Nagy [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 13, 2008 12:13 PM To: solr-user@lucene.apache.org Subject: multicore can't find the data directories I am trying to setup a multicore system implementation. I just upgraded to today's snapshot and have converted my multicore.xml to solr.war and I also changed the xml to match the new schema. However, now that I have done that, Solr is not finding my data directory. With the use of multicore - does each core have its own data directory? I have the following setup: solr/core0/conf solr/core1/conf solr/solr.xml When I look at the admin (http://localhost:8080/solr/core0/admin/stats.jsp) it shows the following: readerDir : org.apache.lucene.store.FSDirectory@/usr/local/solr/data/indexmailto:o rg.apache.lucene.store.FSDirectory@/usr/local/solr/data/index and numDocs is reporting 0. However I have my index with plenty of records located in solr/core0/data/index Is Solr looking in the wrong directory, or do I have my index in the wrong place? Thanks Andrew
RE: more multicore fun
Doug - I had this same problem today. If you look at my post from earlier
today you will see the problem. You will need to adjust the solr.data.dir
value in the solrconfig.xml.
Maybe this also needs to be changed in the example solrconfig.xml document?
Andrew
-Original Message-
From: Doug Steigerwald [mailto:[EMAIL PROTECTED]
Sent: Wednesday, August 13, 2008 2:53 PM
To: solr-user@lucene.apache.org
Subject: more multicore fun
OK. Last question for a while (hopefully), but something else with
multicore seems to be wrong.
solr persistent=true
cores adminPath=/admin/multicore
core name=core0 instanceDir=core0/
core name=core1 instanceDir=core1/
/cores
/solr
$ java -jar start.jar
...
INFO: [core0] Opening new SolrCore at solr/core0/, dataDir=./solr/data/
...
INFO: [core1] Opening new SolrCore at solr/core1/, dataDir=./solr/data/
...
The instanceDir seems to be fine, but the dataDir isn't being set
correctly. The dataDir is actually example/solr/data instead of
example/solr/core{0|1}/data.
http://localhost:8983/solr/admin/multicore shows the exact same path
to the index for both cores. Am I missing something that the example
multicore config doesn't use?
Thanks.
Doug
Solr1.3 Freeze
I read on the Solr 1.3 wiki page that there is a code freeze as of today, is this still accurate? Moreover - does this mean that Solr1.3 will most likely ship with Lucene 2.4-dev or is there any plan to wait for lucene 2.4 to be released? I know scheduling questions are annoying, but I am curious as to how to better manage a project that uses solr and how releases should be scheduled around that. Thanks! Andrew
RE: NOTICE: multicore.xml changed to solr.xml, format changes as well
Chris - thanks for the alert. Can you please clarify the usage of the default attribute that is documented to be used in the core node. Solr-545 has a note about this being removed and it is not shown in the new example solr.xml file. Thanks Andrew -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 12, 2008 2:01 PM To: solr-user@lucene.apache.org Subject: NOTICE: multicore.xml changed to solr.xml, format changes as well If you've bene using the trunk (and/or nightly builds) and you take advantage of the MultiCore features in Solr pelase be aware... As of r685244 (committed a few moments ago) Solr no longer looks for a multicore.xml file. It instead looks for a solr.xml file. solr.xml supports all of the options that multicore.xml supported, however they have been tweaked slightly (in some cases renamed, in other attributes have been moved from one XML tag to another). A detailed example can be seen in example/multicore/solr.xml... http://svn.apache.org/viewvc/lucene/solr/trunk/example/multicore/solr.x ml?view=markup For more information, please see SOLR-689... https://issues.apache.org/jira/browse/SOLR-689 Volunteers to help update the wiki documentation would be appreciated. -Hoss
TimeExceededException
Hello - I am a part of a larger group working on an import tool called SolrMarc. I am running into an error that I'm not sure what is causing it and looking for any leads. I am getting the following exception on the SolrCore constructor: Exception in thread main java.lang.NoClassDefFoundError: org/apache/lucene/search/TimeLimitedCollector$TimeExceededException at org.apache.solr.core.SolrConfig.init(SolrConfig.java:128) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:97) ... Any ideas what might cause this? I am working from the July 25 nightly snapshot. Could I be missing a jar or something? Thanks! Andrew
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
Shalin - yes the allfields field exists in my schema.xml file. It is a field that has all of the text from all of the fields concatenated together into one field. My spellCheckIndexDir is created and has 2 segment files, but I think the index is empty. When I initiate the 1st spellcheck.build=true ... the results load immediately ... I would imagine some time delay as it builds the index. Any other ideas? Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Friday, July 25, 2008 3:35 PM To: solr-user@lucene.apache.org Subject: Re: Multiple search components in one handler - ie spellchecker On Sat, Jul 26, 2008 at 12:37 AM, Andrew Nagy [EMAIL PROTECTED] wrote: Exactly - however the spellcheck component is not working for my setup. The spelling suggestions never show in the response. I think I have the solrconfig setup incorrectly. Also my solr/data/spell index that is created is empty. Something is not configured correctly, any ideas? Andrew From: Geoffrey Young [EMAIL PROTECTED] Sent: Friday, July 25, 2008 3:04 PM To: solr-user@lucene.apache.org Subject: Re: Multiple search components in one handler - ie spellchecker Andrew Nagy wrote: Thanks for getting back to me Geoff. Although, that is pretty much what I have. Maybe if I show my solrconfig someone might be able to point out what I have incorrect? The problem is that nothing related to the spelling options are show in the results, just the normal expected search results. right. the spellcheck component does not issue a separate query *after* running the spellcheck, it merely offers suggestions in parallel with your existing query. the results are more like below are the results for $query. did you mean $suggestions? HTH --Geoff Is the allfields in your spell checker configuration in your schema.xml? Can you see the spellcheckIndexDir created inside the Solr's data directory? -- Regards, Shalin Shekhar Mangar.
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
-Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 10:09 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Can you show us the query you are issuing? Make sure you add spellcheck=true to the query as a parameter to turn on spell checking. http://localhost:8080/solr/select?q=*:*spellcheck=truespellcheck.q=scandanavaspellcheck.build=true Shows this: response lst name=responseHeader int name=status0/int int name=QTime73/int /lst result name=response numFound=7650 start=0 ... /result /response Andrew
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
I was just reviewing the solr logs and I noticed the following: Jul 28, 2008 11:52:01 AM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.component.SpellCheckComponent' It looks like the SpellCheckComponent is not getting loaded. What could cause this? Im running the july25 nightly build. Here is a list of the libs from my /tmp/jetty/webapp/WEB-INF/lib dir: -rw-r--r-- 1 root root 84199 Jul 25 08:14 apache-solr-common-nightly.jar -rw-r--r-- 1 root root 889903 Jul 25 08:14 apache-solr-nightly.jar -rw-r--r-- 1 root root 46725 May 10 2007 commons-codec-1.3.jar -rw-r--r-- 1 root root 22017 Jan 6 2008 commons-csv-1.0-SNAPSHOT-r609327.jar -rw-r--r-- 1 root root 53082 Mar 1 2007 commons-fileupload-1.2.jar -rw-r--r-- 1 root root 305001 Sep 11 2007 commons-httpclient-3.1.jar -rw-r--r-- 1 root root 83613 Jun 15 2007 commons-io-1.3.1.jar -rw-r--r-- 1 root root 38015 Jun 14 2007 commons-logging-1.0.4.jar -rw-r--r-- 1 root root 249154 Sep 21 2007 junit-4.3.jar -rw-r--r-- 1 root root 115101 Jun 19 13:46 lucene-analyzers-2.4-dev.jar -rw-r--r-- 1 root root 730352 Jun 19 13:46 lucene-core-2.4-dev.jar -rw-r--r-- 1 root root 87390 Jun 19 13:46 lucene-highlighter-2.4-dev.jar -rw-r--r-- 1 root root 32693 Jun 19 13:46 lucene-queries-2.4-dev.jar -rw-r--r-- 1 root root 91029 Jun 19 13:46 lucene-snowball-2.4-dev.jar -rw-r--r-- 1 root root 18422 Jun 19 13:46 lucene-spellchecker-2.4-dev.jar -rw-r--r-- 1 root root 179348 Jun 14 2007 stax-1.2.0-dev.jar -rw-r--r-- 1 root root 25863 Jun 14 2007 stax-api-1.0.jar -rw-r--r-- 1 root root 128475 Jun 14 2007 stax-utils.jar could I be missing a jar? Thanks Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 11:24 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Hi Andrew, Your configuration which you specified in the earlier thread looks fine. Your query is also ok. The complete lack of spell check results in the response you pasted suggests that the SpellCheckComponent is not added to the SearchHandler's list of components. Can you check your solrconfig.xml again? I'm sorry but it doesn't seem like a problem with the spell checker itself. Also check if there are any exceptions in the Solr log/console. On Mon, Jul 28, 2008 at 8:32 PM, Andrew Nagy [EMAIL PROTECTED]wrote: -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 10:09 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Can you show us the query you are issuing? Make sure you add spellcheck=true to the query as a parameter to turn on spell checking. http://localhost:8080/solr/select?q=*:*spellcheck=truespellcheck.q=sc andanavaspellcheck.build=true Shows this: response lst name=responseHeader int name=status0/int int name=QTime73/int /lst result name=response numFound=7650 start=0 ... /result /response Andrew -- Regards, Shalin Shekhar Mangar.
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
Hmm ... sorry, that was the output of a java program that uses solr that I ran and noticed the error. That error doesn't happen when I start solr. Sorry for the confusion. I just changed my schema to have a dedicated field for spelling called spelling and I created a new field type for the spellcheck component called textSpell. Here is the segment of my solrconfig.xml: searchComponent name=spellcheck class=org.apache.solr.handler.component.SpellCheckComponent lst name=spellchecker str name=fieldspelling/str str name=accuracy0.7/str !-- 0.5 is the default setting -- str name=spellcheckIndexDir./spellchecker/str /lst str name=queryAnalyzerFieldTypetextSpell/str /searchComponent queryConverter name=queryConverter class=org.apache.solr.spelling.SpellingQueryConverter/ requestHandler name=/search class=org.apache.solr.handler.component.SearchHandler lst name=defaults str name=echoParamsexplicit/str /lst arr name=last-components strspellcheck/str !--strelevator/str-- /arr /requestHandler I will need to reindex my documents again - I will check to see if that has any effect on my problem. Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 12:07 PM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) No, SpellCheckComponent was in the nightly long before July 25. There must be a stack trace after that error message. Can you post that? On Mon, Jul 28, 2008 at 9:26 PM, Andrew Nagy [EMAIL PROTECTED]wrote: I was just reviewing the solr logs and I noticed the following: Jul 28, 2008 11:52:01 AM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.component.SpellCheckComponent' It looks like the SpellCheckComponent is not getting loaded. What could cause this? Im running the july25 nightly build. Here is a list of the libs from my /tmp/jetty/webapp/WEB-INF/lib dir: -rw-r--r-- 1 root root 84199 Jul 25 08:14 apache-solr-common- nightly.jar -rw-r--r-- 1 root root 889903 Jul 25 08:14 apache-solr-nightly.jar -rw-r--r-- 1 root root 46725 May 10 2007 commons-codec-1.3.jar -rw-r--r-- 1 root root 22017 Jan 6 2008 commons-csv-1.0-SNAPSHOT-r609327.jar -rw-r--r-- 1 root root 53082 Mar 1 2007 commons-fileupload- 1.2.jar -rw-r--r-- 1 root root 305001 Sep 11 2007 commons-httpclient- 3.1.jar -rw-r--r-- 1 root root 83613 Jun 15 2007 commons-io-1.3.1.jar -rw-r--r-- 1 root root 38015 Jun 14 2007 commons-logging-1.0.4.jar -rw-r--r-- 1 root root 249154 Sep 21 2007 junit-4.3.jar -rw-r--r-- 1 root root 115101 Jun 19 13:46 lucene-analyzers-2.4- dev.jar -rw-r--r-- 1 root root 730352 Jun 19 13:46 lucene-core-2.4-dev.jar -rw-r--r-- 1 root root 87390 Jun 19 13:46 lucene-highlighter-2.4- dev.jar -rw-r--r-- 1 root root 32693 Jun 19 13:46 lucene-queries-2.4- dev.jar -rw-r--r-- 1 root root 91029 Jun 19 13:46 lucene-snowball-2.4- dev.jar -rw-r--r-- 1 root root 18422 Jun 19 13:46 lucene-spellchecker-2.4- dev.jar -rw-r--r-- 1 root root 179348 Jun 14 2007 stax-1.2.0-dev.jar -rw-r--r-- 1 root root 25863 Jun 14 2007 stax-api-1.0.jar -rw-r--r-- 1 root root 128475 Jun 14 2007 stax-utils.jar could I be missing a jar? Thanks Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 11:24 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Hi Andrew, Your configuration which you specified in the earlier thread looks fine. Your query is also ok. The complete lack of spell check results in the response you pasted suggests that the SpellCheckComponent is not added to the SearchHandler's list of components. Can you check your solrconfig.xml again? I'm sorry but it doesn't seem like a problem with the spell checker itself. Also check if there are any exceptions in the Solr log/console. On Mon, Jul 28, 2008 at 8:32 PM, Andrew Nagy [EMAIL PROTECTED]wrote: -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 10:09 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Can you show us the query you are issuing? Make sure you add spellcheck=true to the query as a parameter to turn on spell checking. http://localhost:8080/solr/select?q=*:*spellcheck=truespellcheck.q=sc andanavaspellcheck.build=true Shows this: response lst name=responseHeader int name=status0/int int name=QTime73/int /lst result
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
Well I will include the stack trace for the aforementioned error: Jul 28, 2008 12:20:17 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.component.SpellCheckComponent' at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:227) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:232) at org.apache.solr.util.plugin.AbstractPluginLoader.create(AbstractPluginLoader.java:83) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:140) at org.apache.solr.core.SolrCore.loadSearchComponents(SolrCore.java:565) at org.apache.solr.core.SolrCore.init(SolrCore.java:371) at org.solrmarc.marc.MarcImporter.init(MarcImporter.java:95) at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:559) Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.component.SpellCheckComponent at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:580) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:242) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:211) ... 7 more The line 95 of MarcImporter.java (the solr import program I am using) is the instantiation of SolrCore. So maybe somehow the spellCheckComponent is not getting loaded? This is the error output I get thrown by instantiating SolrCore: org.apache.solr.common.SolrException: Unknown Search Component: spellcheck at org.apache.solr.core.SolrCore.getSearchComponent(SolrCore.java:597) at org.apache.solr.handler.component.SearchHandler.inform(SearchHandler.java:107) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:264) at org.apache.solr.core.SolrCore.init(SolrCore.java:398) at org.solrmarc.marc.MarcImporter.init(MarcImporter.java:95) at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:559) Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 12:07 PM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) No, SpellCheckComponent was in the nightly long before July 25. There must be a stack trace after that error message. Can you post that? On Mon, Jul 28, 2008 at 9:26 PM, Andrew Nagy [EMAIL PROTECTED]wrote: I was just reviewing the solr logs and I noticed the following: Jul 28, 2008 11:52:01 AM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.component.SpellCheckComponent' It looks like the SpellCheckComponent is not getting loaded. What could cause this? Im running the july25 nightly build. Here is a list of the libs from my /tmp/jetty/webapp/WEB-INF/lib dir: -rw-r--r-- 1 root root 84199 Jul 25 08:14 apache-solr-common- nightly.jar -rw-r--r-- 1 root root 889903 Jul 25 08:14 apache-solr-nightly.jar -rw-r--r-- 1 root root 46725 May 10 2007 commons-codec-1.3.jar -rw-r--r-- 1 root root 22017 Jan 6 2008 commons-csv-1.0-SNAPSHOT-r609327.jar -rw-r--r-- 1 root root 53082 Mar 1 2007 commons-fileupload- 1.2.jar -rw-r--r-- 1 root root 305001 Sep 11 2007 commons-httpclient- 3.1.jar -rw-r--r-- 1 root root 83613 Jun 15 2007 commons-io-1.3.1.jar -rw-r--r-- 1 root root 38015 Jun 14 2007 commons-logging-1.0.4.jar -rw-r--r-- 1 root root 249154 Sep 21 2007 junit-4.3.jar -rw-r--r-- 1 root root 115101 Jun 19 13:46 lucene-analyzers-2.4- dev.jar -rw-r--r-- 1 root root 730352 Jun 19 13:46 lucene-core-2.4-dev.jar -rw-r--r-- 1 root root 87390 Jun 19 13:46 lucene-highlighter-2.4- dev.jar -rw-r--r-- 1 root root 32693 Jun 19 13:46 lucene-queries-2.4- dev.jar -rw-r--r-- 1 root root 91029 Jun 19 13:46 lucene-snowball-2.4- dev.jar -rw-r--r-- 1 root root 18422 Jun 19 13:46 lucene-spellchecker-2.4- dev.jar -rw-r--r-- 1 root root 179348 Jun 14 2007 stax-1.2.0-dev.jar -rw-r--r-- 1 root root 25863 Jun 14 2007 stax-api-1.0.jar -rw-r--r-- 1 root root 128475 Jun 14 2007 stax-utils.jar could I be missing a jar? Thanks Andrew -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 11:24 AM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent
RE: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker)
-Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 12:38 PM To: solr-user@lucene.apache.org Subject: Re: SpellCheckComponent problems (was: Multiple search components in one handler - ie spellchecker) Well that means the nightly solr jar you are using is older than you think it is. Try running solr normally without the program and see if you can get it working. Well my import program has an older copy of the solr libs ... so we can ignore that problem. However my problem still stands when I run solr normally from my July25 snapshot. There are no errors - and no output to the solr logs when I post a query. Have you or anyone been able to successfully add the spellcheckcomponent to the default select searchhandler? Thanks Andrew
Multiple search components in one handler - ie spellchecker
Hello - I am attempting to add the spellCheck component in my search requesthandler so when a users does a search, they get the results and spelling corrections all in one query just like the way the facets work. I am having some trouble accomplishing this - can anyone point me to documentation (other than http://wiki.apache.org/solr/SpellCheckComponent) on how to do this or an example solrconfig that would do this correctly? My guess is there is something preventing this from happening since the requesthandler tag is loading the search component and not the spellchecker component? Thanks and sorry if my message is a bit vague Andrew
RE: Multiple search components in one handler - ie spellchecker
Thanks for getting back to me Geoff. Although, that is pretty much what I have. Maybe if I show my solrconfig someone might be able to point out what I have incorrect? The problem is that nothing related to the spelling options are show in the results, just the normal expected search results. Here is my query: http://localhost:8080/solr/select?q=*:*rows=1spellcheck=truespellcheck.q=scandanavaspellcheck.build=true searchComponent name=spellcheck class=org.apache.solr.handler.component.SpellCheckComponent lst name=spellchecker str name=fieldallfields/str str name=accuracy0.7/str !-- 0.5 is the default setting -- str name=spellcheckIndexDir./spellchecker/str /lst str name=queryAnalyzerFieldTypetext/str /searchComponent queryConverter name=queryConverter class=org.apache.solr.spelling.SpellingQueryConverter/ requestHandler name=/search class=org.apache.solr.handler.component.SearchHandler lst name=defaults str name=echoParamsexplicit/str /lst arr name=last-components strspellcheck/str !--strelevator/str-- /arr /requestHandler From: Geoffrey Young [EMAIL PROTECTED] Sent: Friday, July 25, 2008 2:13 PM To: solr-user@lucene.apache.org Subject: Re: Multiple search components in one handler - ie spellchecker Andrew Nagy wrote: Hello - I am attempting to add the spellCheck component in my search requesthandler so when a users does a search, they get the results and spelling corrections all in one query just like the way the facets work. I am having some trouble accomplishing this - can anyone point me to documentation (other than http://wiki.apache.org/solr/SpellCheckComponent) on how to do this or an example solrconfig that would do this correctly? http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200806.mbox/[EMAIL PROTECTED] in general, just add the arr name=last-components strspellcheck/str /arr bit to your existing handler after following setup in the twiki docs. you can ignore the part about the exceptions, as that has been fixed in trunk. HTH --Geoff
RE: Multiple search components in one handler - ie spellchecker
Exactly - however the spellcheck component is not working for my setup. The spelling suggestions never show in the response. I think I have the solrconfig setup incorrectly. Also my solr/data/spell index that is created is empty. Something is not configured correctly, any ideas? Andrew From: Geoffrey Young [EMAIL PROTECTED] Sent: Friday, July 25, 2008 3:04 PM To: solr-user@lucene.apache.org Subject: Re: Multiple search components in one handler - ie spellchecker Andrew Nagy wrote: Thanks for getting back to me Geoff. Although, that is pretty much what I have. Maybe if I show my solrconfig someone might be able to point out what I have incorrect? The problem is that nothing related to the spelling options are show in the results, just the normal expected search results. right. the spellcheck component does not issue a separate query *after* running the spellcheck, it merely offers suggestions in parallel with your existing query. the results are more like below are the results for $query. did you mean $suggestions? HTH --Geoff
RE: POSTing repeated fields to Solr
On 4-Jun-08, at 2:22 PM, Andrew Nagy wrote: Hello - I was wondering if there is a work around with POSTing repeated fields to Solr. I am using Jetty as my container with Solr 1.2. I tried something like: http://localhost:8080/solr/select/?q=author:(smith)rows=0start=0face t=truefacet.mincount=1facet.limit=10facet.field=authorlastfacet.fie ld=authorfirst I am only getting back facets from the last facet.field. With a GET request I get back all of the facet fields. I am assuming this is a limitation with Jetty? How are others doing this? It could be, but I would be surprised. How are you constructing the POST request (you listed a url, which only makes sense as a GET)? I was just listing the GET representation of the URL. I am using the PEAR::HTTP_Request library for PHP. If you don't think this is a Jetty limitation, then it is probably something with the library that I am using. I will investigate there 1st. Thanks! Andrew
POSTing repeated fields to Solr
Hello - I was wondering if there is a work around with POSTing repeated fields to Solr. I am using Jetty as my container with Solr 1.2. I tried something like: http://localhost:8080/solr/select/?q=author:(smith)rows=0start=0facet=truefacet.mincount=1facet.limit=10facet.field=authorlastfacet.field=authorfirst I am only getting back facets from the last facet.field. With a GET request I get back all of the facet fields. I am assuming this is a limitation with Jetty? How are others doing this? Thanks Andrew
Limit on # of fields to query on
I am testing around with a new feature in my system that uses Solr and I am testing a query that has a search on the same field OR'd together over 150 times. I know this sounds pretty ridiculous and as I said I am just playing around. However Solr just returns a blank page and doesn't process the query. Is there a limit to the number of fields that you can have in your query? My example is to do some inventory checking with something like: id:1 or id:5 or id:7 or id:8 ... and so on for roughly 176 times (which I think would be the worst case scenario in my catalog) Thanks! Andrew
RE: Re:
Sorry for not explaining my self clearly: I have header=true as you can see from the curl command and there is a header line in the csv file. And sorry for the missing subject line. Andrew From: Ryan McKinley [EMAIL PROTECTED] Sent: Sunday, December 02, 2007 5:15 PM To: solr-user@lucene.apache.org Subject: Re: Andrew Nagy wrote: Hello - I am trying out the CSV importer and am curious with an error that I am consistently running into. What am I doing incorrectly here? I am importing a pipe delimited CSV file with quotes encapsulation. Thanks Andrew curl http://localhost:8080/solr/update/csv?header=true%26seperator=%7C%26encapsulator=%22%26commit=true%26stream.file=import/homes.csv html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 400 /title /head bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true/pre pRequestURI=/solr/update/csv/ppismalla href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/ take a look at the error: CSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true either list the fieldnames or use the 1st line as the field names (header=true) ryan
[no subject]
Hello - I am trying out the CSV importer and am curious with an error that I am consistently running into. What am I doing incorrectly here? I am importing a pipe delimited CSV file with quotes encapsulation. Thanks Andrew curl http://localhost:8080/solr/update/csv?header=true%26seperator=%7C%26encapsulator=%22%26commit=true%26stream.file=import/homes.csv html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 400 /title /head bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true/pre pRequestURI=/solr/update/csv/ppismalla href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/
RE: Re:
On Dec 2, 2007, at 5:43 PM, Ryan McKinley wrote: try \ rather then %26 or just put quotes around the whole url. I think curl does the right thing here. I tried all the methods: converting to %26, converting to \ and encapsulating the url with quotes. All give the same error. curl http://localhost:8080/solr/update/csv?header=true\seperator=%7C\encapsulator=%22\commit=true\stream.file=import/homes.csv html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 400 /title /head bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true/pre pRequestURI=/solr/update/csv/ppismalla href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/ Any other ideas? How do other do this? Thanks Andrew
RE: RE: Re:
Ugh ... I shouldn't be coding on a sunday night - especially after the eagles lost again! I spelled separator correctly this time :) - But still no luck. curl 'http://localhost:8080/solr/update/csv?header=trueseparator=%7Cencapsulator=%22commit=truestream.file=import/homes.csv' -H 'Content-type:text/plain; charset=utf-8' html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 400 /title /head bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true/pre pRequestURI=/solr/update/csv/ppismalla href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/
RE: Re:
Ryan, i didn't know there was a debugger - this could come in handy for other things. Thanks! I tried it out and it looks like everything is being parsed correctly when passing the url in quotes: curl http://localhost:8080/solr/debug/dump?header=trueseparator=%7Cencapsulator=%22commit=true; --data-binary @homes.csv -H 'Content-type:text/plain; charset=utf-8' ?xml version=1.0 encoding=UTF-8? response lst name=responseHeaderint name=status0/intint name=QTime0/intstr name=handlerorg.apache.solr.handler.DumpRequestHandler/strlst name=paramsstr name=headertrue/strstr name=separator|/strstr name=committrue/strstr name=encapsulator/str/lst/lstlst name=paramsstr name=echoHandlertrue/strstr name=echoParamsexplicit/strstr name=headertrue/strstr name=separator|/strstr name=committrue/strstr name=encapsulator/str/lstlst name=contextstr name=path/debug/dump/str/lst /response I tried again with update/csv and same error occurs: curl http://localhost:8080/solr/update/csv?header=trueseparator=%7Cencapsulator=%22commit=true; --data-binary @homes.csv -H 'Content-type:text/plain; charset=utf-8' html head meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/ titleError 400 /title /head bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify fieldnames=lt;fieldsgt;* or header=true/pre pRequestURI=/solr/update/csv/ppismalla href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/
RE: Re:
Perfect! That did it.
Thanks for debugging this with me :)
Andrew
From: Ryan McKinley [EMAIL PROTECTED]
Sent: Sunday, December 02, 2007 7:10 PM
To: solr-user@lucene.apache.org
Subject: Re:
hymmm - give it a try without specifying header=true
Looks like if you don't specify header=true, it defaults to true - but
if you do, it throws an error.
I think there may be a bug... Yonik, should line 243 be:
} else if (!hasHeader) {
^!!!
ryan
Andrew Nagy wrote:
Ryan, i didn't know there was a debugger - this could come in handy for other
things. Thanks!
I tried it out and it looks like everything is being parsed correctly when
passing the url in quotes:
curl
http://localhost:8080/solr/debug/dump?header=trueseparator=%7Cencapsulator=%22commit=true;
--data-binary @homes.csv -H 'Content-type:text/plain; charset=utf-8'
?xml version=1.0 encoding=UTF-8?
response
lst name=responseHeaderint name=status0/intint
name=QTime0/intstr
name=handlerorg.apache.solr.handler.DumpRequestHandler/strlst
name=paramsstr name=headertrue/strstr name=separator|/strstr
name=committrue/strstr name=encapsulator/str/lst/lstlst
name=paramsstr name=echoHandlertrue/strstr
name=echoParamsexplicit/strstr name=headertrue/strstr
name=separator|/strstr name=committrue/strstr
name=encapsulator/str/lstlst name=contextstr
name=path/debug/dump/str/lst
/response
I tried again with update/csv and same error occurs:
curl
http://localhost:8080/solr/update/csv?header=trueseparator=%7Cencapsulator=%22commit=true;
--data-binary @homes.csv -H 'Content-type:text/plain; charset=utf-8'
html
head
meta http-equiv=Content-Type content=text/html; charset=ISO-8859-1/
titleError 400 /title
/head
bodyh2HTTP ERROR: 400/h2preCSVLoader: must specify
fieldnames=lt;fieldsgt;* or header=true/pre
pRequestURI=/solr/update/csv/ppismalla
href=http://jetty.mortbay.org/;Powered by Jetty:///a/small/i/pbr/
performance questions
Here are a few SOLR performance questions: 1. I have noticed with 500,000+ records that my facets run quite fast regarding my dataset when there is a large number of matches, but on a small result set (say 10 - 50) the facet queries become very slow. Any suggestions as to how to improve this? 2. Someone asked me if SOLR utilizes anything like a stored procedure to make queries faster. Does SOLR support anything such as this? Thanks Andrew
Apache web server logs in solr
Hello, I was thinking that solr - with its built in faceting - would make for a great apache log file storage system. I was wondering if anyone knows of any module or library for apache to write log files directly to solr or to a lucene index? Thanks Andrew
Facet Field Limits
Hello, I would like to generate a list of facets, let's say on 5 fields. I have the facet limit set to 5 so that for each of the 5 fields there will only by up to 5 values. My question is: Is there a way to change the limit per field? Let's say on facet 2 I would like to display 10 values instead of 5 like the other facets. Thanks! Andrew
RE: compile error with SOLR 69 MoreLikeThis patch
That did the trick. However, now I am trying to apply the patch to a fresh copy of solr on another server. Whenever I do so, I get the following error from the patch command: patching file src/java/org/apache/solr/request/StandardRequestHandler.java Hunk #1 succeeded at 23 with fuzz 2 (offset -1 lines). Hunk #2 FAILED at 31. Hunk #3 succeeded at 146 with fuzz 2 (offset 13 lines). 1 out of 3 hunks FAILED -- saving rejects to file src/java/org/apache/solr/request/StandardRequestHandler.java.rej (Stripping trailing CRs from patch.) patching file src/java/org/apache/solr/handler/MoreLikeThisHandler.java (Stripping trailing CRs from patch.) patching file src/java/org/apache/solr/util/MoreLikeThisParams.java (Stripping trailing CRs from patch.) patching file example/solr/conf/solrconfig.xml Hunk #1 FAILED at 385. 1 out of 1 hunk FAILED -- saving rejects to file example/solr/conf/solrconfig.xml.rej Any ideas on how to get passed this? Andrew From: Brian Whitman [EMAIL PROTECTED] Sent: Wednesday, May 16, 2007 11:14 AM To: solr-user Subject: Re: compile error with SOLR 69 MoreLikeThis patch Change it to DEFALT or change the spelling error in the Lucene version. On May 16, 2007, at 12:13 PM, Andrew Nagy wrote: I downloaded and patched my solr source with the latest solr69 patch and whenever I run ant I get an error: [javac] /office/src/apache-solr-nightly/src/java/org/apache/solr/ handler/MoreLikeThisHandler.java:145: cannot find symbol [javac] symbol : variable DEFAULT_MIN_DOC_FREQ [javac] location: class org.apache.lucene.search.similar.MoreLikeThis [javac] mlt.setMinDocFreq(params.getInt (MoreLikeThisParams.MIN_DOC_FREQ, MoreLikeThis.DEFAULT_MIN_DOC_FREQ)); [javac] ^ Did I miss something, or maybe I have a non-compatible version of SOLR? Thanks Andrew -- http://variogr.am/ [EMAIL PROTECTED]
RE: compile error with SOLR 69 MoreLikeThis patch
That's the problem, as I suspected. I was using a fresh copy of the 1.1.0. I want to use the MLT feature in my production environment, but I guess I will have to roll out the trunk version to my production server. Thanks! From: Brian Whitman [EMAIL PROTECTED] Sent: Thursday, May 24, 2007 10:36 AM To: solr-user@lucene.apache.org Subject: Re: compile error with SOLR 69 MoreLikeThis patch On May 24, 2007, at 10:29 AM, Andrew Nagy wrote: That did the trick. However, now I am trying to apply the patch to a fresh copy of solr on another server. Whenever I do so, I get the following error from the patch command: I just patched trunk two days ago with the latest (posted three days ago) SOLR-69 patch, cleanly. Are you sure you are trying to patch the trunk version?
RE: compile error with SOLR 69 MoreLikeThis patch
Chris, thanks for the tip. I think I am okay with pushing the trunk to my production server. As we say around here, if you want to be on the bleeding edge, you have to be okay with bleeding every once in a while :) Thanks again Andrew From: Chris Hostetter [EMAIL PROTECTED] Sent: Thursday, May 24, 2007 2:18 PM To: solr-user@lucene.apache.org Subject: RE: compile error with SOLR 69 MoreLikeThis patch : That's the problem, as I suspected. I was using a fresh copy of the : 1.1.0. I want to use the MLT feature in my production environment, but : I guess I will have to roll out the trunk version to my production : server. if your goal is to use the MoreLikeThisHanler then you don't need the entire trun .. it should work fine with Solr 1.1.0 ... the mlt functionality added to StandardRequestHandler might be a bit more challenging, but it should still be possible to retrofit the newer StandardRequestHandler into the 1.1.0 release if you wnated to. (Not that i dont' think the trunk is stable, i'm just saying there are ways to get it working without completley upgrading) -Hoss
RE: compile error with SOLR 69 MoreLikeThis patch
duh++ Also, Where should I put the jar file for the patch? From: Erik Hatcher [EMAIL PROTECTED] Sent: Thursday, May 24, 2007 2:37 PM To: solr-user@lucene.apache.org Subject: Re: compile error with SOLR 69 MoreLikeThis patch Andrew, Nightlies are available here: http://people.apache.org/builds/lucene/ solr/nightly/ (a link exists on the wiki main page, for future reference). Erik On May 24, 2007, at 2:28 PM, Andrew Nagy wrote: While I am on this topic, I think it might be nice to have a nightly build for downloading or is their something like that in place, now? From: Chris Hostetter [EMAIL PROTECTED] Sent: Thursday, May 24, 2007 2:18 PM To: solr-user@lucene.apache.org Subject: RE: compile error with SOLR 69 MoreLikeThis patch : That's the problem, as I suspected. I was using a fresh copy of the : 1.1.0. I want to use the MLT feature in my production environment, but : I guess I will have to roll out the trunk version to my production : server. if your goal is to use the MoreLikeThisHanler then you don't need the entire trun .. it should work fine with Solr 1.1.0 ... the mlt functionality added to StandardRequestHandler might be a bit more challenging, but it should still be possible to retrofit the newer StandardRequestHandler into the 1.1.0 release if you wnated to. (Not that i dont' think the trunk is stable, i'm just saying there are ways to get it working without completley upgrading) -Hoss
compile error with SOLR 69 MoreLikeThis patch
I downloaded and patched my solr source with the latest solr69 patch and whenever I run ant I get an error: [javac] /office/src/apache-solr-nightly/src/java/org/apache/solr/handler/MoreLikeThisHandler.java:145: cannot find symbol [javac] symbol : variable DEFAULT_MIN_DOC_FREQ [javac] location: class org.apache.lucene.search.similar.MoreLikeThis [javac] mlt.setMinDocFreq( params.getInt(MoreLikeThisParams.MIN_DOC_FREQ, MoreLikeThis.DEFAULT_MIN_DOC_FREQ)); [javac] ^ Did I miss something, or maybe I have a non-compatible version of SOLR? Thanks Andrew
Ant build question
Hello, I would like to play with patch SOLR-69 and am trying to rebuild solr using ant with some difficulties. When I try to run ant, I get an error saying that it can't find junit, but junit is in the ant lib directory. I had a file called ant-junit.jar in the lib directory and copied it to junit.jar, but it still gave me the error. Any ideas as to what is causing this? Thanks Andrew # ant example Buildfile: build.xml checkJunitPresence: BUILD FAILED /office/src/apache-solr-nightly/build.xml:240: ## JUnit not found. Please make sure junit.jar is in ANT_HOME/lib, or made available to Ant using other mechanisms like -lib or CLASSPATH. ## Total time: 0 seconds # echo $ANT_HOME /usr/local/ant # ls /usr/local/ant/lib/ ant-antlr.jar ant-apache-oro.jar ant-commons-net.jar ant-javamail.jar ant-junit.jar ant-starteam.jar ant-vaj.jar junit.jar ant-apache-bcel.jar ant-apache-regexp.jarant-icontract.jar ant-jdepend.jar ant-launcher.jar ant-stylebook.jar ant-weblogic.jar README ant-apache-bsf.jarant-apache-resolver.jar ant-jai.jar ant-jmf.jar ant-netrexx.jar ant-swing.jar ant-xalan1.jar xercesImpl.jar ant-apache-log4j.jar ant-commons-logging.jar ant.jar ant-jsch.jar ant-nodeps.jarant-trax.jar ant-xslp.jar xml-apis.jar
Re: Ant build question
Erik Hatcher wrote: ant-junit.jar != junit.jar rename it back, and grab junit.jar from junit.org Easy enough. Thanks! Andrew
Re: Schema validator/debugger
Yonik Seeley wrote: Oh wait... Andrew, were you always testing via ping? Check out what the ping query is configured as in solrconfig.xml: pingQuery qt=dismaxamp;q=solramp;start=3amp;fq=id:[* TO *]amp;fq=cat:[* TO *] /pingQuery Perhaps we should change it to something simple by default??? q=solr? That solves the Jetty failure mystery... so it looks like you either have a tomcat setup problem, or a Solr bug that only shows under tomcat. Yes, this is the problem! Good catch :) I have been testing via ping. However this still does not solve my original problem ... I will dig a bit more and see what I can find. Thanks Andrew
Re: Schema validator/debugger
Yonik Seeley wrote: I dropped your schema.xml directly into the Solr example (using Jetty), fired it up, and everything works fine!? Okay, I switched over to Jetty and now I get a different error: SEVERE: org.apache.solr.core.SolrException: undefined field text Are you sure you are using the stock solrconfig.xml, etc? Yep. I apologize ahead of time if I am being blatantly stupid about this. :)
Re: Schema validator/debugger
Ryan McKinley wrote: With a clean checkout, you can run: $ ant example $ cd example $ java -jar start.jar and things work OK. But, when you delete all but the two fields, you get an exception somewhere? Well, I was working from my own directory, not the example directory. I can give that a try ... but all I did was copy everything from the example dir to a different dir. But, basically ... yes. I will delete all instances of solr on my server and start from scratch ... i must have something screwy somewhere.
Re: Schema validator/debugger
Andrew Nagy wrote: Ryan McKinley wrote: What errors are you getting? Are there exceptions in the log when it starts up? Just a null pointer exception. I added a few fields to my schema, and then replaced my solr war file with the latest build (see my message from a week ago). It wouldn't work, so I assumed something wrong with the latest build. I then reverted back to my 1.1.0 copy and am getting these errors. The XML is valid, but that is all I can test. I just installed SOLR-75 patch and the schema browser is able to view the schema perfectly. When I used the default schema with SOLR, everything is fine, but when I replace my schema ... it's throws this NullPointerException Any ideas? Andrew
Re: Schema validator/debugger
Ryan McKinley wrote: Are you using the example solrconfig.xml? The stack trace looks like an error finding the solr.home index directory, that is configured in solrconfig.xml, not schema.xml Yeah, I noticed that too ... but I don't understand why it can't find the home. I have the data home set to ./solr/data and I always start tomcat from the parent directory. If the example schema.xml works, can you try adding a little bit of your schema at a time? Yeah, that is my last resort. I guess I have no choice! Thanks Andrew
Re: Schema validator/debugger
Greg Ludington wrote: I just installed SOLR-75 patch and the schema browser is able to view the schema perfectly. When I used the default schema with SOLR, everything is fine, but when I replace my schema ... it's throws this NullPointerException One thing to note is that the schema browser as currently in SOLR-75 just transforms the xml file -- it does not represent any of the actual parsing of the configuration by Solr upon startup, and so would not be able to catch any errors Solr might encounter in the schema. Sure, but it's an easier way for my non-textual brain to grasp the schema file :) Nice job, BTW. Off topic a bit, Has anyone set forth to build a new admin interface for SOLR? I build a lot of admin interfaces for my day job and would love to give the admin module a bit of a tune-up (I won't use the term overhaul).
Re: Schema validator/debugger
Andrew Nagy wrote: Ryan McKinley wrote: If the example schema.xml works, can you try adding a little bit of your schema at a time? Yeah, that is my last resort. I guess I have no choice! So I did this, and I ended up removing all of my fields and copyfields with still no luck. I took a fresh copy of the example schema, replaced my fields and copyfields and it still the error persists. One thing I noticed is that in the admin it goes from Solr Admin (example) to Solr Admin (unknown) which makes me think it is having a hard time parsing the schema file? Would it help if I posted my schema file? Thanks! Andrew
Re: Schema validator/debugger
Yonik Seeley wrote: On 4/11/07, Andrew Nagy [EMAIL PROTECTED] wrote: If the example schema.xml works, can you try adding a little bit of your schema at a time? Yeah, that is my last resort. I guess I have no choice! That certainly is strange... Sounds like you definitely hit some sort of Solr bug, but I don't see how an error in schema parsing can be lost w/o it going into a log somewhere. Did you check the other Tomcat log files? The only tomcat log file that has any mention of an error is catalina.out Perhaps try putting your schema.xml into the example distributed with Solr (jetty based) and see what happens. I can try this. Will jetty create a log file somewhere? Thanks Andrew
Re: Schema validator/debugger
Ryan McKinley wrote: Off topic a bit, Has anyone set forth to build a new admin interface for SOLR? I build a lot of admin interfaces for my day job and would love to give the admin module a bit of a tune-up (I won't use the term overhaul). i think we definitely need an updated admin interface, yes! Ideally, we could get rid of all the JSP and do everything with requestHandlers + XSLT. Then everything could work with only a JRE - it would not require a JDK and all the jsp jars. AND this would let other clients have access to the same info with JSON/xml etc check: http://issues.apache.org/jira/browse/SOLR-162 This puts a lot of the functionality as handlers, but still needs an XSLT front end Yes, Erik Hatcher showed this off a bit at a conference I was at in February. I was thinking about building off of these xml files. I do quite a bit with XSLT in my day job, so I would love to play around with it. I think this is something that is desperately needed in SOLR, I know it would make my life a lot easier :) I will take a stab at some of this soon ... Andrew
Schema validator/debugger
Does anyone have a good method of debugging a schema? I have been struggling to get my new schema to run for the past couple of days and just do not see anything wrong with the XML document. Thanks Andrew
SEVERE: Error filterStart
Hello, I downloaded the latest nightly snapshot of Solr and replaced my existing war with the new one. Once I restarted tomcat, I get this error: SEVERE: Error filterStart Apr 5, 2007 10:11:28 AM org.apache.catalina.core.StandardContext start SEVERE: Context [/solr] startup failed due to previous errors Any ideas as to what is causing this? I deleted my index to start with a clean slate but I did not change any of my config files, do I need to update these or are the backwards compatible? Thanks! Andrew
Facet fields question
Hello I have 2 fields that I am faceting on, both of which are of type string. The first field is a copyfield from a text field copied to a string field for faceting. The other is purely a string field. The faceted results of the copyfield are accurate; however the facet results of the 2nd string field all have the ends of the words cut off. How can I get the 2nd string field to display in it's original form? Do I need to make a copyField for all fields that are being faceted on? Also, both fields are multivalued. Thanks! Andrew
fetch only facets
Is their a way to not return any docs and only facets? I tried setting the fl equal to blank, but then i get everything back. Thanks Andrew
cache sizes
Is their a science to choosing a cache sizes? I have about 500,000 records and am seeing a lot of evictions, about 50% of lookups. What factors can i look at to determine what my cache sizes should be? Here are my cache statistics: filterCache class: org.apache.solr.search.LRUCache version: 1.0 description: LRU Cache(maxSize=10, initialSize=5, autowarmCount=5, [EMAIL PROTECTED]) stats: lookups : 3195074 hits : 1541249 hitratio : 0.48 inserts : 1653824 evictions : 1553824 size : 10 cumulative_lookups : 3195074 cumulative_hits : 1541249 cumulative_hitratio : 0.48 cumulative_inserts : 1653825 cumulative_evictions : 1553825 name: queryResultCache class: org.apache.solr.search.LRUCache version: 1.0 description: LRU Cache(maxSize=10, initialSize=5000, autowarmCount=5000, [EMAIL PROTECTED]) stats: lookups : 1976 hits : 872 hitratio : 0.44 inserts : 1199 evictions : 0 size : 1104 cumulative_lookups : 1976 cumulative_hits : 872 cumulative_hitratio : 0.44 cumulative_inserts : 1199 cumulative_evictions : 0 name: documentCache class: org.apache.solr.search.LRUCache version: 1.0 description: LRU Cache(maxSize=512, initialSize=512) stats: lookups : 177530 hits : 13255 hitratio : 0.07 inserts : 164275 evictions : 163763 size : 512 cumulative_lookups : 177530 cumulative_hits : 13255 cumulative_hitratio : 0.07 cumulative_inserts : 164275 cumulative_evictions : 163763
Re: Hierarchical Facets
Hmm ... I had a brain storm. Could I do something like this: collectionDir1/Subdir1/SubSubDir1/collection Then query collection:Dir1/Subdir1 and get the facets on collection at that point to see all of the subsubdirectories? Is their any better method? Andrew Andrew Nagy wrote: I am running into a stumbling block and can only find a way to solve the problem with some sort of hierarchical faceting system. I am in the process of moving my records from eXist (http://exist.sf.net) to Solr, but the problem is with the lack of a directory structure that exist has. I figure I could easily solve this problem by setting a field called collection and then faceting on that field to get my list of directories however this will not allow me to do subdirectories. Any ideas on how to implement such a thing in solr? Thanks! Andrew
Hierarchical Facets
I am running into a stumbling block and can only find a way to solve the problem with some sort of hierarchical faceting system. I am in the process of moving my records from eXist (http://exist.sf.net) to Solr, but the problem is with the lack of a directory structure that exist has. I figure I could easily solve this problem by setting a field called collection and then faceting on that field to get my list of directories however this will not allow me to do subdirectories. Any ideas on how to implement such a thing in solr? Thanks! Andrew
Installation woes
Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Thanks Andrew
Re: Installation woes
Argh! Thanks Yonik for pointing out the log files, duh! I had a malformed line in my schema.xml. Nice feature to add down the line, although I know there is a lot of work going into the admin interface so who knows if it is already thought of. Schema Debugger? Maybe one day I will dig into the code guts and try to contribute. Andrew Yonik Seeley wrote: On 3/7/07, Andrew Nagy [EMAIL PROTECTED] wrote: Hello, I am trying to install another copy of solr on a server. I have done this many times before, but am running into errors now and I am not sure what is causing them. I unzipped a copy of 1.1.0 and placed the .war file into tomcat. Then I created the solr directory with my bin, data, conf directories. When I restart tomcat (from the solr directory) and go to the Solr admin website everything looks fine. But, when I pull up the statistics page, I only have information in the Query Handlers box, the rest are empty. When I click on ping, I see a 500 error from tomcat. What am I forgetting to do? Did you point solr to the solr_home (containing your bin, data, and conf directories)? Or do you rely on the default of $CWD/solr? If that's not it, check all the tomcat log files for the first exception you see. -Yonik
Re: MoreLikeThis similarity-type queries in Solr
Bertrand Delacretaz wrote: On 1/31/07, Brian Whitman [EMAIL PROTECTED] wrote: Does Solr have support for the Lucene query-contrib MoreLikeThis query type or anything like it? ... Yes, there's a patch in http://issues.apache.org/jira/browse/SOLR-69 - if you try it, please add your comments on that page. Anyword on something like this being incorporated into the official SOLR release? Andrew
Re: relevance ranking and scoring
Yonik Seeley wrote: What about term ranking, could I rank the phrases searched in title higher than title2? Absolutely... standard lucene syntax for boosting will give you that in the standard query handler. title:(gone with the wind)^3.0 OR title2:(gone with the wind) That did it! Thanks for the Help! What value do the numbers carry in the ranking? I arbitrarily choose the number 5 cause it's an easy number :) I am a bit nervous about the dismax query system as I have quite a bit of other content that could skew the results. Whats the difference between the dismax query handler and listing all of the fields in my search and separating them with an OR? Thanks! Andrew
Re: relevance ranking and scoring
Yonik Seeley wrote: On 1/23/07, Andrew Nagy [EMAIL PROTECTED] wrote: Yonik Seeley wrote: Things you can try: - post the debugging output (including score explain) for the query I have attached the output. - try disabling length normalization for the title field, then remove the entire index and re-idnex. - try the dismax handler, which can generate sloppy phrase queries to boost results containing all terms. - try a different similarity implementation (org.apache.lucene.misc.SweetSpotSimilarity from lucene) Can you explain what these 3 options mean? I would like to get a better understanding of the guts of SOLR/Lucene but I am too busy working on my application that uses it to spend time with the internals. Let's start with the first... add a debugQuery=on parameter to your request and post the full result here. You can get the same effect through the query form on the solr admin pages by checking the Debug: explain checkbox. I attached the results to my last email, are you not able to see them? Andrew
Re: relevance ranking and scoring
Yonik Seeley wrote: Ok, here is your query: str name=rawquerystringtitle:(gone with the wind) OR title2:(gone with the wind)/str And here it is parsed: str name=parsedquery(title:gone title:wind) (title2:gone title2:wind)/str First, notice how stopwords were removed, so with and the will not count in the results. You are querying across two different fields. Notice how the first two documents both have wind in both title and title2, while the third document gone with the wind has no title2 field (and hence can't match on it). In the first documents, the scores for the matches on title and title2 both contribute to the score. For the third document, it's penalized by not matching in both the title and title2 field. You could look at the dismax handler... it helps constructs queries, a component of which are DisjunctionMaxQueries (they don't add together scores from different fields, but just take the highest score from any matching field for a term). You could also see how changing or removing the stopword list affects relevance. Wow, thanks for the verbose response. This gives me a lot to go on! What about term ranking, could I rank the phrases searched in title higher than title2? Thanks! Andrew
relevance ranking and scoring
I have 2 questions about the SOLR relevancy system. 1. Why is it when I search for an exact phrase of a title of a record I have it generally does not come up as the 1st record in the results? ex: title:(gone with the wind), the record comes up 3rd. A record with the term wind as the first word in the title comes up 1st. ex: title:gone with the wind, the record comes up 1st. Is this because the word wind is the only noun? 2. The score that is associated with each value is quite odd, what does it represent. I generally get results with the top record being somewhere around 3.0 or 2.0 and most records are below 1. Thanks! Andrew
Re: relevance ranking and scoring
Yonik Seeley wrote: Things you can try: - post the debugging output (including score explain) for the query I have attached the output. - try disabling length normalization for the title field, then remove the entire index and re-idnex. - try the dismax handler, which can generate sloppy phrase queries to boost results containing all terms. - try a different similarity implementation (org.apache.lucene.misc.SweetSpotSimilarity from lucene) Can you explain what these 3 options mean? I would like to get a better understanding of the guts of SOLR/Lucene but I am too busy working on my application that uses it to spend time with the internals. Thanks Andrew ?xml version=1.0 encoding=UTF-8? response lst name=responseHeader int name=status0/int int name=QTime102/int /lst result name=response numFound=324 start=0 maxScore=2.7548285 doc float name=score2.7548285/float arr name=authorstrFarnol, Jeffery,/str/arr str name=callnumberPR6011.A75.W56 1939/str str name=formatBook/str str name=id97525/str str name=languageeng/str str name=publishDate1939, c1934./str str name=publisherTriangle Books,/str str name=titleWinds of chance //str str name=title2Winds of change [sic]/str /doc doc float name=score2.5437002/float arr name=authorstrSimpson, John E.,/str/arr str name=callnumberQC939.L37S56 1994/str str name=formatBook/str str name=id433518/str str name=isbn0521452112/str str name=languageeng/str str name=physicalill., maps ;/str str name=publishDate1994./str str name=publisherCambridge University Press,/str arr name=subject4astrSea breeze./str/arr arr name=subject4xstr//arr str name=titleSea breeze and local winds //str str name=title2Sea breeze and local wind./str /doc doc float name=score2.438136/float arr name=authorstrHobbs, William Herbert,/str/arr str name=callnumberG743.H6 1968/str str name=formatBook/str str name=id192408/str str name=languageeng/str str name=physicalillus., maps, ports. ;/str str name=publishDate[1968, c1930]/str str name=publisherGreenwood Press,/str arr name=subject4astrMeteorology/str/arr arr name=subject4xstr//arr arr name=subject5strArctic regions./str/arr str name=titleExploring about the North Pole of the winds./str str name=title2North Pole of the winds./str /doc doc float name=score2.4319565/float arr name=authorstrMitchell, Margaret,/str/arr str name=callnumberPS3525.I972G6 1996/str str name=formatBook/str str name=id426657/str str name=isbn0684826259 (alk. paper)/str str name=languageeng/str str name=physicalill. ;/str str name=publishDatec1996./str str name=publisherScribner,/str arr name=subject4astrWomen/str/arr arr name=subject4xstrHistory/str/arr arr name=subject5strGeorgia/str/arr str name=titleGone with the wind //str /doc doc float name=score2.4319565/float arr name=authorstrGable, Clark,/strstrLeigh, Vivien,/strstrHoward, Leslie,/str strDe Havilland, Olivia./strstrMitchell, Thomas,/strstrMcDaniel, Hattie,/strstrMcQueen, Butterfly./str strFleming, Victor,/strstrMitchell, Margaret,/str/arr str name=callnumberVT3188 VHS/str str name=formatVideo/str str name=id529954/str str name=languageeng/str str name=physicalsd., col. ;/str str name=publishDatec1999./str str name=publisherTime Warner Co.,/str arr name=subject4astrWar films./strstrFeature films./str/arr arr name=subject4xstr/str//arr arr name=subject5strUnited States/str/arr str name=titleGone with the wind/str /doc doc float name=score2.4319565/float arr name=authorstrMitchell, Margaret,/str/arr str name=callnumberPS3525.I972G6 1993/str str name=formatBook/str str name=id534773/str str name=isbn0446365386/str str name=languageeng/str str name=publishDate[1993], c1936./str str name=publisherWarner Books,/str arr name=subject5strUnited States/strstrGeorgia/str/arr str name=titleGone with the wind //str /doc doc float name=score1.7023697/float arr name=authorstrPyron, Darden Asbury./str/arr str name=callnumberPS3525.I972G687 1983/str str name=formatBook/str str name=id27783/str str name=isbn081300747X (pbk. : alk. paper)/str str name=languageeng/str str name=publishDatec1983./str str name=publisherUniversity Presses of Florida,/str arr name=subject1strMitchell, Margaret,/str/arr arr name=subject3strGone with the wind (Motion picture)/str/arr arr name=subject5strSouthern States/str/arr str name=titleRecasting :Gone with the wind in American culture //str /doc doc float name=score1.6493776/float arr name=authorstrStuttgarter BlaÃÂserquintett./strstrHaydn, Joseph,/strstrReicha, Anton,/str strDanzi, Franz,/strstrLickl, Johann Georg,/str/arr str name=callnumberCD257/str str name=contentsDivertimento, Nr. 1, B-Dur : Chorale St. Antoni / Joseph Haydn (10:24) -- BlaÃÂserquintett Es-Dur, op. 88, 2 / Anton Reicha (14:06) -- BlaÃÂserquintett B-Dur, op. 56, 1 / Franz Danzi (13:31) -- Quintetto concertante, F-Dur /
New SOLR installation problems
I installed the 12-8 snapshot of solr on my 64bit RH AS server and whenever I go to the admin page I get the following error: SEVERE: Servlet.service() for servlet jsp threw exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.core.SolrCore Any ideas as to what is causing this? Thanks Andrew
Re: New SOLR installation problems
Thanks Yonik for the reply. I am using tomcat, and there is nothing in the catalina.out file. The access log just reports the same error I see in the browser which is reported below. I am starting tomcat from my solr directory which has the data, bin and conf directories as subdirectories. So the CWD should be correct. I have this same setup on another server that I have been working on with no problem. Im kinda lost with this one. Is their a setting in the solrconfig.xml file that I should be looking at? Andrew Yonik Seeley wrote: On 12/11/06, Andrew Nagy [EMAIL PROTECTED] wrote: I installed the 12-8 snapshot of solr on my 64bit RH AS server and whenever I go to the admin page I get the following error: SEVERE: Servlet.service() for servlet jsp threw exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.core.SolrCore Any ideas as to what is causing this? Look through the logs of whatever servlet container you are using for the first exception thrown. It most likely has something to do with not being able to find the solr config files. -Yonik
Re: New SOLR installation problems
Nevermind, I got it working now. Had the paths setup incorrectly. Dumb++ Andrew Andrew Nagy wrote: Thanks Yonik for the reply. I am using tomcat, and there is nothing in the catalina.out file. The access log just reports the same error I see in the browser which is reported below. I am starting tomcat from my solr directory which has the data, bin and conf directories as subdirectories. So the CWD should be correct. I have this same setup on another server that I have been working on with no problem. Im kinda lost with this one. Is their a setting in the solrconfig.xml file that I should be looking at? Andrew Yonik Seeley wrote: On 12/11/06, Andrew Nagy [EMAIL PROTECTED] wrote: I installed the 12-8 snapshot of solr on my 64bit RH AS server and whenever I go to the admin page I get the following error: SEVERE: Servlet.service() for servlet jsp threw exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.core.SolrCore Any ideas as to what is causing this? Look through the logs of whatever servlet container you are using for the first exception thrown. It most likely has something to do with not being able to find the solr config files. -Yonik
multiple collections
I was wondering how I might create multiple collections that have different field sets under solr. Would I have to have multiple implementations of solr running, or can I have more than one schema.xml file per collection ? Thanks Andrew
Re: Result: numFound inaccuracies
- Original Message - From: Yonik Seeley [EMAIL PROTECTED] Date: Friday, December 8, 2006 6:01 pm Subject: Re: Result: numFound inaccuracies To: solr-user@lucene.apache.org start is 0 based :-) Man do I feel dumb! Andrew
Re: Facet Performance
Yonik Seeley wrote: 1) facet on single-valued strings if you can 2) if you can't do (1) then enlarge the fieldcache so that the number of filters (one per possible term in the field you are filtering on) can fit. I changed the filterCache to the following: filterCache class=solr.LRUCache size=25600 initialSize=5120 autowarmCount=1024/ However a search that normally takes .04s is taking 74 seconds once I use the facets since I am faceting on 4 fields. Can you suggest a better configuration that would solve this performance issue, or should I not use faceting? I figure I could run the query twice, once limited to 20 records and then again with the limit set to the total number of records and develop my own facets. I have infact done this before with a different back-end and my code is processed in under .01 seconds. Why is faceting so slow? Andrew
Re: Facet Performance
Chris Hostetter wrote: : Could you suggest a better configuration based on this? If that's what your stats look like after a single request, then i would guess you would need to make your cache size at least 1.6 million in order for it to be of any use in improving your facet speed. Would this have any strong impacts on my system? Should I just set it to an even 2 million to allow for growth? : My data is 492,000 records of book data. I am faceting on 4 fields: : author, subject, language, format. : Format and language are fairly simple as their are only a few unique : terms. Author and subject however are much different in that there are : thousands of unique terms. by the looks of it, you have a lot more then a few thousand unique terms in those two fields ... are you tokenizing on these fields? that's probably not what you want for ields you're going to facet on. All of these fields are set as string in my schema, so if I understand the fields correctly, they are not being tokenized. I also have an author field that is set as text for searching. Thanks Andrew
Re: Facet Performance
Yonik Seeley wrote: Are they multivalued, and do they need to be. Anything that is of type string and not multivalued will use the lucene FieldCache rather than the filterCache. The author field is multivalued. Will this be a strong performance issue? I could make multiple author fields as to not have the multivalued field and then only facet on the first author. Thanks Andrew
Re: Facet Performance
J.J. Larrea wrote: Unfortunately which strategy will be chosen is currently undocumented and control is a bit oblique: If the field is tokenized or multivalued or Boolean, the FilterQuery method will be used; otherwise the FieldCache method. I expect I or others will improve that shortly. Good to hear, cause I can't really get away with not having a multi-valued field for author. Im really excited by solr and really impressed so far. Thanks! Andrew
Re: Facet Performance
Erik Hatcher wrote: On Dec 8, 2006, at 2:15 PM, Andrew Nagy wrote: My data is 492,000 records of book data. I am faceting on 4 fields: author, subject, language, format. Format and language are fairly simple as their are only a few unique terms. Author and subject however are much different in that there are thousands of unique terms. When encountering difficult issues, I like to think in terms of the user interface. Surely you're not presenting 400k+ authors to the users in one shot. In Collex, we have put an AJAX drop-down that shows the author facet (we call it name on the UI, with various roles like author, painter, etc). You can see this in action here: In our data, we don't have unique authors for each records ... so let's say out of the 500,000 records ... we have 200,000 authors. What I am trying to display is the top 10 authors from the results of a search. So I do a search for title:Gone with the wind and I would like to see the top 10 matching authors from these results. But no worries, I have written my own facet handler and I am now back to under a second with faceting! Thanks for everyone's help and keep up the good work! Andrew
Result: numFound inaccuracies
Hello, me again. I have been running some extensive tests of my search engine and have been seeing inaccuracies with the numFound attribute. It tends to return 1 more than what is actually show in the XML. Is this a bug, or could I be doing something wrong? I have a specific example in front of me at the moment where my query found 2 records, yet I get: result numFound=3 start=1 Any ideas? Andrew
Facet Performance
In September there was a thread [1] on this list about heterogeneous facets and their performance. I am having a similar issue and am unclear as the resolution of this thread. I performed a search against my dataset (492,000 records) and got the results I am looking for in .3 seconds. I then set facet to true and got results in 16 seconds and the facets include data that is not in my result set, it is from the entire set. How do I limit the faceting to my results set and speed up the results? Thanks! Andrew [1] http://www.mail-archive.com/solr-user@lucene.apache.org/msg00955.html
Changing the schema.xml
What is necessary for the effects of changing the schema.xml to take effect for all of my records? I restarted tomcat, but it does not seem that my changes have taken effect. I wanted to change a full-text field from type:string to type:text to allow for better searching, but do no see any differences when I make my changes to the schema.xml. Thanks! Andrew
Initial import problems
Hello, I am new to SOLR but very excited for it's possibilities. I am having some difficulties with my data import which I hope can be solved very easily. First I wrote an xslt to transform my xml into the solr schema and modified the schema.xml to match the fields that I created. I then ran the post.sh on my 492,000 records that I have. Near the end of the process the records stopped being added due to a memory heap error. I obviously maxed the allotted memory for the import. Next time I will import less at a time! I then posted a commit statement. I went to my solr admin site and looked at the statistics. It said 372,000 records (roughly) were stored and 1 commit. I tried to do a search but no matter what I search for I get 0 results. I even tried title:the (assuming it is not blocking the stop word, it should return something!). It appears to me that the search is not searching any records. Any idea as to what I might need to do, or should I start over from scratch and re-import my records in smaller chunks? Thanks! Andrew
