Re: Search on dynamic fields
I'm 100% sure about that :) On Jun 5, 2009, at 9:05 PM, Avlesh Singh wrote: My bad! Please ignore the previous reply. I just read your mail again to realize that there is a problem. Are you sure that the case (upper/lower) in your query and index match? Any difference in cases would not fetch you results for a field of type string. Cheers Avlesh On Sat, Jun 6, 2009 at 12:30 AM, Avlesh Singh avl...@gmail.com wrote: When I search for exact match on New I get zero results! q=17000219_s:New But when I search on 1003356 I get results! q=17000219_s: 1003356 And when I use wildcards I get results too! q=17000219_s: *New or q=17000219_s:New* or q=17000219_s:*New* Nothing your case is unexpected, Reza. Your dynamic field is a * multivalued* field which has three values. The document containing this field would be fetched whenever this field is searched for any of the contained values. I think you want to use this field differently. It would be to nice to hear from you, before I make too many assumptions to figure out your problem and suggest a solution. Cheers Avlesh On Fri, Jun 5, 2009 at 8:39 PM, Reza Safari r.saf...@lukkien.com wrote: When I search for exact match on New I get zero results! q=17000219_s:New But when I search on 1003356 I get results! q=17000219_s: 1003356 And when I use wildcards I get results too! q=17000219_s: *New or q=17000219_s:New* or q=17000219_s:*New* Can anyone explain this? schema: dynamicField name=*_s type=string indexed=true stored=true multiValued=true/ Xml: arr name=17000219_s str1003356/str strNew/str strNew/str /arr -- Reza Safari LUKKIEN Copernicuslaan 15 6716 BM Ede The Netherlands - http://www.lukkien.com t: +31 (0) 318 698000 This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited. -- Reza Safari LUKKIEN Copernicuslaan 15 6716 BM Ede The Netherlands - http://www.lukkien.com t: +31 (0) 318 698000 This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
Query on date fields
Hi, I have two date attributes in my Indexes: DisplayStartDate_dt DisplayEndDate_dt I need to fetch results where today's date lies between displayStartDate and dislayEndDate. However i cannot send hardcoded displayStartdate and displayEndDate date in query as there are 1000 different dates in indexes Please suggest the query. Thanks, Prerna -- View this message in context: http://www.nabble.com/Query-on-date-fields-tp23919342p23919342.html Sent from the Solr - User mailing list archive at Nabble.com.
How to make wordDelimiterFilter[pulled from Solr nighly] to not break non-english words in a wrong way in lucene indexing/searching?
Hi All,
I'm trying to index some indian web page content which are basically a mix
of indian and say 5% of english content in the same page itself. For all
this I can not use standard or simple analyzer as they break the non-english
words in a wrong places say[because the isLetter(ch) happens to be false for
them, even if they are part of a word]. So I wrote/extended the anayzer that
does the following,
public class IndicAnalyzer extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream ts = new WhitespaceTokenizer(reader);
//ts = new WordDelimiterFilter(ts, 1, 1, 1, 1, 0);
ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS);
ts = new LowerCaseFilter(ts);
ts = new PorterStemFilter(ts);
return ts;
}
}
This is working fine to some extent when the line commented above is left as
such, but its not able to give me the results when the documtnt contains a
string say he...@how.com and the searched query is hello, this is expected
as the above code doesnot do any word delimiting around these basic
characters [like @ . , etc ].
Now the problem is when I'm trying to use wordDelimiterFilter[hte commented
out line, this filter I got from solr] it is breaking say hindi words around
some characters which are actually part of a word. After going through the
code for this filter I found that it is using the isLetter() standard
fuction of java which I think returns false for those hindi characters
around which it is breaking the words. As per javadoc isLetter() is Unicode
compliant, right? so can't we say that it is aware of the above characters
that they are word delimiters, then why is this breaking around those
characters. I'm stuck and dont know how to get rid of the problem. And
because of this problem when I search for say a hindi word helo , assuming
its hindi, it highlights this word but alognwith that it also highlights the
letters of this word h/e/l/o whereever it finds it which it should not do,
right?
I request both Solr and Lucene users to guide me in fixing this issue. BTW,
do we need to do some sort of normalization for the content before sending
it to lucene indexer? just a thought, i don know whats the way out?
Terms Component
Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur
Re: Terms Component
I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. Sorry..My mistake! On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur anshuman_ma...@stragure.comwrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur
Re: Terms Component
while on the subject, can anybody tell me when Solr 1.4 might come out? Thanks Anshuman Manur On Mon, Jun 8, 2009 at 5:37 PM, Anshuman Manur anshuman_ma...@stragure.comwrote: I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. Sorry..My mistake! On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur anshuman_ma...@stragure.com wrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur
Re: Terms Component
It seems like your /terms handler is likely expecting a q parameter due to the QueryComponent being one of the Handlers SearchComponents. What does your configuration for /terms look like? -Grant On Jun 8, 2009, at 7:48 AM, Anshuman Manur wrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org .apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java: 104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org .apache .solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 290) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 235) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 233) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 175) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java: 568) at org .ofbiz .catalina .container .CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 286) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 844) at org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint $Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: searchcomponent howto ...
Is there a question here? On Jun 7, 2009, at 2:50 PM, David Palaitis wrote: I'm trying to write a very simple SearchComponent that performs a simple operation on the query string before passing through to QueryComponent. The EncodingSearchComponent will perform something like the following Base64(MD5(querystring)) -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
Re: Does Solr know what host and port it is running on?
All good points, Paul. I've settled on putting it into the config, as much as I don't want to. On Jun 6, 2009, at 4:59 PM, Paul Libbrecht wrote: It is common knowledge that a web-application cannot know it's port and host-name: - many possible cnames to host-name - potential tunnels - most important and most used: proxies, typically an apache mod_proxy or mod_webapp on a port different than the servlet container port. Le 06-juin-09 à 09:38, Noble Paul നോബിള് नोब्ळ् a écrit : can it be added into the solrconfig.xml or solr.xml ? My experience is that it is simple as that: you have to configure something which, in ActiveMath, we have called the advertizedURL. Making it non-configurable really often creates issues and requires the deployer to dig deep into the servlet container configuration (I never was fully able with mod_proxy and tomcat). Defaulting to something such as request.getServerPort etc is probably right though. hope it helps. paul On Sat, Jun 6, 2009 at 10:55 AM, Shalin Shekhar Mangarshalinman...@gmail.com wrote: On Sat, Jun 6, 2009 at 9:58 AM, Grant Ingersoll gsing...@apache.org wrote: Does Solr know what host and port it is running on and, if so, how do I get access to it programmatically? I feel like I'm missing something obvious. No, it does not. You can only get this on the first request. There's an issue open at https://issues.apache.org/jira/browse/SOLR-727 -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search
solr distributed search example - exception
Hi, I was executing a simple example which demonstrates DistributedSearch. example provided in the following link: http://wiki.apache.org/solr/DistributedSearch however, when i startup the server in both port nos: 8983 and 7574, i get the following exception: SEVERE: Could not start SOLR. Check solr/home property java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.w3c.dom.NodeList at org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:131) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:70) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore at java.lang.Class.initializeClass(libgcj.so.7rh) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) Caused by: java.lang.ClassNotFoundException: org.apache.solr.core.JmxMonitoredMap not found in StartLoader[file:/home/ithurs/apache-solr-1.3.0/example7574/, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/jetty-6.1.3.jar, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/jetty-util-6.1.3.jar, file:/home/ithurs/apache-solr-1.3.0/example7574/lib/servlet-api-2.5-6.1.3.jar] at java.net.URLClassLoader.findClass(libgcj.so.7rh) at java.lang.ClassLoader.loadClass(libgcj.so.7rh) at java.lang.ClassLoader.loadClass(libgcj.so.7rh) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:375) at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:337) at java.lang.Class.forName(libgcj.so.7rh) at java.lang.Class.initializeClass(libgcj.so.7rh) ...22 more 2009-06-08
Re: Does Solr know what host and port it is running on?
how about putting it in the requestDispatcher tag? On Mon, Jun 8, 2009 at 6:39 PM, Grant Ingersollgsing...@apache.org wrote: All good points, Paul. I've settled on putting it into the config, as much as I don't want to. On Jun 6, 2009, at 4:59 PM, Paul Libbrecht wrote: It is common knowledge that a web-application cannot know it's port and host-name: - many possible cnames to host-name - potential tunnels - most important and most used: proxies, typically an apache mod_proxy or mod_webapp on a port different than the servlet container port. Le 06-juin-09 à 09:38, Noble Paul നോബിള് नोब्ळ् a écrit : can it be added into the solrconfig.xml or solr.xml ? My experience is that it is simple as that: you have to configure something which, in ActiveMath, we have called the advertizedURL. Making it non-configurable really often creates issues and requires the deployer to dig deep into the servlet container configuration (I never was fully able with mod_proxy and tomcat). Defaulting to something such as request.getServerPort etc is probably right though. hope it helps. paul On Sat, Jun 6, 2009 at 10:55 AM, Shalin Shekhar Mangarshalinman...@gmail.com wrote: On Sat, Jun 6, 2009 at 9:58 AM, Grant Ingersoll gsing...@apache.org wrote: Does Solr know what host and port it is running on and, if so, how do I get access to it programmatically? I feel like I'm missing something obvious. No, it does not. You can only get this on the first request. There's an issue open at https://issues.apache.org/jira/browse/SOLR-727 -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search -- - Noble Paul | Principal Engineer| AOL | http://aol.com
StreamingUpdateSolrServer recommendations?
Hi all, I guess this questions i mainly aimed to you, Ryan. I've been trying out your StreamingUpdateSolrServer implementation for indexin, and clearly see the improvements in indexing-times compared to the CommonsHttpSolrServer :) Great work! My question is, do you have any recommendations as to what values I should use / have you found a sweet-spot? What are the trade-offs? Thread count is obvious with regard to the number of cpus available, but what about the queue size? Any thoughts? I tried 20 / 3 as you have posted in the issue thread, and get averages of about 80 documents / sec (and I have not optimized the document processing etc, which takes the larger part of the time). Anyways, I was just curious on what others are using (and what times you are getting at) Keep up the good work! Aleks -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: Terms Component
You can try out the nightly build of solr (which is the solr 1.4 dev version) containing all the new nice and shiny features of Solr 1.4:) To use Terms Component you simply need to configure the handler as explained in the documentation / wiki. Cheers, Aleksander On Mon, 08 Jun 2009 14:22:15 +0200, Anshuman Manur anshuman_ma...@stragure.com wrote: while on the subject, can anybody tell me when Solr 1.4 might come out? Thanks Anshuman Manur On Mon, Jun 8, 2009 at 5:37 PM, Anshuman Manur anshuman_ma...@stragure.comwrote: I'm using Solr 1.3 apparently.and Solr 1.4 is not out yet. Sorry..My mistake! On Mon, Jun 8, 2009 at 5:18 PM, Anshuman Manur anshuman_ma...@stragure.com wrote: Hello, I want to use the terms component in Solr 1.4: But http://localhost:8983/solr/terms?terms.fl=name But, I get the following error with the above query: java.lang.NullPointerException at org.apache.solr.common.util.StrUtils.splitSmart(StrUtils.java:37) at org.apache.solr.search.OldLuceneQParser.parse(LuceneQParserPlugin.java:104) at org.apache.solr.search.QParser.getQuery(QParser.java:88) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:82) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:148) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:84) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:295) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.ofbiz.catalina.container.CrossSubdomainSessionValve.invoke(CrossSubdomainSessionValve.java:44) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Any help would be great. Thanks Anshuman Manur -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Fetching Dynamic Fields
Hi all, Is there a way to select all the dynamic fields in the fl field without using *. Here is what I am looking for. Fields in the schema, locationName_*, locationId,description,content. I want to select just the locationName_* and locationId. How can I do this without using fl=*, coz I don't want to fetch all the other fields. Any suggestions in this regard will be helpful. Thanks, Kalyan Manepalli
Configure Collection Distribution in Solr 1.3
Hi, we configured multi-core solr 1.3 server in Tomcat 6.0.18 servlet container. Its working great. Now I need to configure collection Distribution to replicate indexing data between master and 2 slaves. Please provide me step by step instructions to configure collection distribution between master and slaves would be helpful. Thanks in advance. Thanks Mahesh. -- View this message in context: http://www.nabble.com/Configure-Collection-Distribution-in-Solr-1.3-tp23927332p23927332.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Configure Collection Distribution in Solr 1.3
You'll find everything you need in the Wiki. http://wiki.apache.org/solr/SolrCollectionDistributionOperationsOutline http://wiki.apache.org/solr/SolrCollectionDistributionScripts If things are still uncertain I've written a guide for when we used the solr distribution scrips on our lucene index earlier. You can read that guide here: http://www.integrasco.no/index.php?option=com_contentview=articleid=51:lucene-index-replicationcatid=35:blogItemid=53 Cheers, Aleksander On Mon, 08 Jun 2009 18:22:01 +0200, MaheshR mahesh.ray...@gmail.com wrote: Hi, we configured multi-core solr 1.3 server in Tomcat 6.0.18 servlet container. Its working great. Now I need to configure collection Distribution to replicate indexing data between master and 2 slaves. Please provide me step by step instructions to configure collection distribution between master and slaves would be helpful. Thanks in advance. Thanks Mahesh. -- Aleksander M. Stensby Lead software developer and system architect Integrasco A/S www.integrasco.no http://twitter.com/Integrasco Please consider the environment before printing all or any of this e-mail
Re: query issue /special character and case
Regarding being able to search SCHOLKOPF (o with no umlaut) and match SCHÖLKOPF (with umlaut) try using the ISOLatin1AccentFilterFactory in your analysis chain: filter class=solr.ISOLatin1AccentFilterFactory / This filter removes accented chars and replaces them with non-accented versions. As always, make sure to add it to the for both type index and type query. -Jay On Fri, Jun 5, 2009 at 11:10 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sat, May 30, 2009 at 9:48 AM, revas revas...@gmail.com wrote: Hi , When i give a query like the following ,why does it become a phrase query as shown below? The field type is the default text field in the schema. str name=querystringvolker-blanz/str str name=parsedqueryPhraseQuery(content:volker blanz)/str What is the query that was sent to Solr? Also when i have special characters in the query as SCHÖLKOPF , i am not able to convert the o with spl character to lower case on my unix os/it works fine on windows xp OS .Also if i have a spl character in my query ,i would like to search for it wihtout the special character as SCHOLKOPF ,this works fine in windows with strtr (string translate php fucntion) ,but again not in windows OS. Hmm, not sure. If you are using Tomcat, have you enabled UTF-8? http://wiki.apache.org/solr/SolrTomcat#head-20147ee4d9dd5ca83ed264898280ab60457847c4 You can try using the analysis.jsp on the text field with this token and see how it is being analyzed. See if that gives some hints. -- Regards, Shalin Shekhar Mangar.
Re: query issue /special character and case
On Sat, Jun 6, 2009 at 11:40 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sat, May 30, 2009 at 9:48 AM, revas revas...@gmail.com wrote: Hi , When i give a query like the following ,why does it become a phrase query as shown below? The field type is the default text field in the schema. str name=querystringvolker-blanz/str str name=parsedqueryPhraseQuery(content:volker blanz)/str What is the query that was sent to Solr? The query is content:volker-blanz and this is a default text field Also when i have special characters in the query as SCHÖLKOPF , i am not able to convert the o with spl character to lower case on my unix os/it works fine on windows xp OS .Also if i have a spl character in my query ,i would like to search for it wihtout the special character as SCHOLKOPF ,this works fine in windows with strtr (string translate php fucntion) ,but again not in windows OS. Hmm, not sure. If you are using Tomcat, have you enabled UTF-8? http://wiki.apache.org/solr/SolrTomcat#head-20147ee4d9dd5ca83ed264898280ab60457847c4 You can try using the analysis.jsp on the text field with this token and see how it is being analyzed. See if that gives some hints. Yes i am using tomcat and have enabled utf -8 in tomcat. -- Regards, Shalin Shekhar Mangar.
Re: questions about optimizations SOLR-1111,SOLR-1169,SOLR-1179
Marc Sturlese wrote: Hey there, I have a few questions about the new solr speed and memory imporvements. They look like at pretty low level but I would like to understand some stuff... Sorry if something I say sounds odd. I am trying to optimize couple of custom components with these new features but not sure if I am in the right way- 1.-SOLR-111 I have seen there where some memory problems with FieldCache and older nightly builds. I use it in stats component, field collapsing and for a custom sort. Is FieldCache working alrite in the last nightly? 2.-SOLR-1169 I have seen that HashDocSet has been replaced by SortedIntDocSet. Is it already that way in the last nighlty? I am asking because I am testing field collapsing patch too. A HashDocsSet is being used in there... would be a good idea with the last nigthly to replace it with a SortedIntDocSet? 3.-SOLR-1179 In my custom component I have a DocList with all docs that match a query sorted by relevance. I have to iterate over that DocList and its not really good in performance... Iterate over a DocSet it's much faster but I will not have the elements sorted by relevance anymore. Is it possible to get them sorted by relevance with the new SortedIntDocSet? Thanks in advance -- View this message in context: http://www.nabble.com/questions-about-optimizations-SOLR-%2CSOLR-1169%2CSOLR-1179-tp23889097p23929482.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Fetching Dynamic Fields
Hope this helps - http://wiki.apache.org/solr/CommonQueryParameters#head-db2785986af2355759faaaca53dc8fd0b012d1ab Cheers Avlesh On Mon, Jun 8, 2009 at 9:10 PM, Manepalli, Kalyan kalyan.manepa...@orbitz.com wrote: Hi all, Is there a way to select all the dynamic fields in the fl field without using *. Here is what I am looking for. Fields in the schema, locationName_*, locationId,description,content. I want to select just the locationName_* and locationId. How can I do this without using fl=*, coz I don't want to fetch all the other fields. Any suggestions in this regard will be helpful. Thanks, Kalyan Manepalli
Trie Patches- Backportable?
Hi, I am still using Solr 1.2 with the Lucene 2.2 that came with that version of Solr. I am interested in taking advantage of the trie filtering to alleviate some performance problems and was wondering how back-portable these patches are? I am also trying to understand how the Trie algorithm cuts down the number of term queries compared to a normal range query. I was at the recent Bay Area lucene/solr meetup where this was covered but missed some of the details. I know the ideal case is to upgrade to a newer Solr/Lucene but we are resource constrained and can't devote the time right now to test and upgrade our production systems to a newer Solr. Thanks! Amit
Re: searchcomponent howto ...
sorry. my 'tab' character is linked to 'send'. i meant to include this bit
of source
@Override
public void process(ResponseBuilder rb) throws IOException {
SolrQueryRequest request = rb.req;
SolrParams params = request.getParams();
String qs = rb.getQueryString();
String defType = params.get(QueryParsing.DEFTYPE);
defType = defType==null ? QParserPlugin.DEFAULT_QTYPE :
defType;
try {
QParser parser =
QParser.getParser(encoder.process(rb.getQueryString()), defType, request);
rb.setQuery( parser.getQuery() );
rb.setQparser(parser);
} catch (ParseException e) {
throw new
SolrException(SolrException.ErrorCode.BAD_REQUEST, e);
}
}
and then ask,
- how can i set the value of query so that it is reflected in the 'q'
node of the search results e.g. str name=qsolr/str.
the example 'process' method above works, but the original query is still
written to the search results page.
On Mon, Jun 8, 2009 at 9:08 AM, Grant Ingersoll gsing...@apache.org wrote:
Is there a question here?
On Jun 7, 2009, at 2:50 PM, David Palaitis wrote:
I'm trying to write a very simple SearchComponent that performs a simple
operation on the query string before passing through to QueryComponent.
The EncodingSearchComponent will perform something like the following
Base64(MD5(querystring))
--
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
Solr/Lucene:
http://www.lucidimagination.com/search
Re: Use the same SQL Field in Dataimporthandler twice?
Thanks for your answer.
${db.tableA.id} that specifies the sql query that the Dataimporthandler
should Use the sql field id in table tableA located in Database db.
like in the example from the Solr Wiki:
entity name=item_category query=select CATEGORY_ID from item_category
where item_id='${item.ID}'
It´s strange I know but when I use something other than id as the foreign
key for the query everything works!
like:
${db.tableA.anotherid}
Noble Paul നോബിള് नोब्ळ्-2 wrote:
what is ${db.tableA.id} ?
I think there is something extra in that
can you paste the whole data-config.xml?
can you paste
On Sun, Jun 7, 2009 at 1:09 AM, gateway0reiterwo...@yahoo.de wrote:
Hi,
I tried to do the following:
field column=id name=db_id/
entity name=comment dataSource=db query=select com_text from
comment
where ftype=1 and fid='${db.tableA.id}'
field column=com_text name=comment/
/entity
So I use the SQL Table Field id twice once for db_id in my index and
for
the sql query as fid=id.
That doesn´t work!
But when I change the query from fid=id to like fid=otherkey it does
work!
Like:
field column=id name=db_id/
entity name=comment dataSource=db query=select com_text from
comment
where ftype=1 and fid='${db.tableA.otherkey}'
field column=com_text name=comment/
/entity
Is there any other kind of a workaround so I can use the SQL Field id
twice as I wanted to? Thanks
kind regards, Sebastian
--
View this message in context:
http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23904968.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
--
View this message in context:
http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23930286.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Fetching Dynamic Fields
No I am looking for specific scenario. I want to query for the dynamic fields along with few normal fields. Thanks, Kalyan Manepalli -Original Message- From: Avlesh Singh [mailto:avl...@gmail.com] Sent: Monday, June 08, 2009 1:22 PM To: solr-user@lucene.apache.org Subject: Re: Fetching Dynamic Fields Hope this helps - http://wiki.apache.org/solr/CommonQueryParameters#head-db2785986af2355759faaaca53dc8fd0b012d1ab Cheers Avlesh On Mon, Jun 8, 2009 at 9:10 PM, Manepalli, Kalyan kalyan.manepa...@orbitz.com wrote: Hi all, Is there a way to select all the dynamic fields in the fl field without using *. Here is what I am looking for. Fields in the schema, locationName_*, locationId,description,content. I want to select just the locationName_* and locationId. How can I do this without using fl=*, coz I don't want to fetch all the other fields. Any suggestions in this regard will be helpful. Thanks, Kalyan Manepalli
Query faceting
Hi, I have a field called service with following values: - Shuttle Services - Senior Discounts - Laundry Rooms - ... When I conduct query with facet=truefacet.field=servicefacet.limit=-1, I get something like this back: - shuttle 2 - service 3 - senior 0 - laundry 0 - room 3 - ... Questions: - How not to break up fields values in words, so I can get something like Shuttle Services 2 back? - How to tell Solr not to return facet with 0 value? The query takes long time to finish, seemingly because of the long list of items with 0 count. thanks for any advice. _ Insert movie times and more without leaving Hotmail®. http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
Re: Query faceting
In order to get the the values you want for the service field you will need to change the fieldType definition in schema.xml for service to use something that doesn't alter your original values. Try the string fieldType to start and look at the fieldType definition for string. I'm guessing you have it set to text or something else with a chain of filters during analysis. If you don't want back facets with a count of 0 set this param: facet.mincount=1 Have a look at all the values you can set on facets: http://wiki.apache.org/solr/SimpleFacetParameters -Jay On Mon, Jun 8, 2009 at 2:09 PM, siping liu siping...@hotmail.com wrote: Hi, I have a field called service with following values: - Shuttle Services - Senior Discounts - Laundry Rooms - ... When I conduct query with facet=truefacet.field=servicefacet.limit=-1, I get something like this back: - shuttle 2 - service 3 - senior 0 - laundry 0 - room 3 - ... Questions: - How not to break up fields values in words, so I can get something like Shuttle Services 2 back? - How to tell Solr not to return facet with 0 value? The query takes long time to finish, seemingly because of the long list of items with 0 count. thanks for any advice. _ Insert movie times and more without leaving Hotmail®. http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
Collapse Results
Hi,
I am using Solr 1.3 with collapse patch. In the results it returns
standard response as well as collapse response. How do I get just
collapse response?
Regards
Chandra
responseHeader:{
status:0,
QTime:25,
params:{
collapse:true,
wt:json,
collapse.field:resource_id,
indent:on,
fl:resource_id,
q:George,
qt:provider,
version:2.2}},
response:{numFound:5364,start:0,docs:[
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001}]
},
spellcheck:{
suggestions:[]},
collapse_counts:[
field,resource_id,
doc,[
10001,1628,
64308,88,
66931,148,
79374,23,
107152,621,
135179,87,
242091,154,
254375,192,
264802,143,
388030,352],
count,[
1001,1628,
1060,88,
1064,148,
1084,23,
1127,621,
1141,87,
1899,154,
1913,192,
1924,143,
2024,352],
debug,[
Docset type,HashDocSet(16),
Total collapsing time(ms),10,
Create uncollapsed docset(ms),8,
Collapsing normal time(ms),1,
Creating collapseinfo time(ms),1,
Convert to bitset time(ms),0,
Create collapsed docset time(ms),0]],
response:{numFound:16,start:0,docs:[
{
resource_id:1001},
{
resource_id:1060},
{
resource_id:1064},
{
resource_id:1084},
{
resource_id:1127},
{
resource_id:1141},
{
resource_id:1899},
{
resource_id:1913},
{
resource_id:1924},
{
resource_id:2024}]
}}
Re: Questions regarding IT search solution
Hi Jeff, Thanks for the link. You are my lifesaver :)This is exactly simillar to what I am looking for. Thanks,Surfer --- On Fri, 6/5/09, Jeff Hammerbacher ham...@cloudera.com wrote: From: Jeff Hammerbacher ham...@cloudera.com Subject: Re: Questions regarding IT search solution To: solr-user@lucene.apache.org, silentsurfe...@yahoo.com Date: Friday, June 5, 2009, 12:15 AM Hey, Your system sounds similar to the work don by Stu Hood at Rackspace in their Mailtrust unit. See http://highscalability.com/how-rackspace-now-uses-mapreduce-and-hadoop-query-terabytes-datafor more details and inspiration. Regards, Jeff On Thu, Jun 4, 2009 at 4:58 PM, silentsurfe...@yahoo.com wrote: Hi, This is encouraging to know that solr/lucene solution may work. Can anyone using solr/lucene for such scenario can confirm that the solution is used and working fine? That would be really helpful, as I just started looking into the solr/lucene solution only couple of days back and might be difficult to be 100% confident before proposing the solution approach in next couple of days. Thanks,Surfer --- On Thu, 6/4/09, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: From: Otis Gospodnetic otis_gospodne...@yahoo.com Subject: Re: Questions regarding IT search solution To: solr-user@lucene.apache.org Date: Thursday, June 4, 2009, 10:26 PM My guess is Solr/Lucene would work. Not sure how well/fast, but it would, esp. if you avoid range queries (or use tdate), and esp. if you shard/segment indices smartly, so that at query time you send (or distribute if you have to) the query to only those shards that have the data (if your query is for a limited time period). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Silent Surfer silentsurfe...@yahoo.com To: solr-user@lucene.apache.org Sent: Thursday, June 4, 2009 5:52:21 PM Subject: Re: Questions regarding IT search solution Hi, As Alex correctly pointed out my main intention is to figure out whether Solr/lucene offer functionalities to replicate what Splunk is doing in terms of building indexes etc for enabling search capabilities. We evaluated Splunk, but it is not very cost effective solution for us as we may have logs running into few GBs per day as there can be around 25-20 servers running, and Splunk licensing model is based of size of logs per day that too, the license valid for only 1 year. With this back ground, any further inputs on this are greatly appreciated. Thanks,Surfer --- On Thu, 6/4/09, Alexandre Rafalovitch wrote: From: Alexandre Rafalovitch Subject: Re: Questions regarding IT search solution To: solr-user@lucene.apache.org Date: Thursday, June 4, 2009, 9:27 PM I would also be interested to know what other existing solutions exist. Splunk's advantage is that it does extraction of the fields with advanced searching functionality (it has lexers/parsers for multiple content types). I believe that's the Solr's function desired in original posting. At the time they came out (2004), I was not aware of any good open source solutions to do what they did. And I would have loved one, as I was analyzing multi-gigabite logs. Hadoop might be a way to process the files, but what would do the indexing and searching? Regards, Alex. On Thu, Jun 4, 2009 at 11:56 AM, Walter Underwoodwrote: Why build one? Don't those already exist? Personally, I'd start with Hadoop instead of Solr. Putting logs in a search index is guaranteed to not scale. People were already trying different approaches ten years ago. wunder On 6/4/09 8:41 AM, Silent Surfer wrote: Hi, Any help/pointers on the following message would really help me.. Thanks,Surfer --- On Tue, 6/2/09, Silent Surfer wrote: From: Silent Surfer Subject: Questions regarding IT search solution To: solr-user@lucene.apache.org Date: Tuesday, June 2, 2009, 5:45 PM Hi, I am new to Lucene forum and it is my first question.I need a clarification from you. Requirement:--1. Build a IT search tool for logs similar to that of Splunk(Only wrt searching logs but not in terms of reporting, graphs etc) using solr/lucene. The log files are mainly the server logs like JBoss, Custom application server logs (May or may not be log4j logs) and the files size can go potentially upto 100 MB2. The logs are spread across multiple servers (25 to 30 servers)2. Capability to be do search almost realtime3. Support distributed search Our search criterion can be based on a keyword or timestamp or IP address etc. Can anyone throw some light if solr/lucene is right solution for this ? Appreciate any quick help in this regard. Thanks,Surfer
Re: Collapse Results
both CollapseComponent and QueryComponent
execute searches and add results to the Response.
To get only Collapse results, remove the QueryComponent from the
requestHandler.
In solrconfig.xml, I added:
searchComponent name=collapse
class=org.apache.solr.handler.component.CollapseComponent /
requestHandler name=standard class=solr.StandardRequestHandler
!-- default values for query parameters --
lst name=defaults
str name=echoParamsexplicit/str
!--
int name=rows10/int
str name=fl*/str
str name=version2.1/str
--
/lst
arr name=components
strcollapse/str
/arr
/requestHandler
On Mon, Jun 8, 2009 at 5:38 PM, Nirkhe, Chandra chan...@servicelive.comwrote:
Hi,
I am using Solr 1.3 with collapse patch. In the results it returns
standard response as well as collapse response. How do I get just
collapse response?
Regards
Chandra
responseHeader:{
status:0,
QTime:25,
params:{
collapse:true,
wt:json,
collapse.field:resource_id,
indent:on,
fl:resource_id,
q:George,
qt:provider,
version:2.2}},
response:{numFound:5364,start:0,docs:[
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001},
{
resource_id:1001}]
},
spellcheck:{
suggestions:[]},
collapse_counts:[
field,resource_id,
doc,[
10001,1628,
64308,88,
66931,148,
79374,23,
107152,621,
135179,87,
242091,154,
254375,192,
264802,143,
388030,352],
count,[
1001,1628,
1060,88,
1064,148,
1084,23,
1127,621,
1141,87,
1899,154,
1913,192,
1924,143,
2024,352],
debug,[
Docset type,HashDocSet(16),
Total collapsing time(ms),10,
Create uncollapsed docset(ms),8,
Collapsing normal time(ms),1,
Creating collapseinfo time(ms),1,
Convert to bitset time(ms),0,
Create collapsed docset time(ms),0]],
response:{numFound:16,start:0,docs:[
{
resource_id:1001},
{
resource_id:1060},
{
resource_id:1064},
{
resource_id:1084},
{
resource_id:1127},
{
resource_id:1141},
{
resource_id:1899},
{
resource_id:1913},
{
resource_id:1924},
{
resource_id:2024}]
}}
Re: Use the same SQL Field in Dataimporthandler twice?
On Tue, Jun 9, 2009 at 12:41 AM, gateway0reiterwo...@yahoo.de wrote:
Thanks for your answer.
${db.tableA.id} that specifies the sql query that the Dataimporthandler
should Use the sql field id in table tableA located in Database db.
The naming convention does not work like that.
if the entity name is 'tableA' then the field 'id' is addressed as 'tableA.id'
As I said earlier, if you could privide mw with the entire
data-config.xml it would be more helpful
like in the example from the Solr Wiki:
entity name=item_category query=select CATEGORY_ID from item_category
where item_id='${item.ID}'
It´s strange I know but when I use something other than id as the foreign
key for the query everything works!
like:
${db.tableA.anotherid}
Noble Paul നോബിള് नोब्ळ्-2 wrote:
what is ${db.tableA.id} ?
I think there is something extra in that
can you paste the whole data-config.xml?
can you paste
On Sun, Jun 7, 2009 at 1:09 AM, gateway0reiterwo...@yahoo.de wrote:
Hi,
I tried to do the following:
field column=id name=db_id/
entity name=comment dataSource=db query=select com_text from
comment
where ftype=1 and fid='${db.tableA.id}'
field column=com_text name=comment/
/entity
So I use the SQL Table Field id twice once for db_id in my index and
for
the sql query as fid=id.
That doesn´t work!
But when I change the query from fid=id to like fid=otherkey it does
work!
Like:
field column=id name=db_id/
entity name=comment dataSource=db query=select com_text from
comment
where ftype=1 and fid='${db.tableA.otherkey}'
field column=com_text name=comment/
/entity
Is there any other kind of a workaround so I can use the SQL Field id
twice as I wanted to? Thanks
kind regards, Sebastian
--
View this message in context:
http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23904968.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
--
View this message in context:
http://www.nabble.com/Use-the-same-SQL-Field-in-Dataimporthandler-twice--tp23904968p23930286.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
Solr relevancy score - conversion
Hi, I am using solr to inxdex some of the legal documents, where i need the solr search engine to return relevancy ranking score for each search results. As of now i am getting score like 3.12, 1.23, 0.23 so on. Would need an more proportionate score like rounded to 100% (95% relevant, 80 % relevant and so on). Is there a way to make solr returns such scores of such relevance. Any other approach to arrive at this scores also be appreciated thanks vijay -- View this message in context: http://www.nabble.com/Solr-relevancy-score---conversion-tp23936413p23936413.html Sent from the Solr - User mailing list archive at Nabble.com.
spellcheck /too many open files
Hi , 1)Does the spell check component support all languages? 2) I have a scnenario where i have abt 20 webapps in a single container.We get too many open files at index time /while restarting tomcat. The mergefactor is at default. If i reduce the merge factor to 2 and optimize the index ,will the open files be closed automatically or would i have to reindex to close the open files or how do i close the already opened files.This is on linux with solr 1.3 and tomcat 5.5 Regards Revas
Re: solr distributed search example - exception
Hi Mark, yea i would like to open a JIRA issue for it. how do i go about that? Regards, Raakhi On Mon, Jun 8, 2009 at 7:58 PM, Mark Miller markrmil...@gmail.com wrote: That is a very odd cast exception to get. Do you want to open a JIRA issue for this? It looks like an odd exception because the call is: NodeList nodes = (NodeList)solrConfig.evaluate(configPath, XPathConstants.NODESET); // cast exception is we get an ArrayList rather than NodeList Which leads to: Object o = xpath.evaluate(xstr, doc, type); where type = XPathConstants.NODESET So you get back an Object based on the XPathConstant passed. There does not appear to be a value that would return an ArrayList. Using XPathConstants.NODESET gets you a NodeList according to the XPath API. I'm not sure what could cause this to happen. - Mark Rakhi Khatwani wrote: Hi, I was executing a simple example which demonstrates DistributedSearch. example provided in the following link: http://wiki.apache.org/solr/DistributedSearch however, when i startup the server in both port nos: 8983 and 7574, i get the following exception: SEVERE: Could not start SOLR. Check solr/home property java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.w3c.dom.NodeList at org.apache.solr.search.CacheConfig.getMultipleConfigs(CacheConfig.java:61) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:131) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:70) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:117) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at org.mortbay.start.Main.start(Main.java:497) at org.mortbay.start.Main.main(Main.java:115) 2009-06-08 18:36:28.016::WARN: failed SolrRequestFilter java.lang.NoClassDefFoundError: org.apache.solr.core.SolrCore at java.lang.Class.initializeClass(libgcj.so.7rh) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:77) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) at org.mortbay.jetty.Server.doStart(Server.java:210) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) at java.lang.reflect.Method.invoke(libgcj.so.7rh) at org.mortbay.start.Main.invokeMain(Main.java:183) at
