Re: How to iterate the solrdocumentlist result
SolrDocumentList extends an ArrayList. You can iterate on it the way you would do on a list. Here's an example for(SolrDocument doc : listingSearchResponse.getResults()){ System.out.print(doc.getFieldValue(yourFieldName)); } Cheers Avlesh On Mon, May 4, 2009 at 10:36 AM, ahmed baseet ahmed.bas...@gmail.comwrote: Hi All, I'm able to get the whole result bundle by using the following method, QueryResponse qr = server.query(query); SolrDocumentList sdl = qr.getResults(); but I'm not able to iterate over the results. I converted this to string and displayed that and that is a full result bundle, I think its in XML. Actually I want to display the result in a browser and each one separately, I mean not as a bundle. There must be some standard methods for this, right ? Can some one give me some pointers in this regard... I'm trying to integrate the java method calls withing html code itself[ the solr server is on my box, and I want to do the testing on my box, so I want to access the indexer from my local box's browser only]. Any good ideas on this? Thanks, Ahmed.
org.apache.*.*.... class not found exception in Internet Explorer
Hi, I'm trying to query solr indexer thru a web page and trying to display the result. I've the following class to query solr [I'm using the query(string) method], import org.apache.solr.common.SolrDocumentList; import org.apache.solr.common.SolrDocument; import java.util.Map; import java.util.Iterator; import java.util.List; import java.util.ArrayList; import java.util.HashMap; import org.apache.solr.client.solrj.SolrServerException; import org.apache.solr.client.solrj.SolrQuery; import org.apache.solr.client.solrj.response.QueryResponse; import org.apache.solr.client.solrj.response.FacetField; import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer; public class SolrjTest { public String query(String q) { CommonsHttpSolrServer server = null; try { server = new CommonsHttpSolrServer(http://localhost:8983/solr/ ); } catch(Exception e) { e.printStackTrace(); } SolrQuery query = new SolrQuery(); query.setQuery(q); query.setQueryType(dismax); query.setIncludeScore(true); try { QueryResponse qr = server.query(query); SolrDocumentList sdl = qr.getResults(); //System.out.println(Found: + sdl.getNumFound()); //System.out.println(Start: + sdl.getStart()); //System.out.println(Max Score: + sdl.getMaxScore()); //System.out.println(); //System.out.println(Result doc : + sdl); return sdl.toString(); } catch (SolrServerException e) { e.printStackTrace(); return null; } } } and the following to pass the queries to solr, get results and display it on the browser, import java.applet.Applet; import java.awt.Graphics; import java.awt.Font; public class ControlJava extends Applet { Font f = new Font(TimesRoman, Font.BOLD, 20); String Message; public void init() { Message = new String(ubuntu); } public void SetMessage(String MsgText) { SolrjTest solr = new SolrjTest(); Message = solr.query(MsgText); repaint(); } public void paint(Graphics g) { g.setFont(f); g.drawString(Message, 15, 50); } } and finally the html page is this, HTML HEAD TITLEControl a Java Applet/TITLE /HEAD BODY H1Control a Java Applet/H1 HR The Java applet below displays text in a large font. You can enter new text to display in the form below, and JavaScript will call the Java applet to change the text. HR FORM NAME=form1 INPUT TYPE=TEXT NAME=text1 INPUT TYPE=BUTTON VALUE=Change Text onClick=document.ControlJava.SetMessage(document.form1.text1.value); /FORM HR APPLET NAME=ControlJava CODE=ControlJava.class WIDTH=450 HEIGHT=125 /APPLET HR End of page. /BODY /HTML When I'm trying to access this page and putting the query in the box that this html shows, the browser [IE] gives some error and after checking I found that the error is some class not found exception, its not able to find the org.apache.*.* classes and hence giving errors. Now instead of calling that I wrote a simpe class not using any apache.solr classes and called the method therein [just returns a string] and it worked fine. I added both both the classes [.class files] given above to the same location where this web page resides. The problem is that browser is not able to find those org.apache.*** classes and creating the mess. Can anyone help this newbie fixing the problem. Thanks a lot. Do let me know if some information is missing/want some extra information on this issue. --Ahmed.
Re: org.apache.*.*.... class not found exception in Internet Explorer
Missed some information. I'm working on Windows XP and my class path is this, .;E:\Program Files\Java\jdk1.6.0_05\bin;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-httpclient-3.1.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-common.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\apache-solr-1.3.0.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\solr-solrj-1.3.0.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-io-1.3.1.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-codec-1.3.jar;D:\firefox download\apache-solr-1.3.0\apache-solr-1.3.0\dist\solrj-lib\commons-logging-1.0.4.jar Thanks, Ahmed. On Mon, May 4, 2009 at 12:10 PM, ahmed baseet ahmed.bas...@gmail.comwrote: Hi, I'm trying to query solr indexer thru a web page and trying to display the result. I've the following class to query solr [I'm using the query(string) method], import org.apache.solr.common.SolrDocumentList; import org.apache.solr.common.SolrDocument; import java.util.Map; import java.util.Iterator; import java.util.List; import java.util.ArrayList; import java.util.HashMap; import org.apache.solr.client.solrj.SolrServerException; import org.apache.solr.client.solrj.SolrQuery; import org.apache.solr.client.solrj.response.QueryResponse; import org.apache.solr.client.solrj.response.FacetField; import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer; public class SolrjTest { public String query(String q) { CommonsHttpSolrServer server = null; try { server = new CommonsHttpSolrServer( http://localhost:8983/solr/;); } catch(Exception e) { e.printStackTrace(); } SolrQuery query = new SolrQuery(); query.setQuery(q); query.setQueryType(dismax); query.setIncludeScore(true); try { QueryResponse qr = server.query(query); SolrDocumentList sdl = qr.getResults(); //System.out.println(Found: + sdl.getNumFound()); //System.out.println(Start: + sdl.getStart()); //System.out.println(Max Score: + sdl.getMaxScore()); //System.out.println(); //System.out.println(Result doc : + sdl); return sdl.toString(); } catch (SolrServerException e) { e.printStackTrace(); return null; } } } and the following to pass the queries to solr, get results and display it on the browser, import java.applet.Applet; import java.awt.Graphics; import java.awt.Font; public class ControlJava extends Applet { Font f = new Font(TimesRoman, Font.BOLD, 20); String Message; public void init() { Message = new String(ubuntu); } public void SetMessage(String MsgText) { SolrjTest solr = new SolrjTest(); Message = solr.query(MsgText); repaint(); } public void paint(Graphics g) { g.setFont(f); g.drawString(Message, 15, 50); } } and finally the html page is this, HTML HEAD TITLEControl a Java Applet/TITLE /HEAD BODY H1Control a Java Applet/H1 HR The Java applet below displays text in a large font. You can enter new text to display in the form below, and JavaScript will call the Java applet to change the text. HR FORM NAME=form1 INPUT TYPE=TEXT NAME=text1 INPUT TYPE=BUTTON VALUE=Change Text onClick=document.ControlJava.SetMessage(document.form1.text1.value); /FORM HR APPLET NAME=ControlJava CODE=ControlJava.class WIDTH=450 HEIGHT=125 /APPLET HR End of page. /BODY /HTML When I'm trying to access this page and putting the query in the box that this html shows, the browser [IE] gives some error and after checking I found that the error is some class not found exception, its not able to find the org.apache.*.* classes and hence giving errors. Now instead of calling that I wrote a simpe class not using any apache.solr classes and called the method therein [just returns a string] and it worked fine. I added both both the classes [.class files] given above to the same location where this web page resides. The problem is that browser is not able to find those org.apache.*** classes and creating the mess. Can anyone help this newbie fixing the problem. Thanks a lot. Do let me know if some information is missing/want some extra information on this issue. --Ahmed.
Re: Last modified time for cores, taking into account uncommitted changes
Hi, I use SolrIndexReader.isCurrent() for this purpose On Fri, May 1, 2009 at 1:42 AM, James Brady james.colin.br...@gmail.comwrote: Hi, The lastModified field the Solr status seems to only be updated when a commit/optimize operation takes place. Is there any way to determine when a core has been changed, including any uncommitted add operations? Thanks, James -- Andrew Klochkov
Re: Update an existing Solr Index
As I know when you resend another index request with some old ID, old field but new content, the old one gets overwritten by the new one. @solr-users, Views??? --Ahmed On Mon, May 4, 2009 at 5:26 PM, appleman1982 prasad.jug...@gmail.comwrote: Hi All, I have a requirement wherein i want to update an existing index in solr. For example : I have issued an index command in solr as add doc field name=id123/field field name=namexxx/field /doc /add The id field is a unique key here. My requirement is that i should be able to update this inex i.e add another field to it without the need to build the entire index again. For example if i issue the following solr command add doc field name=id123/field field name=locationdelhi/field /doc /add it should give me a merged index like add doc field name=id123/field field name=namexxx/field field name=locationdelhi/field /doc /add Any pointers or workarounds to achieve this in solr would be highly appreciated. Thanks, Jugesh -- View this message in context: http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Does solrj return result in XML format? If not then how to make it do that.
use this constructor: public CommonsHttpSolrServer(String solrServerUrl, HttpClient httpClient, ResponseParser parser) throws MalformedURLException { this(new URL(solrServerUrl), httpClient, parser, false); } and give it the XMLResponseParser -- - - - Is this just helpful for debugging with packet sniffing? the XML format will be a bit slower then the binary format. ryan On May 4, 2009, at 8:22 AM, Erik Hatcher wrote: Just out of curiosity, what's the use case for getting the result back in XML from SolrJ? Erik On May 4, 2009, at 8:13 AM, ahmed baseet wrote: Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed.
DIH ConcurrentModificationException
I've got a ConcurrentModificationException during a cron-ed delta import of DIH, I'm using multicore solr nightly from hudson 2009-04-02_08-06-47. I don't know if this stacktrace maybe useful to you, but here it is: java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at org.apache.solr.handler.dataimport.DataImporter.getStatusMessages(DataImporter.java:384) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:210 ) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) of-course due to the nature of this exception I doubt it can be reproduced easily (this is the only one I've got, and the croned job runned a lot of times), but maybe should a synchronized be put somewhere? ciao, Walter
Re: Does solrj return result in XML format? If not then how to make it do that.
The point of using solrj is that you don't have to do any parsing yourself -- you get access to the results in object form. If you need to do parsing, just grab the xml directly: http://host/solr/select?q=*:*wt=xml On May 4, 2009, at 9:36 AM, ahmed baseet wrote: As I know when we query solr from solr admin interface we get back the results in xml format, so thought there must be something similar for solrj as well, which I'll make to go thru an xml parser at the other end and display all the results in the browser. Otherwise I've to iterate the solrdocumentlist and create a list[may be] to put the results and return it back to the browser which will handle displaying that list/map etc. --Ahmed. On Mon, May 4, 2009 at 5:52 PM, Erik Hatcher e...@ehatchersolutions.com wrote: Just out of curiosity, what's the use case for getting the result back in XML from SolrJ? Erik On May 4, 2009, at 8:13 AM, ahmed baseet wrote: Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed.
autoSuggest
Hi, I would like to know how work /autoSuggest. I do have result when I hit : /autoSuggest?terms=trueindent=trueterms.fl=titleterms.rows=5terms.lower=simpomitHeader=true I've: response lst name=terms lst name=title int name=simp74/int int name=simpa129/int int name=simpal2/int int name=simpalma2/int int name=simpan1/int /lst /lst /response How can I ask it to suggest first expression which are more frequent in the database ? How can I look for even for two words, ie: I look for bara ... make it suggesting barack obama ??? thanks a lot, -- View this message in context: http://www.nabble.com/autoSuggest-tp23367848p23367848.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Highlight MoreLikeThis results?
There was a thread about this last week and verdict is currently you can't highlight MoreLikeThis results. Thanks, Matt Weber On May 4, 2009, at 1:22 AM, jli...@gmail.com wrote: My query returns a number of MoreLikeThis results for a given document. I wonder if there is a way to highlight the terms in the MoreLikeThis results? Thanks.
Re: autoSuggest
I am not sure you can return the results in order of frequency, you will have to sort the results yourself. Also, for autoSuggest you will want to add the terms.prefix=input term and terms.lower.incl=false so your example will be: /autoSuggest? terms = true indent = true terms .fl = title terms .rows = 5 terms .lower=simpterms.lower.incl=falseterms.prefix=simpomitHeader=true To get results for more multiple words such as barack obama, you need to set the terms.fl parameter to an untokenized, un-processed field just as you would with a facet. So in your schema.xml, add a new string field, then use a copyfield to copy the value of title into the new field and set terms.fl to the new field you just created after reindexing. Thanks, Matt Weber On May 4, 2009, at 6:46 AM, sunnyfr wrote: Hi, I would like to know how work /autoSuggest. I do have result when I hit : /autoSuggest? terms = true indent =trueterms.fl=titleterms.rows=5terms.lower=simpomitHeader=true I've: response lst name=terms lst name=title int name=simp74/int int name=simpa129/int int name=simpal2/int int name=simpalma2/int int name=simpan1/int /lst /lst /response How can I ask it to suggest first expression which are more frequent in the database ? How can I look for even for two words, ie: I look for bara ... make it suggesting barack obama ??? thanks a lot, -- View this message in context: http://www.nabble.com/autoSuggest-tp23367848p23367848.html Sent from the Solr - User mailing list archive at Nabble.com. Thanks, Matt Weber eSr Technologies http://www.esr-technologies.com
RE: DC/NOVA LuceneSolr meetup
Hi, Actually I have not yet reached the search/Lucene part on my project (which consists in selling pastries over the internet) but I would like to participate to benefit for the discussion. When will this gathering occur? Regards, Mohamed -Original Message- From: Erik Hatcher [mailto:e...@ehatchersolutions.com] Sent: Monday, May 04, 2009 8:50 AM To: solr-user@lucene.apache.org; java-u...@lucene.apache.org ; core-u...@hadoop.apache.org ; solr-user@lucene.apache.org Subject: DC/NOVA LuceneSolr meetup My company is co-sponsoring a Lucene/Solr meetup later this month in the Northern VA / DC area (Reston). Details will be coming out soon. We've got one night of talks planned and considering adding another consecutive night. If you're in the area and have a Lucene (any of the Lucene family of projects, including Mahout, Solr, Tika, etc) project you're eager to discuss for 20-30 minutes, write me offline and tell me about it. Thanks, Erik This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message.
fieldType without tokenizer
Hi, I would like to create a field without tokenizer but I've an error, I tried : fieldType name=textSimple class=solr.TextField positionIncrementGap=100 analyzer filter class=solr.WordDelimiterFilterFactory generateWordParts=0 generateNumberParts=0 catenateWords=1 catenateNumbers=1 catenateAll=0/ filter class=solr.RemoveDuplicatesTokenFilterFactory/ /analyzer /fieldType But I've : May 4 17:49:41 solr-test jsvc.exec[5786]: May 4, 2009 5:49:41 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: analyzer without class or tokenizer filter list ^Iat org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:808) ^Iat org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:58) ^Iat org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:425) ^Iat org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:443) ^Iat org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:141) ^Iat org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:452) ^Iat org.apache.solr.schema.IndexSchema.init(IndexSchema.java:95) ^Iat org.apache.solr.core.CoreContainer.create(CoreContainer.java:344) ^Iat org.apache.solr.core.CoreContainer.load(CoreContainer.java:222) ^Iat org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:107) ^Iat org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) Thanks for your help ! -- View this message in context: http://www.nabble.com/fieldType-without-tokenizer-tp23371300p23371300.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: fieldType without tokenizer
On Mon, May 4, 2009 at 9:28 PM, sunnyfr johanna...@gmail.com wrote: Hi, I would like to create a field without tokenizer but I've an error, You can use KeywordTokenizer which does not do any tokenization. -- Regards, Shalin Shekhar Mangar.
Custom Sorting Based on Relevancy
Hi There, I'm working on a sorting issue. Our site currently sorts by creation date descending, so users list similar products multiple times to show up at the top of the results. When sorting based on score, we want to move items by the same user with the same title down search results. It would be best if the first item stayed in place based on score, and each additional item is moved out (rows * repeated user/title). Is custom sorting the best way? or is there something else I'm not thinking about. At the moment I'm looking at doing roughly the opposite of the Query Elevate Search component. Thanks, David
Solr autocompletion in rails
Hi, I am new to solr. I am using solr server to index the data and make search in a Ruby on rails project.I want to add autocompletion feature. I tried with the xml patch in the schema.xml file of solr, but dont know how to test if the feature is working.also havent been able to integrate the same in the Rails project that is using Solr.Can anyone please provide some help in this regards?? the patch of codes in Schema.xml is : fieldType name=autocomplete class=solr.TextField analyzer type=index tokenizer class=solr.NGramTokenizerFactory minGramSize=3 maxGramSize=15 / filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=([^a-z0-9]) replacement= replace=all / filter class=solr.EdgeNGramFilterFactory maxGramSize=100 minGramSize=1 / /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=([^a-z0-9]) replacement= replace=all / filter class=solr.PatternReplaceFilterFactory pattern=^(.{20})(.*)? replacement=$1 replace=all / /analyzer /fieldType -- View this message in context: http://www.nabble.com/Solr-autocompletion-in-rails-tp23372020p23372020.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr autocompletion in rails
You will probably want to use the new TermsComponent in Solr 1.4. See http://wiki.apache.org/solr/TermsComponent . I just recently wrote a blog post about using autocompletion with TermsComponent, a servlet, and jQuery. You can probably follow these instructions, but instead of writing a servlet you can write a rails handler parsing the json output directly. http://www.mattweber.org/2009/05/02/solr-autosuggest-with-termscomponent-and-jquery/ . Thanks, Matt Weber On May 4, 2009, at 9:39 AM, manisha_5 wrote: Hi, I am new to solr. I am using solr server to index the data and make search in a Ruby on rails project.I want to add autocompletion feature. I tried with the xml patch in the schema.xml file of solr, but dont know how to test if the feature is working.also havent been able to integrate the same in the Rails project that is using Solr.Can anyone please provide some help in this regards?? the patch of codes in Schema.xml is : fieldType name=autocomplete class=solr.TextField analyzer type=index tokenizer class=solr.NGramTokenizerFactory minGramSize=3 maxGramSize=15 / filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=([^a-z0-9]) replacement= replace=all / filter class=solr.EdgeNGramFilterFactory maxGramSize=100 minGramSize=1 / /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=([^a-z0-9]) replacement= replace=all / filter class=solr.PatternReplaceFilterFactory pattern=^(.{20})(.*)? replacement=$1 replace=all / /analyzer /fieldType -- View this message in context: http://www.nabble.com/Solr-autocompletion-in-rails-tp23372020p23372020.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Update an existing Solr Index
I believe you do have to re-index the entire document. From what I have read, this is a Lucene limitation not a Solr one. I have run into this problem too and it's a pain to deal with. If you have data relevant for ranking but not searching, then consider storing it elsewhere and write a custom function to bring the data into Solr. - Amit On Mon, May 4, 2009 at 5:18 AM, ahmed baseet ahmed.bas...@gmail.com wrote: As I know when you resend another index request with some old ID, old field but new content, the old one gets overwritten by the new one. @solr-users, Views??? --Ahmed On Mon, May 4, 2009 at 5:26 PM, appleman1982 prasad.jug...@gmail.com wrote: Hi All, I have a requirement wherein i want to update an existing index in solr. For example : I have issued an index command in solr as add doc field name=id123/field field name=namexxx/field /doc /add The id field is a unique key here. My requirement is that i should be able to update this inex i.e add another field to it without the need to build the entire index again. For example if i issue the following solr command add doc field name=id123/field field name=locationdelhi/field /doc /add it should give me a merged index like add doc field name=id123/field field name=namexxx/field field name=locationdelhi/field /doc /add Any pointers or workarounds to achieve this in solr would be highly appreciated. Thanks, Jugesh -- View this message in context: http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html Sent from the Solr - User mailing list archive at Nabble.com.
Re[2]: Highlight MoreLikeThis results?
I see. Thanks. There was a thread about this last week and verdict is currently you can't highlight MoreLikeThis results. Thanks, Matt Weber My query returns a number of MoreLikeThis results for a given document. I wonder if there is a way to highlight the terms in the MoreLikeThis results? Thanks.
schema.xml: default values for @indexed and @stored
From the apache-solr-1.3.0\example\solr\conf\schema.xml file: !-- since fields of this type are by default not stored or indexed, any data added to them will be ignored outright -- fieldtype name=ignored stored=false indexed=false class=solr.StrField / So for both fieldtype/@stored and fieldtype/@indexed, the default is true, correct? And does the fieldtype configuration constitute a default for field so that field/@stored and field/@indexed take their effective values according to field/@type? Or do these default to true regardless of what's specified in the respective fieldtype? Michael Ludwig
Slow Post Performance
I have an index of about 6 million small documents. I find that after a few days of on and off posting, searching still works fine but posting gets really slow - client http connection always times out. I restarted server. Then posting speed seemed to have returned to normal. Is this the expected behavior? Should I start considering sharking my data? I didn't think 6 million was a large number for one instance of solr but I'd like to know others' experience. Thanks!
Re: Slow Post Performance
On Mon, May 4, 2009 at 1:09 PM, jli...@gmail.com wrote: I have an index of about 6 million small documents. I find that after a few days of on and off posting, searching still works fine but posting gets really slow - client http connection always times out. I restarted server. Then posting speed seemed to have returned to normal. Is this the expected behavior? Definitely not... what servlet container are you using? -Yonik http://www.lucidimagination.com
Re: Authenticated Indexing Not working
Hi, My error is as below. If authentication error while indexing with authentication enable (Basic GET/POST) org.apache.solr.client.solrj.SolrServerException: org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated. at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:470) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:242) at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:259) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:48) at com.infy.icode.SolrAuthenticationTest.index(SolrAuthenticationTest.java:84) http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html If I am indexing without authentication it works fine. I followed procedure listed in above blog. Regards, Allahbaksh On Wed, Apr 29, 2009 at 9:44 PM, Allahbaksh Asadullah allahbaks...@gmail.com wrote: Hi, I followed the procedure given at http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html Regards, Allahbaksh On 4/28/09, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sun, Apr 26, 2009 at 11:04 AM, Allahbaksh Asadullah allahbaks...@gmail.com wrote: HI Otis, I am using HTTPClient for authentication. When I use the server with Authentication for searching it works fine. But when I use it for indexing it throws error. What is the error? Is it thrown by Solr or your servlet container? One difference between a search request and update request with Solrj is that a search request uses HTTP GET by default but an update request uses an HTTP POST by default. Perhaps your authentication scheme is not configured correctly for POST requests? -- Regards, Shalin Shekhar Mangar. -- Allahbaksh Mohammedali Asadullah, Software Engineering Technology Labs, Infosys Technolgies Limited, Electronic City, Hosur Road, Bangalore 560 100, India. (Board: 91-80-28520261 | Extn: 73927 | Direct: 41173927. Fax: 91-80-28520362 | Mobile: 91-9845505322. -- Allahbaksh Mohammedali Asadullah, Software Engineering Technology Labs, Infosys Technolgies Limited, Electronic City, Hosur Road, Bangalore 560 100, India. (Board: 91-80-28520261 | Extn: 73927 | Direct: 41173927. Fax: 91-80-28520362 | Mobile: 91-9845505322.
Re: schema.xml: default values for @indexed and @stored
Attribute values for fields should be inherited from attribute values of their field types. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Michael Ludwig m...@as-guides.com To: solr-user@lucene.apache.org Sent: Monday, May 4, 2009 1:08:43 PM Subject: schema.xml: default values for @indexed and @stored From the apache-solr-1.3.0\example\solr\conf\schema.xml file: class=solr.StrField / So for both fieldtype/@stored and fieldtype/@indexed, the default is true, correct? And does the configuration constitute a default for so that field/@stored and field/@indexed take their effective values according to field/@type? Or do these default to true regardless of what's specified in the respective ? Michael Ludwig
Re: Update an existing Solr Index
Correct. For an attempt to allow only field-level updates (without requiring the whole document to be re-analyzed and re-indexed) please see SOLR-139. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Amit Nithian anith...@gmail.com To: solr-user@lucene.apache.org Sent: Monday, May 4, 2009 1:03:04 PM Subject: Re: Update an existing Solr Index I believe you do have to re-index the entire document. From what I have read, this is a Lucene limitation not a Solr one. I have run into this problem too and it's a pain to deal with. If you have data relevant for ranking but not searching, then consider storing it elsewhere and write a custom function to bring the data into Solr. - Amit On Mon, May 4, 2009 at 5:18 AM, ahmed baseet wrote: As I know when you resend another index request with some old ID, old field but new content, the old one gets overwritten by the new one. @solr-users, Views??? --Ahmed On Mon, May 4, 2009 at 5:26 PM, appleman1982 wrote: Hi All, I have a requirement wherein i want to update an existing index in solr. For example : I have issued an index command in solr as 123 xxx The id field is a unique key here. My requirement is that i should be able to update this inex i.e add another field to it without the need to build the entire index again. For example if i issue the following solr command 123 delhi it should give me a merged index like 123 xxx delhi Any pointers or workarounds to achieve this in solr would be highly appreciated. Thanks, Jugesh -- View this message in context: http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom Sorting Based on Relevancy
Or you could collapse search results with SOLR-236. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: David Giffin da...@giffin.org To: solr-user@lucene.apache.org Sent: Monday, May 4, 2009 12:37:29 PM Subject: Custom Sorting Based on Relevancy Hi There, I'm working on a sorting issue. Our site currently sorts by creation date descending, so users list similar products multiple times to show up at the top of the results. When sorting based on score, we want to move items by the same user with the same title down search results. It would be best if the first item stayed in place based on score, and each additional item is moved out (rows * repeated user/title). Is custom sorting the best way? or is there something else I'm not thinking about. At the moment I'm looking at doing roughly the opposite of the Query Elevate Search component. Thanks, David
Re: offline solr indexing
This should be fine. You won't have to replicate your index, just reopen the searcher when commit is done, that's all. Index updates and searches can be happening at the same time. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Charles Federspiel charles.federsp...@gmail.com To: solr-user@lucene.apache.org Sent: Saturday, May 2, 2009 12:40:13 PM Subject: Re: offline solr indexing Thanks. I imagine replication to a slave would require a filesystem writable by that slave. I think it helps to realize that indexing is really a function of Content Management. After some discussion with coworkers, I've learned that our internal CMS server runs within tomcat and shares a filesystem with our public app-servers. So I'm hoping to deploy solr both to the tomcat instance (for indexing) and our other instances (for searching) simply sharing a Solr home between them. How bad is this? Does updating and commiting the index interrupt search? It would only affect our internal instance, but I still need to know all the effects of this setup. On Mon, Apr 27, 2009 at 6:32 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Tue, Apr 28, 2009 at 12:38 AM, Charles Federspiel charles.federsp...@gmail.com wrote: Solr Users, Our app servers are setup on read-only filesystems. Is there a way to perform indexing from the command line, then copy the index files to the app-server and use Solr to perform search from inside the servlet container? If the filesystem is read-only, then how can you index at all? But what I think you are describing is the regular master-slave setup that we use. A dedicated master on which writes are performed. Multiple slaves on which searches are performed. The index is replicated to slaves through script or the new java based replication. If the Solr implementation is bound to http requests, can Solr perform searches against an index that I create with Lucene? thank you, It can but it is a little tricky to get the schema and analysis correct between your Lucene writer and Solr searcher. -- Regards, Shalin Shekhar Mangar.
Shutting down an instance of EmbeddedSolrServer
Hi all, I notice that when I use EmbeddedSolrServer I have to use Control C to stop the process. I think the way to shut it down is by calling coreContainer.shutdown(). However, is it possible to get the coreContainer from a SolrServer object? Right now it is defined as protected final CoreContainer coreContainer;. I wanted to do: ((EmbeddedSolrServer)solr)getCoreContainer.shutdown(); But is seem I need to keep my own reference to the coreContainer? Is changing this worth a patch? Eric - Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com Free/Busy: http://tinyurl.com/eric-cal
Re: Slow Post Performance
I'm using the embedded Jetty. I have an index of about 6 million small documents. I find that after a few days of on and off posting, searching still works fine but posting gets really slow - client http connection always times out. I restarted server. Then posting speed seemed to have returned to normal. Is this the expected behavior? Definitely not... what servlet container are you using? -Yonik
Re: Hierarchal Faceting Field Type
Hello Koji, I applied the patch: http://issues.apache.org/jira/browse/SOLR-64 It's working but I have a question about facet.prefix. When we add a facet.prefix it filters the facet results but not the query results. Is there a way to filter both? Thank you, Nasseam On Apr 14, 2009, at 3:32 PM, Koji Sekiguchi wrote: Nasseam Elkarra wrote: Background: Set up a system for hierarchal categories using the following scheme: level one# level one#level two# level one#level two#level three# Trying to find the right combination of field type and query to get the desired results. Saw some previous posts about hierarchal facets which helped in the generating the right query but having an issue using the built in text field which ignores our delimiter and the string field which prevents us from doing a start with search. Does anyone have any insight into the field declaration? Any help is appreciated. Thank you. Out of need in my project, I'll get started to work for SOLR-64, expected any day. I'm thinking introducing a field type for hierarchical facet. Koji
Re: Hierarchal Faceting Field Type
On Mon, May 4, 2009 at 5:28 PM, Nasseam Elkarra nass...@bodukai.com wrote: Hello Koji, I applied the patch: http://issues.apache.org/jira/browse/SOLR-64 It's working but I have a question about facet.prefix. When we add a facet.prefix it filters the facet results but not the query results. Is there a way to filter both? Use a normal filter (fq parameter) instead of facet.prefix. -Yonik http://www.lucidimagination.com
Distributed Sizing Question
Hi, We're in the process of converting a Lucene deployment of 500 M documents / 60 G deployment into a Solr Distributed search deployment. The primary reason for the change is instability in the Lucene deployment due to memory constraints. The existing infrastructure is deployed on 48 machines with all indices on each machine using the MultiSearcher. The new deployment will leverage Solr's distributed search model for to deploy smaller index shards in various clusters. Under average load, the system should be able to easily handle 8-10 requests per second. We're looking for some guidance on best practices for sizing the cluster's correctly. Our current thought is to divide the indices into 4 equal parts and build several 4 machine clusters. So each machine will be hosting ~ 15 G each. Has anyone had experience with a similar size deployment? Any suggestions on the architectural strategy? Thanks for the help. Todd
Getting access to current core's conf dir
How do you get access to the current SolrCore in code? More specifically, I am looking for the conf/ directory for the current core (i.e. I have a configuration file housed in SOLR_HOME/core_name/conf and would like to get at the SOLR_HOME/core_name/conf directory.) I was able to do this in 1.2 but of course it only had one core so life was easy but I am not sure how to do this in 1.3 Thanks! Amit
Re: Distributed Sizing Question
Hi Todd, It depends what kind of hardware you run this on. 4 instances of 15GB shard 1 @ machines 1-4 behind VIP1 4 instances of 15GB shard 2 @ machines 5-8 behind VIP2 4 instances of 15GB shard 3 @ machines 9-12 behind VIP3 4 instances of 15GB shard 4 @ machines 13-16 behind VIP4 This is what you are building, right? That sounds fine, though you may be able to get away with even just 2 boxes per shard (e.g. machines 1-2 for shard 1 instead of 4 machines), depending on the query rate and their actual latency. In that case you could break the index into even smaller shards, thus lowering your per-machine RAM requirements: 2 instances of 7.5GB shard 1 @ machines 1-2 behind VIP1 2 instances of 7.5GB shard 2 @ machines 3-4 behind VIP2 2 instances of 7.5GB shard 3 @ machines 5-6 behind VIP3 2 instances of 7.5GB shard 4 @ machines 7-8 behind VIP4 2 instances of 7.5GB shard 5 @ machines 9-10 behind VIP5 2 instances of 7.5GB shard 6 @ machines 11-12 behind VIP6 2 instances of 7.5GB shard 7 @ machines 13-14 behind VIP7 2 instances of 7.5GB shard 8 @ machines 15-16 behind VIP88 You should be okay with 8GB of RAM per machine, or even 4 GB perhaps. It depends how much heap you give to the JVM, how big your Solr caches are, etc. And don't forget you can always put a caching HTTP proxy in front of Solr! Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Todd Benge todd.be...@gmail.com To: solr-user@lucene.apache.org Sent: Monday, May 4, 2009 6:35:47 PM Subject: Distributed Sizing Question Hi, We're in the process of converting a Lucene deployment of 500 M documents / 60 G deployment into a Solr Distributed search deployment. The primary reason for the change is instability in the Lucene deployment due to memory constraints. The existing infrastructure is deployed on 48 machines with all indices on each machine using the MultiSearcher. The new deployment will leverage Solr's distributed search model for to deploy smaller index shards in various clusters. Under average load, the system should be able to easily handle 8-10 requests per second. We're looking for some guidance on best practices for sizing the cluster's correctly. Our current thought is to divide the indices into 4 equal parts and build several 4 machine clusters. So each machine will be hosting ~ 15 G each. Has anyone had experience with a similar size deployment? Any suggestions on the architectural strategy? Thanks for the help. Todd
Re: Does solrj return result in XML format? If not then how to make it do that.
The we interface returns the data in xml for mat because that is the most readable format.SolrJ uses a compact binary format by default. But you can make it use xml as well. but there is no real use case for using xml with solrJ On Mon, May 4, 2009 at 7:06 PM, ahmed baseet ahmed.bas...@gmail.com wrote: As I know when we query solr from solr admin interface we get back the results in xml format, so thought there must be something similar for solrj as well, which I'll make to go thru an xml parser at the other end and display all the results in the browser. Otherwise I've to iterate the solrdocumentlist and create a list[may be] to put the results and return it back to the browser which will handle displaying that list/map etc. --Ahmed. On Mon, May 4, 2009 at 5:52 PM, Erik Hatcher e...@ehatchersolutions.comwrote: Just out of curiosity, what's the use case for getting the result back in XML from SolrJ? Erik On May 4, 2009, at 8:13 AM, ahmed baseet wrote: Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed. -- --Noble Paul
Re: Getting access to current core's conf dir
What component is trying to get the SolrCore? if it is implemented as SolrCoreAware it gets a callback after the core is completely initialized. On Tue, May 5, 2009 at 4:29 AM, Amit Nithian anith...@gmail.com wrote: How do you get access to the current SolrCore in code? More specifically, I am looking for the conf/ directory for the current core (i.e. I have a configuration file housed in SOLR_HOME/core_name/conf and would like to get at the SOLR_HOME/core_name/conf directory.) I was able to do this in 1.2 but of course it only had one core so life was easy but I am not sure how to do this in 1.3 Thanks! Amit -- --Noble Paul
Core Reload issue
Hi, I came across a strange problem while reloading the core in multicore scenario. In the config of one of the core I am making changes in the synonym and stopword files and then reloading the core. The core gets reloaded but the changes in stopword and synonym fiels does not get reflected when I query in. The filters for index and query are the same. I face this problem even if I reindex the documents. But when I restart the servlet container in which the solr is embedded I problem does not resurfaces. My ultimate goal is/was to reflect the changes made in the text files inside the config folder. Is this the expected behaviour or some problem at my side. Could anyone suggest me the possible work around? Thanks in advance! Regards, Sagar Khetkade _ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/india/windows/windowslive/
Re: Shutting down an instance of EmbeddedSolrServer
hi Eric, there should be a getter for CoreContainer in EmbeddedSolrServer. Open an issue --Noble On Tue, May 5, 2009 at 12:17 AM, Eric Pugh ep...@opensourceconnections.com wrote: Hi all, I notice that when I use EmbeddedSolrServer I have to use Control C to stop the process. I think the way to shut it down is by calling coreContainer.shutdown(). However, is it possible to get the coreContainer from a SolrServer object? Right now it is defined as protected final CoreContainer coreContainer;. I wanted to do: ((EmbeddedSolrServer)solr)getCoreContainer.shutdown(); But is seem I need to keep my own reference to the coreContainer? Is changing this worth a patch? Eric - Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com Free/Busy: http://tinyurl.com/eric-cal -- --Noble Paul
Re: DIH ConcurrentModificationException
hi Walter, it needs synchronization. I shall open a bug. On Mon, May 4, 2009 at 7:31 PM, Walter Ferrara walters...@gmail.com wrote: I've got a ConcurrentModificationException during a cron-ed delta import of DIH, I'm using multicore solr nightly from hudson 2009-04-02_08-06-47. I don't know if this stacktrace maybe useful to you, but here it is: java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at org.apache.solr.handler.dataimport.DataImporter.getStatusMessages(DataImporter.java:384) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:210 ) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) of-course due to the nature of this exception I doubt it can be reproduced easily (this is the only one I've got, and the croned job runned a lot of times), but maybe should a synchronized be put somewhere? ciao, Walter -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Core Reload issue
If you change the the conf files and if you reindex the documents it must be reflected are you sure you re-indexed? On Tue, May 5, 2009 at 10:00 AM, Sagar Khetkade sagar.khetk...@hotmail.com wrote: Hi, I came across a strange problem while reloading the core in multicore scenario. In the config of one of the core I am making changes in the synonym and stopword files and then reloading the core. The core gets reloaded but the changes in stopword and synonym fiels does not get reflected when I query in. The filters for index and query are the same. I face this problem even if I reindex the documents. But when I restart the servlet container in which the solr is embedded I problem does not resurfaces. My ultimate goal is/was to reflect the changes made in the text files inside the config folder. Is this the expected behaviour or some problem at my side. Could anyone suggest me the possible work around? Thanks in advance! Regards, Sagar Khetkade _ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/india/windows/windowslive/ -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Getting access to current core's conf dir
I am trying to get at the configuration directory in an implementation of the SolrEventListener. 2009/5/4 Noble Paul നോബിള് नोब्ळ् noble.p...@gmail.com What component is trying to get the SolrCore? if it is implemented as SolrCoreAware it gets a callback after the core is completely initialized. On Tue, May 5, 2009 at 4:29 AM, Amit Nithian anith...@gmail.com wrote: How do you get access to the current SolrCore in code? More specifically, I am looking for the conf/ directory for the current core (i.e. I have a configuration file housed in SOLR_HOME/core_name/conf and would like to get at the SOLR_HOME/core_name/conf directory.) I was able to do this in 1.2 but of course it only had one core so life was easy but I am not sure how to do this in 1.3 Thanks! Amit -- --Noble Paul
Re: Getting access to current core's conf dir
Make your class implement the interface SolrCoreAware and you will get a callback with the core. On Tue, May 5, 2009 at 11:11 AM, Amit Nithian anith...@gmail.com wrote: I am trying to get at the configuration directory in an implementation of the SolrEventListener. 2009/5/4 Noble Paul നോബിള് नोब्ळ् noble.p...@gmail.com What component is trying to get the SolrCore? if it is implemented as SolrCoreAware it gets a callback after the core is completely initialized. On Tue, May 5, 2009 at 4:29 AM, Amit Nithian anith...@gmail.com wrote: How do you get access to the current SolrCore in code? More specifically, I am looking for the conf/ directory for the current core (i.e. I have a configuration file housed in SOLR_HOME/core_name/conf and would like to get at the SOLR_HOME/core_name/conf directory.) I was able to do this in 1.2 but of course it only had one core so life was easy but I am not sure how to do this in 1.3 Thanks! Amit -- --Noble Paul -- - Noble Paul | Principal Engineer| AOL | http://aol.com