Need help to finalize my autocomplete
Hello, I'm using the following code for my autocomplete feature : The field type : fieldType name=autoComplete class=solr.TextField omitNorms=true analyzer tokenizer class=solr.WhitespaceTokenizerFactory / filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory maxGramSize=20 minGramSize=2 / /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory / filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=^(.{20})(.*)? replacement=$1 replace=all / /analyzer /fieldType The field : dynamicField name=*_ac type=autoComplete indexed=true stored=true / The query : ?q=*:*fq=query_ac:harry*wt=jsonrows=15start=0fl=*indent=onfq=model:SearchQuery It gives me a list of results I can parse and use with jQuery autocomplete plugin and all that works very well. Example of results : harry harry potter the last fighting harry harry potter 5 comic relief harry potter What I would like to do now is only to have results starting with the query, so it should be : harry harry potter harry potter 5 Can anybody tell me if it is possible and so how to do it ? Thank you ! Vincent And -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25468885.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Need help to finalize my autocomplete
Hello, I tried to replace the class as you suggested, but I still get the same result (and not results where the query start only with the giving query). fieldType name=autoComplete class=solr.TextField omitNorms=true analyzer tokenizer class=solr.KeywordTokenizerFactory / filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory maxGramSize=20 minGramSize=2 / /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory / filter class=solr.LowerCaseFilterFactory / filter class=solr.PatternReplaceFilterFactory pattern=^(.{20})(.*)? replacement=$1 replace=all / /analyzer /fieldType -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25469239.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Need help to finalize my autocomplete
After re-indexing it works very well ! Thanks a lot ! Vincent -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25469931.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
Wow, it's like the 'mm' parameters is just appeared for the first time... Yes, I read the doc few times, but never understood that the documents who doesn't match any of the expressions will not be return... my apologize everything seems more clear now thanks to the min number parameter. Thank you, Vincent hossman wrote: : The 'qf' parameter used in the dismax seems to work with a 'AND' separator. : I have much more results without dixmax. Is there any way to keep the same : amount of document and process the 'qf' ? did you read any of the docs on dismax? http://wiki.apache.org/solr/DisMaxRequestHandler did you look at the mm param? http://wiki.apache.org/solr/DisMaxRequestHandler#mm -Hoss -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p25041314.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
Hello, Thank you for your answer, I finally used only a 'qf' parameter in the dismax requesthandler and it seems that I have now better and more relevant results. I just don't understand why a result is mainly boosted by his last update by default ! Vincent -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24903143.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
I actually have an other question... The 'qf' parameter used in the dismax seems to work with a 'AND' separator. I have much more results without dixmax. Is there any way to keep the same amount of document and process the 'qf' ? My dismax : requestHandler name=dismax class=solr.SearchHandler lst name=defaults str name=defTypedismax/str str name=echoParamsexplicit/str float name=tie0.01/float str name=qf text^0.5 title_ac^4.0 name_ac^4.0 authors_list_sm^4.0 /str /lst /requestHandler -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24903219.html Sent from the Solr - User mailing list archive at Nabble.com.
Relevant results with DisMaxRequestHandler
Hello, I did notice several strange behaviors on queries. I would like to share with you an example, so maybe you can explain to me what is going wrong. Using the following query : http://localhost:8983/solr/others/select/?debugQuery=trueq=anna%20lewisrows=20start=0fl=*qt=dismax I get back around 100 results. Follow the two first : doc str name=idPerson:151/str str name=name_sVictoria Davisson/str /doc doc str name=idPerson:37/str str name=name_sAnna Lewis/str /doc And the related debugs : 57.998047 = (MATCH) sum of: 0.048290744 = (MATCH) sum of: 0.024546575 = (MATCH) max plus 0.01 times others of: 0.024546575 = (MATCH) weight(text:anna^0.5 in 64288), product of: 0.027395602 = queryWeight(text:anna^0.5), product of: 0.5 = boost 5.734427 = idf(docFreq=564, numDocs=30400) 0.009554783 = queryNorm 0.8960042 = (MATCH) fieldWeight(text:anna in 64288), product of: 1.0 = tf(termFreq(text:anna)=1) 5.734427 = idf(docFreq=564, numDocs=30400) 0.15625 = fieldNorm(field=text, doc=64288) 0.02374417 = (MATCH) max plus 0.01 times others of: 0.02374417 = (MATCH) weight(text:lewi^0.5 in 64288), product of: 0.026944114 = queryWeight(text:lewi^0.5), product of: 0.5 = boost 5.6399217 = idf(docFreq=620, numDocs=30400) 0.009554783 = queryNorm 0.88123775 = (MATCH) fieldWeight(text:lewi in 64288), product of: 1.0 = tf(termFreq(text:lewi)=1) 5.6399217 = idf(docFreq=620, numDocs=30400) 0.15625 = fieldNorm(field=text, doc=64288) 57.949757 = (MATCH) FunctionQuery(ord(name_s)), product of: 1213.0 = ord(name_s)=1213 5.0 = boost 0.009554783 = queryNorm 5.006892 = (MATCH) sum of: 0.038405567 = (MATCH) sum of: 0.021955125 = (MATCH) max plus 0.01 times others of: 0.021955125 = (MATCH) weight(text:anna^0.5 in 62632), product of: 0.027395602 = queryWeight(text:anna^0.5), product of: 0.5 = boost 5.734427 = idf(docFreq=564, numDocs=30400) 0.009554783 = queryNorm 0.80141056 = (MATCH) fieldWeight(text:anna in 62632), product of: 2.236068 = tf(termFreq(text:anna)=5) 5.734427 = idf(docFreq=564, numDocs=30400) 0.0625 = fieldNorm(field=text, doc=62632) 0.016450444 = (MATCH) max plus 0.01 times others of: 0.016450444 = (MATCH) weight(text:lewi^0.5 in 62632), product of: 0.026944114 = queryWeight(text:lewi^0.5), product of: 0.5 = boost 5.6399217 = idf(docFreq=620, numDocs=30400) 0.009554783 = queryNorm 0.61053944 = (MATCH) fieldWeight(text:lewi in 62632), product of: 1.7320508 = tf(termFreq(text:lewi)=3) 5.6399217 = idf(docFreq=620, numDocs=30400) 0.0625 = fieldNorm(field=text, doc=62632) 4.968487 = (MATCH) FunctionQuery(ord(name_s)), product of: 104.0 = ord(name_s)=104 5.0 = boost 0.009554783 = queryNorm I'm using a simple boost function : requestHandler name=dismax class=solr.SearchHandler lst name=defaults str name=defTypedismax/str str name=echoParamsexplicit/str float name=tie0.01/float str name=qf text^0.5 name_s^5.0 /str str name=pf name_s^5.0 /str str name=bf name_s^5.0 /str /lst /requestHandler Can anyone explain to me why the first result is on top (the query is 'anna lewis') with a huge weight and nothing related (it seems the weight come from the name_s field...) ? A second general question... is it possible to boost a field if the query match exactly the content of a field? Thank you ! Vincent -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24716870.html Sent from the Solr - User mailing list archive at Nabble.com.
Stats for all documents and not current search
Hello, I need to retrieve the stats of my index (using StatsComponent). It's not a problem when my query is empty, but the stats are update according the current search... and I need the stats of the whole index everytime. I'm currently doing two request (one with empty keyword to get the stats, one to get the results). Any idea which could save me one request? Thanks ! Vincent -- View this message in context: http://www.nabble.com/Stats-for-all-documents-and-not-current-search-tp24001883p24001883.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr update performance decrease after a while
Thank for your answer, I will have a look and try to find a solution. Cheers -- View this message in context: http://www.nabble.com/Solr-update-performance-decrease-after-a-while-tp23945947p23980681.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr update performance decrease after a while
Hello, We are indexing approximately 500 documents per day. My benchmark says an update is done in 0.7 sec just after Solr has been started. But it quickly decrease to 2.2 secs per update ! I have just been focused on the Schema until now, and didn't changed many stuffs in the solrconfig file. Maybe you have some tips which could help me to be more linear ? Thanks a lot Vincent -- View this message in context: http://www.nabble.com/Solr-update-performance-decrease-after-a-while-tp23945947p23945947.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: User search in Facebook like
Thanks a lot for your answer it fixed all my issues !!! It's really well working ! Cheers, Vincent -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23818867.html Sent from the Solr - User mailing list archive at Nabble.com.
User search in Facebook like
Hello, I built a feature which allow users to search for other user thanks to a dynamic text box. Like facebook, when you search for your friends, the name is display in a javascript dropdown list with a small picture. But I'm not completely happy with the search... I'm using a standard search like ?q=vincent and I get back the results list. If I type 'vinc' I will not get any results (But I would like to display all the users where the name start with 'vinc'). Maybe I need an extra param? I also tried the autosuggest, but I get a list of terms and not direct results... Could you suggest me some solr feature which could help me to get better results? Thanks a lot ! Vincent -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23804854.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: User search in Facebook like
Thanks very much, that's solve my problem ! Now I see another question : how can I manage the lower/upper cases in my search? Thanks ! Dietrich Featherston-2 wrote: try searching for matches where the name starts with whatever the user has entered so far with a wildcard ?q=vinc* Are you always going to be searching for names? If so you could see if the user has entered two terms and suffix each with a wildcard to get potentially more relevant searches. For example, if a user enters vince p, you might substitute that with the query vince* p* to get the following hits Vincent Pérès Vincent Price Vince Price Vince Pérès etc... D 2009/5/31 Vincent Pérès vincent.pe...@gmail.com Hello, I built a feature which allow users to search for other user thanks to a dynamic text box. Like facebook, when you search for your friends, the name is display in a javascript dropdown list with a small picture. But I'm not completely happy with the search... I'm using a standard search like ?q=vincent and I get back the results list. If I type 'vinc' I will not get any results (But I would like to display all the users where the name start with 'vinc'). Maybe I need an extra param? I also tried the autosuggest, but I get a list of terms and not direct results... Could you suggest me some solr feature which could help me to get better results? Thanks a lot ! Vincent -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23804854.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23805261.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
It seems I can only search on the field 'text'. With the following url : http://localhost:8983/solr/select/?q=novelqt=dismaxfl=title_s,idversion=2.2start=0rows=10indent=ondebugQuery=on I get answers, but on the debug area, it seems it's only searching on the 'text' field (with or without 'qt' the results are displayed within the same order) : lst name=debug str name=rawquerystringnovel/str str name=querystringnovel/str − str name=parsedquery +DisjunctionMaxQuery((text:novel^0.5 | title_s:novel^5.0 | id:novel^10.0)~0.01) () /str − str name=parsedquery_toString +(text:novel^0.5 | title_s:novel^5.0 | id:novel^10.0)~0.01 () /str − lst name=explain − str name=33395 0.014641666 = (MATCH) sum of: 0.014641666 = (MATCH) max plus 0.01 times others of: 0.014641666 = (MATCH) weight(text:novel^0.5 in 114927), product of: 0.01362607 = queryWeight(text:novel^0.5), product of: 0.5 = boost 3.4734163 = idf(docFreq=10634, numDocs=43213) 0.007845918 = queryNorm 1.0745333 = (MATCH) fieldWeight(text:novel in 114927), product of: 1.4142135 = tf(termFreq(text:novel)=2) 3.4734163 = idf(docFreq=10634, numDocs=43213) 0.21875 = fieldNorm(field=text, doc=114927) /str etc. I should have a debug below with a search of the term into 'title_s' and 'id' no? Thanks for your answers ! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23649624.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
Hello, I'm sorry I wrote a mistake, I mean : http://localhost:8983/solr/listings/select/?q=novelqf=title_s^5.0fl=title_s+isbn_sversion=2.2start=0rows=5indent=ondebugQuery=on (using qf (Query Fields)) But it seems I need to add dismax as well and configure it by default in solr config? Thanks for your answer ! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23631022.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
I tried the following request after changed the dismax : http://localhost:8983/solr/listings/select/?q=novelqt=dismaxqf=title_s^2.0fl=title_s+isbn_sversion=2.2start=0rows=5indent=ondebugQuery=on But I don't get any results : lst name=debug str name=rawquerystringnovel/str str name=querystringnovel/str str name=parsedquery+DisjunctionMaxQuery((title_s:novel^2.0)~0.01) ()/str str name=parsedquery_toString+(title_s:novel^2.0)~0.01 ()/str lst name=explain/ str name=QParserDismaxQParser/str null name=altquerystring/ − arr name=boostfuncs str /str /arr − lst name=timing double name=time3.0/double − lst name=prepare double name=time1.0/double − lst name=org.apache.solr.handler.component.QueryComponent double name=time1.0/double /lst etc Do I need to add some specific attributes to my string fields? It looks now like this : dynamicField name=*_s type=string indexed=true stored=true omitNorms=false/ and below : copyField source=*_s dest=text/ Thanks !! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23631247.html Sent from the Solr - User mailing list archive at Nabble.com.
How to change the weight of the fields ?
Hello, I'm stuck my the boost feature... I'm doing the following query : http://localhost:8983/solr/select/?q=novelbf=title_s^5.0fl=title_s+isbn_sversion=2.2start=0rows=5indent=ondebugQuery=on Below a part of my debug (the results are correct) : lst name=debug str name=rawquerystringnovel/str str name=querystringnovel/str str name=parsedquerytext:novel/str str name=parsedquery_toStringtext:novel/str − lst name=explain − str name=Listing:397 0.6041825 = (MATCH) fieldWeight(text:novel in 395), product of: 1.7320508 = tf(termFreq(text:novel)=3) 3.1892564 = idf(docFreq=55, numDocs=500) 0.109375 = fieldNorm(field=text, doc=395) /str − str name=Listing:445 0.51787066 = (MATCH) fieldWeight(text:novel in 443), product of: 1.7320508 = tf(termFreq(text:novel)=3) 3.1892564 = idf(docFreq=55, numDocs=500) 0.09375 = fieldNorm(field=text, doc=443) /str It seems Solr is only evaluating the 'text' field and skip the 'title_s' field. The line in the schema for dynamic string is : dynamicField name=*_s type=string indexed=true stored=true omitNorms=false / Do I need to enable something else to be able to change the weight of my fields? Thanks very much Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23619971.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to update only few fields in a document
Hello, I went through the ticket but it seems that nothing is envisaged until version 1.5 Thanks for your answer !! Vincent Otis Gospodnetic wrote: Vincent, Unfortunately things haven't changed yet. If all your fields are stored, have a look at SOLR-139. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Vincent Pérès vincent.pe...@gmail.com To: solr-user@lucene.apache.org Sent: Friday, May 15, 2009 9:45:06 AM Subject: How to update only few fields in a document Hello, I did just find only post about updating document, maybe things evolved since that time. I need to update a field in few thousand documents in one time (or multiple request), but I wouldn't like to have to add a new document instead of the current one (I mean it's how it works if I well understand). Example : curl http://localhost:8982/solr/update --data-binary ' name=id15 name=id22' -H 'Content-type:text/xml; charset=utf-8' This request will replace the current documents by two new one. The documents contains big text parts and I wouldn't have to send them every time. Is there any feature which could allow me to do that? Thanks ! Vincent -- View this message in context: http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23560169.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23595879.html Sent from the Solr - User mailing list archive at Nabble.com.
How to update only few fields in a document
Hello, I did just find only post about updating document, maybe things evolved since that time. I need to update a field in few thousand documents in one time (or multiple request), but I wouldn't like to have to add a new document instead of the current one (I mean it's how it works if I well understand). Example : curl http://localhost:8982/solr/update --data-binary 'adddocfield name=id1/fieldfield name=rating5/field/docdocfield name=id2/fieldfield name=rating2/field/doc/add' -H 'Content-type:text/xml; charset=utf-8' This request will replace the current documents by two new one. The documents contains big text parts and I wouldn't have to send them every time. Is there any feature which could allow me to do that? Thanks ! Vincent -- View this message in context: http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23560169.html Sent from the Solr - User mailing list archive at Nabble.com.
DataImportHandler with multiple values
Hello, I'm trying to import a simple book table with the full-import command. The datas are stored in mysql. It worked well when I tried to import few fields from the 'book' table : title, author, publisher etc. Now I would like to create a facet (multi valued field) with the categories which belong to the book. There is my sql request to get the list of categories from a book (009959241X for example, return 7 categories) : SELECT abn.name AS cat, ab.isbn AS isbn_temp FROM (amazon_books AS ab LEFT JOIN amazon_book_browse_nodes AS abbn ON ab.isbn = abbn.amazon_book_id) LEFT JOIN amazon_browse_nodes AS abn ON abbn.amazon_browse_node_id = abn.id WHERE ab.isbn = '009959241X' I tried to integrate it on my dataconfig : dataConfig dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost:33061/completelynovel user=root password= / document name=books entity name=book pk=ID query=select isbn, listing_id AS id, title, publisher_name, author_name AS author_name_s from amazon_books where publisher_name IS NOT NULL AND author_name IS NOT NULL LIMIT 0, 10 field column=ID name=id / field column=ISBN name=isbn / field column=TITLE name=title / field column=PUBLISHER_NAME name=publisher_name / field column=AUTHOR_NAME_S name=author_name_s / entity name=book_category pk=id query=SELECT abn.name AS cat, ab.isbn AS isbn_temp FROM (amazon_books AS ab LEFT JOIN amazon_book_browse_nodes AS abbn ON ab.isbn = abbn.amazon_book_id) LEFT JOIN amazon_browse_nodes AS abn ON abbn.amazon_browse_node_id = abn.id WHERE ab.isbn = '${book.ISBN}' field column=cat name=cat / /entity /entity /document /dataConfig And my solr schema : field name=id type=sint indexed=true stored=true required=true / field name=isbn type=string indexed=true stored=true / field name=title type=string indexed=true stored=true / field name=publisher_name type=string indexed=true stored=true/ field name=cat type=text_ws indexed=true stored=true multiValued=true omitNorms=true termVectors=true / And below standart solr 1.4 dynamics fields... Ten fields are well created... but without the 'cat' multi value field. doc arr name=author_name_s strTerry Pratchett/str /arr int name=id47/int str name=isbn0552124753/str str name=publisher_nameCorgi Books/str date name=timestamp2009-04-13T12:54:38.553Z/date str name=titleThe Colour of Magic (Discworld Novel)/str /doc I guess I missed something, could you help me or redirect me to the right doc? Thank you ! Vincent -- View this message in context: http://www.nabble.com/DataImportHandler-with-multiple-values-tp23022195p23022195.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: DataImportHandler with multiple values
I changed the ISBN to lowercase (and the other fields as well) and it works ! Thanks very much ! -- View this message in context: http://www.nabble.com/DataImportHandler-with-multiple-values-tp23022195p23023374.html Sent from the Solr - User mailing list archive at Nabble.com.
What are the way to update / delete solr datas?
Hello, I'm trying to find the best way to update / delete datas according to my project (developed with javascript and rails). I would like to do something like that : http://localhost:8983/solr/update/?q=id:1rate=4 and http://localhost:8983/solr/delete/?q=id:1 Is it possible ? But I found only these two ways : http://localhost:8983/solr/update/csv?commit=trueseparator=%09escape=\stream.file=/tmp/result.text or using an xml file with that : deletequeryload_id:20070424150841/query/delete The last possibility is to use the solr-ruby library. Is there any way I forgot ? Thanks, Vincent -- View this message in context: http://www.nabble.com/What-are-the-way-to-update---delete-solr-datas--tp20268507p20268507.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: What are the way to update / delete solr datas?
Thanks for your quick answer. I'm using only HTTP to display my results, that's why I would like to continue with this way. If I can use HTTP instead of solr, it will be better for me. Erik Hatcher wrote: On Oct 31, 2008, at 11:40 AM, Vincent Pérès wrote: The last possibility is to use the solr-ruby library. If you're using Ruby, that's what I'd use. Were your other proposals to still do those calls from Ruby, but with the HTTP library directly? Erik -- View this message in context: http://www.nabble.com/What-are-the-way-to-update---delete-solr-datas--tp20268507p20268773.html Sent from the Solr - User mailing list archive at Nabble.com.
How to get the min and max values from facets?
Hello, I'm using Solr 1.3. I would like to get only minimum and maximum values from a facet. In fact I'm using a range to get the results : [value TO value], and I don't need to get the facets list in my XML results (which could be more than hundred thousands)... so, I have to display the range (minimum and maximum values) from a facet. Is there any way to do that? I found the new statistics components, follow the link : http://wiki.apache.org/solr/StatsComponent But it's for solr 1.4. Does anyone have any idea? Thank you ! Vincent -- View this message in context: http://www.nabble.com/How-to-get-the-min-and-max-values-from-facets--tp20243462p20243462.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Create custom facets after building index
Hello ! Thank you it is working. I've done a query, and my facet query is : facet_queries:{ published_year_facet:[1999 TO 2005]:95, rating_facet:[3 TO 3.99]:25, rating_facet:[1 TO 1.99]:1}, Is it possible to 'group' kind of queries (published together, rating together etc.) ? Or I have to match every query with regex ? Thanks ! -- View this message in context: http://www.nabble.com/Create-custom-facets-after-building-index-tp20086166p20091818.html Sent from the Solr - User mailing list archive at Nabble.com.