boost date for fresh result query
Hi I am facing problem to boost on date field. I have following field in schema field name=release_date type=date indexed=true stored=true/ solr version 3.4 I don't want to sort by date but want to give 50 to 60% boost those result which have latest date... following are the query : http://localhost:8083/solr/movie/select/?defType=dismaxq=titanicfq=title,tagsqf=tags ^2.0,title^1.0mm=1bf=recip(rord(release_date),1,1000,1000) but it seems the result has no effect because still all old date results are on top or I am unable to make query syntax. Is there anything missing in query.. Please help me to make the function possible.. thanks regards Jonty
Re: boost date for fresh result query
please suggest me I am stuck here.. On Mon, May 28, 2012 at 3:21 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi I am facing problem to boost on date field. I have following field in schema field name=release_date type=date indexed=true stored=true/ solr version 3.4 I don't want to sort by date but want to give 50 to 60% boost those result which have latest date... following are the query : http://localhost:8083/solr/movie/select/?defType=dismaxq=titanicfq=title,tagsqf=tags ^2.0,title^1.0mm=1bf=recip(rord(release_date),1,1000,1000) but it seems the result has no effect because still all old date results are on top or I am unable to make query syntax. Is there anything missing in query.. Please help me to make the function possible.. thanks regards Jonty
Sorting result first which come first in sentance
Hi all, I need suggetion: I Hi all, I need suggetion: I have many title like: 1 bomb blast in kabul 2 kabul bomb blast 3 3 people killed in serial bomb blast in kabul I want 2nd result should come first while user search by kabul. Because kabul is on 1st postion in that sentance. Similarly 1st result should come on 2nd and 3rd should come last. Please suggest me hot to implement this.. Regard Jonty
Re: Sorting result first which come first in sentance
I am using solr version 3.4
How to increase maxConnection and maxConnectionPerHost in SolrJ
Hi All, I have two tomcat on same server. One is for Solr and other is my application server. I am conneting solr server with solrj from application server. As I am connecting locally so the default connection seems to be very less. My server stop responding every few hour only up when I reset the application server. When I removing the module of search then it working perfectly. So I think I need to increase the maxConnection and maxConnectionPerHost. I try following code which I find in the mailing list but no effect. I have multi core setup and using seperate connection for each core but same instance throughout application. I also increase the maxThread to 500 on both tomcat. please help. regards Jonty
Re: java.net.SocketException: Too many open files
Hi Kuli, Did you get the solution of this problem? I am still facing this problem. Please help me to overcome this problem. regards On Wed, Oct 26, 2011 at 1:16 PM, Michael Kuhlmann k...@solarier.de wrote: Hi; we have a similar problem here. We already raised the file ulimit on the server to 4096, but this only defered the problem. We get a TooManyOpenFilesException every few months. The problem has nothing to do with real files. When we had the last TooManyOpenFilesException, we investigated with netstat -a and saw that there were about 3900 open sockets in Jetty. Curiously, we only have one SolrServer instance per Solr client, and we only have three clients (our running web servers). We have set defaultMaxConnectionsPerHost to 20 and maxTotalConnections to 100. There should be room enough. Sorry that I can't help you, we still have not solved tghe problem on our own. Greetings, Kuli Am 25.10.2011 22:03, schrieb Jonty Rhods: Hi, I am using solrj and for connection to server I am using instance of the solr server: SolrServer server = new CommonsHttpSolrServer( http://localhost:8080/solr/core0;); I noticed that after few minutes it start throwing exception java.net.SocketException: Too many open files. It seems that it related to instance of the HttpClient. How to resolved the instances to a certain no. Like connection pool in dbcp etc.. I am not experienced on java so please help to resolved this problem. solr version: 3.4 regards Jonty
Re: server stop responding in few hours due to CLOSE_WAIT
Hi, I set the parameter as describe in the blog link but still same problem I am facing. In test class when I run it, It run perfectly few minutes but after certain thread it start responding slow and at same time the CLOSE_WAIT start increasing. I also notice that when I terminate test class all CLOSE_WAIT become TIME_WAIT and in a minute TIME_WAIT cleaned. Please help.. On Fri, Jan 13, 2012 at 11:10 PM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, It sounds like disabled http keep alive (connection cache). Here is the solution http://blog.griddynamics.com/2011/04/fast-hessian-methods-leads-to.html for jdk's http client. Unfortunately I have no experience with your Commons Http Client, but cm.closeIdleConnections(0L) looks very suspicious. Please let me know whether it works for you. Regards On Fri, Jan 13, 2012 at 7:24 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi All , I am facing problem of too many CLOSE_WAIT. My env is : solr 3.4 in Linux RHEL 5.2. I am getting around 1 million request per day on application server on my production. Production server is communicating locally with solr server. I have 5 core setup and for each core I am using seprate instance of solrServer with following snippet of code. MultiThreadedHttpConnectionManager cm=new MultiThreadedHttpConnectionManager(); cm.getParams().setMaxTotalConnections(100); cm.getParams().setDefaultMaxConnectionsPerHost(100); cm.closeIdleConnections(0L); HttpClient httpClient=new HttpClient(cm); server=new CommonsHttpSolrServer(url, httpClient); At starting my server responding fine but as time progress it start responding slow and then stop working (within a day). Only solution at that time is to reset the application server. What I notice is at the time when server start responding slow that time CLOSE_WAIT increased. When I reset the application server CLOSE_WAIT become 0. I search and find it is due to CLOSE_WAIT. Please suggest me how to resolve this CLOSE_WAIT issue. BR Jonty -- Sincerely yours Mikhail Khludnev Lucid Certified Apache Lucene/Solr Developer Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
server stop responding in few hours due to CLOSE_WAIT
Hi All , I am facing problem of too many CLOSE_WAIT. My env is : solr 3.4 in Linux RHEL 5.2. I am getting around 1 million request per day on application server on my production. Production server is communicating locally with solr server. I have 5 core setup and for each core I am using seprate instance of solrServer with following snippet of code. MultiThreadedHttpConnectionManager cm=new MultiThreadedHttpConnectionManager(); cm.getParams().setMaxTotalConnections(100); cm.getParams().setDefaultMaxConnectionsPerHost(100); cm.closeIdleConnections(0L); HttpClient httpClient=new HttpClient(cm); server=new CommonsHttpSolrServer(url, httpClient); At starting my server responding fine but as time progress it start responding slow and then stop working (within a day). Only solution at that time is to reset the application server. What I notice is at the time when server start responding slow that time CLOSE_WAIT increased. When I reset the application server CLOSE_WAIT become 0. I search and find it is due to CLOSE_WAIT. Please suggest me how to resolve this CLOSE_WAIT issue. BR Jonty
java.net.SocketException: Too many open files
Hi, I am using solrj and for connection to server I am using instance of the solr server: SolrServer server = new CommonsHttpSolrServer( http://localhost:8080/solr/core0;); I noticed that after few minutes it start throwing exception java.net.SocketException: Too many open files. It seems that it related to instance of the HttpClient. How to resolved the instances to a certain no. Like connection pool in dbcp etc.. I am not experienced on java so please help to resolved this problem. solr version: 3.4 regards Jonty
Re: java.net.SocketException: Too many open files
Hi Yonik, thanks for reply. Currently I have more than 50 classes and every class have their own SolrServer server = new CommonsHttpSolrServer( http://localhost:8080/solr/core0;); Majority of classes connect to core0 however there are many cores which is connecting from different classes. My senario is expecting 40 to 50 hit on server every day and the server is deployed on tomcat 6.20 with 12GB heap size to catalina. My OS is Red hat Linux (production) and using Ubuntu as development server. Logically I can make a common class for connecting solr server but now question is : 1. If I using a common class then I must have to use max connection on httpclient what will the ideal default setting for my current problem. 2. I am expecting concurrent connection at pick time is minimum 5000 hits to solr server. 3. If I will use SolrServer server = new CommonsHttpSolrServer( http://localhost:8080/solr/core0;); using common class across all classes then will it help to resolve current problem with the current load I dont want my user experience slow response from solr server to get result. 4. As other users overcome with this issue to increase or to make unlimited TCP/IP setting on OS level. Is it right approach? As I am newly shift to Java language so any piece of code will much appreciated and will much easier for me to understand. thanks. regards Jonty On Wed, Oct 26, 2011 at 1:37 AM, Yonik Seeley yo...@lucidimagination.comwrote: On Tue, Oct 25, 2011 at 4:03 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi, I am using solrj and for connection to server I am using instance of the solr server: SolrServer server = new CommonsHttpSolrServer( http://localhost:8080/solr/core0;); Are you reusing the server object for all of your requests? By default, Solr and SolrJ use persistent connections, meaning that sockets are reused and new ones are not opened for every request. -Yonik http://www.lucidimagination.com I noticed that after few minutes it start throwing exception java.net.SocketException: Too many open files. It seems that it related to instance of the HttpClient. How to resolved the instances to a certain no. Like connection pool in dbcp etc.. I am not experienced on java so please help to resolved this problem. solr version: 3.4 regards Jonty
Re: SolrServer instances
do I also required to close the connection from solr server (CommonHttpSolrServer). regards On Fri, Aug 26, 2011 at 9:45 AM, Jonty Rhods jonty.rh...@gmail.com wrote: Deal all please help I am stuck here as I have not much experience.. thanks On Thu, Aug 25, 2011 at 6:51 PM, Jonty Rhods jonty.rh...@gmail.comwrote: Hi All, I am using SolrJ (3.1) and Tomcat 6.x. I want to open solr server once (20 concurrence) and reuse this across all the site. Or something like connection pool like we are using for DB (ie Apache DBCP). There is a way to use static method which is a way but I want better solution from you people. I read one threade where Ahmet suggest to use something like that String serverPath = http://localhost:8983/solr;; HttpClient client = new HttpClient(new MultiThreadedHttpConnectionManager()); URL url = new URL(serverPath); CommonsHttpSolrServer solrServer = new CommonsHttpSolrServer(url, client); But how to use instance of this across all class. Please suggest. regards Jonty
SolrServer instances
Hi All, I am using SolrJ (3.1) and Tomcat 6.x. I want to open solr server once (20 concurrence) and reuse this across all the site. Or something like connection pool like we are using for DB (ie Apache DBCP). There is a way to use static method which is a way but I want better solution from you people. I read one threade where Ahmet suggest to use something like that String serverPath = http://localhost:8983/solr;; HttpClient client = new HttpClient(new MultiThreadedHttpConnectionManager()); URL url = new URL(serverPath); CommonsHttpSolrServer solrServer = new CommonsHttpSolrServer(url, client); But how to use instance of this across all class. Please suggest. regards Jonty
Re: SolrServer instances
Deal all please help I am stuck here as I have not much experience.. thanks On Thu, Aug 25, 2011 at 6:51 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi All, I am using SolrJ (3.1) and Tomcat 6.x. I want to open solr server once (20 concurrence) and reuse this across all the site. Or something like connection pool like we are using for DB (ie Apache DBCP). There is a way to use static method which is a way but I want better solution from you people. I read one threade where Ahmet suggest to use something like that String serverPath = http://localhost:8983/solr;; HttpClient client = new HttpClient(new MultiThreadedHttpConnectionManager()); URL url = new URL(serverPath); CommonsHttpSolrServer solrServer = new CommonsHttpSolrServer(url, client); But how to use instance of this across all class. Please suggest. regards Jonty
Re: commit time and lock
Thanks for clarity. One more thing I want to know about optimization. Right now I am planning to optimize the server in 24 hour. Optimization is also time taking ( last time took around 13 minutes), so I want to know that : 1. when optimization is under process that time will solr server response or not? 2. if server will not response then how to do optimization of server fast or other way to do optimization so our user will not have to wait to finished optimization process. regards Jonty On Fri, Jul 22, 2011 at 2:44 PM, Pierre GOSSE pierre.go...@arisem.comwrote: Solr still respond to search queries during commit, only new indexations requests will have to wait (until end of commit?). So I don't think your users will experience increased response time during commits (unless your server is much undersized). Pierre -Message d'origine- De : Jonty Rhods [mailto:jonty.rh...@gmail.com] Envoyé : jeudi 21 juillet 2011 20:27 À : solr-user@lucene.apache.org Objet : Re: commit time and lock Actually i m worried about the response time. i k commiting around 500 docs in every 5 minutes. as i know,correct me if i m wrong; at the time of commiting solr server stop responding. my concern is how to minimize the response time so user not need to wait. or any other logic will require for my case. please suggest. regards jonty On Tuesday, June 21, 2011, Erick Erickson erickerick...@gmail.com wrote: What is it you want help with? You haven't told us what the problem you're trying to solve is. Are you asking how to speed up indexing? What have you tried? Have you looked at: http://wiki.apache.org/solr/FAQ#Performance? Best Erick On Tue, Jun 21, 2011 at 2:16 AM, Jonty Rhods jonty.rh...@gmail.com wrote: I am using solrj to index the data. I have around 5 docs indexed. As at the time of commit due to lock server stop giving response so I was calculating commit time: double starttemp = System.currentTimeMillis(); server.add(docs); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); It taking around 9 second to commit the 5000 docs with 15 fields. However I am not confirm the lock time of index whether it is start since server.add(docs); time or server.commit(); time only. If I am changing from above to following server.add(docs); double starttemp = System.currentTimeMillis(); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); then commit time becomes less then 1 second. I am not sure which one is right. please help. regards Jonty
previous and next rows of current record
Hi, Is there any special query in solr to get the previous and next record of current record. I am getting single record detail using id from solr server. I need to get next and previous on detail page. regards Jonty
Re: previous and next rows of current record
Pls help.. On Thursday, July 21, 2011, Jonty Rhods jonty.rh...@gmail.com wrote: Hi, Is there any special query in solr to get the previous and next record of current record.I am getting single record detail using id from solr server. I need to get next and previous on detail page. regardsJonty
Re: random record from solr server
Thanks.. On Monday, July 18, 2011, Ahmet Arslan iori...@yahoo.com wrote: How can I get random 100 record from last two days record from solr server. I am using solr 3.1 Hello, add this random field definition to you schema.xml fieldType name=random class=solr.RandomSortField indexed=true / dynamicField name=random_* type=random / Generate some seed value ( e.g. 125) at query time, and issue a query something like this: q:add_date[NOW-2DAYS TO *]sort=random_125start=0rows=100 If you use different seed values each time you will get random 100 record in each request. I assume you have date field to store add date or similar.
Re: commit time and lock
Actually i m worried about the response time. i k commiting around 500 docs in every 5 minutes. as i know,correct me if i m wrong; at the time of commiting solr server stop responding. my concern is how to minimize the response time so user not need to wait. or any other logic will require for my case. please suggest. regards jonty On Tuesday, June 21, 2011, Erick Erickson erickerick...@gmail.com wrote: What is it you want help with? You haven't told us what the problem you're trying to solve is. Are you asking how to speed up indexing? What have you tried? Have you looked at: http://wiki.apache.org/solr/FAQ#Performance? Best Erick On Tue, Jun 21, 2011 at 2:16 AM, Jonty Rhods jonty.rh...@gmail.com wrote: I am using solrj to index the data. I have around 5 docs indexed. As at the time of commit due to lock server stop giving response so I was calculating commit time: double starttemp = System.currentTimeMillis(); server.add(docs); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); It taking around 9 second to commit the 5000 docs with 15 fields. However I am not confirm the lock time of index whether it is start since server.add(docs); time or server.commit(); time only. If I am changing from above to following server.add(docs); double starttemp = System.currentTimeMillis(); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); then commit time becomes less then 1 second. I am not sure which one is right. please help. regards Jonty
Re: previous and next rows of current record
Hi in my case there is no id sequence. id is generated sequence wise for all category. but when we filter by category then same id become random. If i m on detail page which have id 5 and nrxt id is 9 so on same page my requirment is to get next id is 9. On Thursday, July 21, 2011, Bob Sandiford bob.sandif...@sirsidynix.com wrote: Well, it sort of depends on what you mean by the 'previous' and the 'next' record. Do you have some type of sequencing built into your concept of your solr / lucene indexes? Do you have sequential id's? i.e. What's the use case, and what's the data available to support your use case? Bob Sandiford | Lead Software Engineer | SirsiDynix P: 800.288.8020 X6943 | bob.sandif...@sirsidynix.com www.sirsidynix.com -Original Message- From: Jonty Rhods [mailto:jonty.rh...@gmail.com] Sent: Thursday, July 21, 2011 2:18 PM To: solr-user@lucene.apache.org Subject: Re: previous and next rows of current record Pls help.. On Thursday, July 21, 2011, Jonty Rhods jonty.rh...@gmail.com wrote: Hi, Is there any special query in solr to get the previous and next record of current record.I am getting single record detail using id from solr server. I need to get next and previous on detail page. regardsJonty
random record from solr server
Hi, How can I get random 100 record from last two days record from solr server. I am using solr 3.1 regards Jonty
commit time and lock
I am using solrj to index the data. I have around 5 docs indexed. As at the time of commit due to lock server stop giving response so I was calculating commit time: double starttemp = System.currentTimeMillis(); server.add(docs); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); It taking around 9 second to commit the 5000 docs with 15 fields. However I am not confirm the lock time of index whether it is start since server.add(docs); time or server.commit(); time only. If I am changing from above to following server.add(docs); double starttemp = System.currentTimeMillis(); server.commit(); System.out.println(total time in commit = + (System.currentTimeMillis() - starttemp)/1000); then commit time becomes less then 1 second. I am not sure which one is right. please help. regards Jonty
query about the server configuration
Dear all, I am quite new and not work on solr for heavy request. I have following server configuration: 16GB RAM 16 CPU I need to index update in every minutes and at least more than 5000 docs per day. Size of the data per day will be around 50 MB. I am expecting 10 to 30 concurrent hit on server which is 2 million hits per day and around 30 to 40 concurrent user at peak our. Right now I had configure core and using static method to call solr server in solrj (SolrServer server = new HttpSolrServer();). I am worried that at peak our static instance of the server in solrj will not able to perform the response and it will become slow. Is there any way to open more then one connection of server instance in the SolrJ like connection pool which we are using in Database related connection pooling (Apache DBCP or Hibernate). Please help me to configure the server as my heavy requirements. thanks for your help. regards jonty
Re: about the SolrServer server = new CommonsHttpSolrServer(URL);
for heavy use (30 to 40 concurrent user) will it work. How to open and maintain more connection at a time like connection pool. So user cat receive fast response.. regards On Fri, Jun 17, 2011 at 12:50 PM, Ahmet Arslan iori...@yahoo.com wrote: SolrServer server = new CommonsHttpSolrServer(URL); through out the class. How can I improve the connection, in my case: should I need to close the server after fetching the result or CommonsHttpSolrServer(URL); will maintain at their end. There is other way: I can make this as static and can use through out the classes. As wiki [1] says, you must use the same instalce through out all of the classes. [1] http://wiki.apache.org/solr/Solrj#CommonsHttpSolrServer
Jonty Rhods wants to chat
--- Jonty Rhods wants to stay in better touch using some of Google's coolest new products. If you already have Gmail or Google Talk, visit: http://mail.google.com/mail/b-26ddccf9dc-56859aec19-TvU2zC9tjv8Q_u4jzhyceWuZkgs You'll need to click this link to be able to chat with Jonty Rhods. To get Gmail - a free email account from Google with over 2,800 megabytes of storage - and chat with Jonty Rhods, visit: http://mail.google.com/mail/a-26ddccf9dc-56859aec19-TvU2zC9tjv8Q_u4jzhyceWuZkgs Gmail offers: - Instant messaging right inside Gmail - Powerful spam protection - Built-in search for finding your messages and a helpful way of organizing emails into conversations - No pop-up ads or untargeted banners - just text ads and related information that are relevant to the content of your messages All this, and its yours for free. But wait, there's more! By opening a Gmail account, you also get access to Google Talk, Google's instant messaging service: http://www.google.com/talk/ Google Talk offers: - Web-based chat that you can use anywhere, without a download - A contact list that's synchronized with your Gmail account - Free, high quality PC-to-PC voice calls when you download the Google Talk client We're working hard to add new features and make improvements, so we might also ask for your comments and suggestions periodically. We appreciate your help in making our products even better! Thanks, The Google Team To learn more about Gmail and Google Talk, visit: http://mail.google.com/mail/help/about.html http://www.google.com/talk/about.html (If clicking the URLs in this message does not work, copy and paste them into the address bar of your browser).
Re: query about the server configuration
I forgot an important point that I need to commit the server in 2 to 5 minutes.. please help.. regards On Sun, Jun 19, 2011 at 11:29 PM, Ranveer ranveer.s...@gmail.com wrote: Please help I am also in same situation. regards On Sunday 19 June 2011 12:59 PM, Jonty Rhods wrote: Dear all, I am quite new and not work on solr for heavy request. I have following server configuration: 16GB RAM 16 CPU I need to index update in every minutes and at least more than 5000 docs per day. Size of the data per day will be around 50 MB. I am expecting 10 to 30 concurrent hit on server which is 2 million hits per day and around 30 to 40 concurrent user at peak our. Right now I had configure core and using static method to call solr server in solrj (SolrServer server = new HttpSolrServer();). I am worried that at peak our static instance of the server in solrj will not able to perform the response and it will become slow. Is there any way to open more then one connection of server instance in the SolrJ like connection pool which we are using in Database related connection pooling (Apache DBCP or Hibernate). Please help me to configure the server as my heavy requirements. thanks for your help. regards jonty
synonym configuration logic require
Hi all, I am pretty new in solr. I want to configure synonym for forwarding result (for exact match). my schema is : fieldType name=text_sync class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ /analyzer /fieldType There is name field which is define as text_sync. When I am searching by newyork show its working fine and giving exact result (there is exact value newyork show in index). Now I want if someone search by newyork live show I want to forward it to newyork show, so it will match exactly. I put newyork live show=newyork show in synonym but it is not working. When changing tokenizer (query time) to standardtokenizerFilterFactory it work but same time my normal result newyork show stop giving exact result. My requirement is to highlight the background in html when exact match come. so I want to use synonym.. please help me to get desire result or let me know if any other better way to get my requirement...
Implementing synonym NewBie
Hi All, I want to use synonym for my search. Still I am in learning phase of solr. So please help me to implement synonym in my search. according to wiki synonym can be implemented in two ways. 1 at index time 2 at search time] I have combination 10 of phrase for synonym so which will be better in my case. something like : live show in new york=live show in clifornia= live show = live show in DC = live show in USA is synonym will effect my original search? thanks with regards Jonty
Re: logic required for newbie
Again thanks for reply.. Actually I am getting result. But I am getting all column of the rows. I want to remove unnecessary column. In case of q=piza hut.. then I want to get only landmark4piza hut/landmark4. Same if search query change to ford motor then want only landmark5ford motor/landmark5. more example if query is piza hut ford motor then expected result should be.. id1/id namesome name/name user_iduser_id/user_id locationnew york/location countryUSA/country landmark4piza hut/landmark4 landmark5ford motor/landmark5 In above expected result.. landmark15th avenue/landmark1, landmark2ms departmental store/landmark2, landmark3base bakery/landmark3 , has been removed because it not carrying any matched text.. More generalized form I want to filter all unmatched column which not carrying matched query. Right now I am getting proper result but getting full column set. My requirement is only match landmark should return.. So I want to filter the column which carry text (match to query). hoping someone will help me to clear my concept.. regards On Thu, Jul 29, 2010 at 9:41 AM, rajini maski rajinima...@gmail.com wrote: First of all I hope that in schema you have mentioned for fields indexed=true and stored=true... Next if you have done so... and now just search as q=landmark:piza... you will get one result set only.. Note : There is one constraint about applying analyzers and tokenizers... IF you apply white space tokenizer...that is , data type=text_ws. The only you will get result set of piza hut even when you query for piza... If no tokenizer applied..You will not get it... I hope this was needed reply..If something elseyou can easy question..;) On Wed, Jul 28, 2010 at 8:42 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi thanks for reply.. Actually requirement is diffrent (sorry if I am unable to clerify in first mail). basically follwoing are the fields name in schema as well: 1. id 2. name 3. user_id 4. location 5. country 6. landmark1 7. landmark2 8. landmark3 9. landmark4 10. landmark5 which carrying text... for example: id1/id namesome name/name user_iduser_id/user_id locationnew york/location countryUSA/country landmark15th avenue/landmark1 landmark2ms departmental store/landmark2 landmark3base bakery/landmark3 landmark4piza hut/landmark4 landmark5ford motor/landmark5 now if user search by piza then expected result like: id1/id namesome name/name user_iduser_id/user_id locationnew york/location countryUSA/country landmark4piza hut/landmark4 it means I want to ignore all other landmark which not match. By filter we can filter the fields but here I dont know the the field name because it depends on text match. is there any other solution.. I am ready to change in schema or in logic. I am using solrj. please help me I stuck here.. with regards On Wed, Jul 28, 2010 at 7:22 PM, rajini maski rajinima...@gmail.com wrote: you can index each of these field separately... field1- Id field2- name field3-user_id field4-country. field7- landmark While quering you can specify q=Landmark9 This will return you results.. And if you want only particular fields in output.. use the fl parameter in query... like http://localhost:8090/solr/select? indent=onq=landmark9fl=ID,user_id,country,landmark This will give your desired solution.. On Wed, Jul 28, 2010 at 12:23 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi All, I am very new and learning solr. I have 10 column like following in table 1. id 2. name 3. user_id 4. location 5. country 6. landmark1 7. landmark2 8. landmark3 9. landmark4 10. landmark5 when user search for landmark then I want to return only one landmark which match. Rest of the landmark should ingnored.. expected result like following if user search by landmark2.. 1. id 2. name 3. user_id 4. location 5. country 7. landmark2 or if search by landmark9 1. id 2. name 3. user_id 4. location 5. country 9. landmark9 please help me to design the schema for this kind of requirement... thanks with regards
logic required for newbie
Hi All, I am very new and learning solr. I have 10 column like following in table 1. id 2. name 3. user_id 4. location 5. country 6. landmark1 7. landmark2 8. landmark3 9. landmark4 10. landmark5 when user search for landmark then I want to return only one landmark which match. Rest of the landmark should ingnored.. expected result like following if user search by landmark2.. 1. id 2. name 3. user_id 4. location 5. country 7. landmark2 or if search by landmark9 1. id 2. name 3. user_id 4. location 5. country 9. landmark9 please help me to design the schema for this kind of requirement... thanks with regards
Re: logic required for newbie
Hi thanks for reply.. Actually requirement is diffrent (sorry if I am unable to clerify in first mail). basically follwoing are the fields name in schema as well: 1. id 2. name 3. user_id 4. location 5. country 6. landmark1 7. landmark2 8. landmark3 9. landmark4 10. landmark5 which carrying text... for example: id1/id namesome name/name user_iduser_id/user_id locationnew york/location countryUSA/country landmark15th avenue/landmark1 landmark2ms departmental store/landmark2 landmark3base bakery/landmark3 landmark4piza hut/landmark4 landmark5ford motor/landmark5 now if user search by piza then expected result like: id1/id namesome name/name user_iduser_id/user_id locationnew york/location countryUSA/country landmark4piza hut/landmark4 it means I want to ignore all other landmark which not match. By filter we can filter the fields but here I dont know the the field name because it depends on text match. is there any other solution.. I am ready to change in schema or in logic. I am using solrj. please help me I stuck here.. with regards On Wed, Jul 28, 2010 at 7:22 PM, rajini maski rajinima...@gmail.com wrote: you can index each of these field separately... field1- Id field2- name field3-user_id field4-country. field7- landmark While quering you can specify q=Landmark9 This will return you results.. And if you want only particular fields in output.. use the fl parameter in query... like http://localhost:8090/solr/select? indent=onq=landmark9fl=ID,user_id,country,landmark This will give your desired solution.. On Wed, Jul 28, 2010 at 12:23 PM, Jonty Rhods jonty.rh...@gmail.com wrote: Hi All, I am very new and learning solr. I have 10 column like following in table 1. id 2. name 3. user_id 4. location 5. country 6. landmark1 7. landmark2 8. landmark3 9. landmark4 10. landmark5 when user search for landmark then I want to return only one landmark which match. Rest of the landmark should ingnored.. expected result like following if user search by landmark2.. 1. id 2. name 3. user_id 4. location 5. country 7. landmark2 or if search by landmark9 1. id 2. name 3. user_id 4. location 5. country 9. landmark9 please help me to design the schema for this kind of requirement... thanks with regards
logic for auto-index
Hi All, I am very new to solr as well as java too. I require to use solrj for indexing also require to index automatically once in 24 hour. I wrote java code for indexing now I want to do further coding for automatic process. Could you suggest or give me sample code for automatic index process.. please help.. with regards Jonty.
Re: logic for auto-index
Hi Peter, actually I want the index process should start automatically. right now I am doing mannually. same thing I want to start indexing when less load on server i.e. late night. So setting auto will fix my problem.. On Wed, Jun 2, 2010 at 2:00 PM, Peter Karich peat...@yahoo.de wrote: Hi Jonty, what is your specific problem? You could use a cronjob or the Java-lib called quartz to automate this task. Or did you mean replication? Regards, Peter. Hi All, I am very new to solr as well as java too. I require to use solrj for indexing also require to index automatically once in 24 hour. I wrote java code for indexing now I want to do further coding for automatic process. Could you suggest or give me sample code for automatic index process.. please help.. with regards Jonty.
Re: how to patch solr-236 in mac os
hi I tried to patch by following command on console patch -p0 SOLR-236-trunk.patch patching file solr/src/java/org/apache/solr/handler/component/CollapseComponent.java patching file solr/src/test/test-files/solr/conf/solrconfig-fieldcollapse.xml patching file solr/src/java/org/apache/solr/search/fieldcollapse/collector/DocumentGroupCountCollapseCollectorFactory.java patching file solr/src/java/org/apache/solr/search/fieldcollapse/collector/FieldValueCountCollapseCollectorFactory.java can't find file to patch at input line 995 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -- |Index: solr/src/java/org/apache/solr/search/DocSetHitCollector.java |=== |--- solr/src/java/org/apache/solr/search/DocSetHitCollector.java (revision 922957) |+++ solr/src/java/org/apache/solr/search/DocSetHitCollector.java (revision ) -- File to patch: Asking for filename, I check the but the file not exist.. please help... On Tue, May 11, 2010 at 11:29 PM, Erick Erickson erickerick...@gmail.comwrote: In Eclipse (you *may* need to have the subclipse plugin installed), just right-click on the projectteamapply patch and follow the wizard HTH Erick On Tue, May 11, 2010 at 12:50 PM, Jonty Rhods jonty.rh...@gmail.com wrote: hi David, thanks for quick reply.. please give me full command. so I can patch. what is meaning of [level]. As I write I had downloaded latest src from trunk.. So please also tell that, in terminal what will be command and from where I can run.. should I try patch -p[level] name_of_patch.patch thanks On Tue, May 11, 2010 at 10:02 PM, David Stuart david.stu...@progressivealliance.co.uk wrote: Hey, In osx you shoud be able to patch in the same way as on liux patch -p[level] name_of_patch.patch. You can do this from the shell including on the mac. David Stuart On 11 May 2010, at 17:15, Jonty Rhods jonty.rh...@gmail.com wrote: hi all, I am very new to solr. Now I required to patch solr (patch no 236). I download the latest src code and patch, but unable to finde suitable way to patch. I have eclipse installed. please guide me..
Re: how to patch solr-236 in mac os
I tried to patch from diffrent src folder but asking for file name to patch.. and I think some dependent patch also required.. please guide me to patch SOLR-236.patch (with dependent patch if any). If it will require then I will again download the src code from trunk.. thanks On Wed, May 12, 2010 at 11:53 AM, Jonty Rhods jonty.rh...@gmail.com wrote: hi I tried to patch by following command on console patch -p0 SOLR-236-trunk.patch patching file solr/src/java/org/apache/solr/handler/component/CollapseComponent.java patching file solr/src/test/test-files/solr/conf/solrconfig-fieldcollapse.xml patching file solr/src/java/org/apache/solr/search/fieldcollapse/collector/DocumentGroupCountCollapseCollectorFactory.java patching file solr/src/java/org/apache/solr/search/fieldcollapse/collector/FieldValueCountCollapseCollectorFactory.java can't find file to patch at input line 995 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -- |Index: solr/src/java/org/apache/solr/search/DocSetHitCollector.java |=== |--- solr/src/java/org/apache/solr/search/DocSetHitCollector.java (revision 922957) |+++ solr/src/java/org/apache/solr/search/DocSetHitCollector.java (revision ) -- File to patch: Asking for filename, I check the but the file not exist.. please help... On Tue, May 11, 2010 at 11:29 PM, Erick Erickson erickerick...@gmail.comwrote: In Eclipse (you *may* need to have the subclipse plugin installed), just right-click on the projectteamapply patch and follow the wizard HTH Erick On Tue, May 11, 2010 at 12:50 PM, Jonty Rhods jonty.rh...@gmail.com wrote: hi David, thanks for quick reply.. please give me full command. so I can patch. what is meaning of [level]. As I write I had downloaded latest src from trunk.. So please also tell that, in terminal what will be command and from where I can run.. should I try patch -p[level] name_of_patch.patch thanks On Tue, May 11, 2010 at 10:02 PM, David Stuart david.stu...@progressivealliance.co.uk wrote: Hey, In osx you shoud be able to patch in the same way as on liux patch -p[level] name_of_patch.patch. You can do this from the shell including on the mac. David Stuart On 11 May 2010, at 17:15, Jonty Rhods jonty.rh...@gmail.com wrote: hi all, I am very new to solr. Now I required to patch solr (patch no 236). I download the latest src code and patch, but unable to finde suitable way to patch. I have eclipse installed. please guide me..
Re: multivalue fields logic required
hi Marco, Thanks for quick reply.. I have another doubt: In 2nd solution: How to set flag for duplicate value. because I am not sure about the no fo duplicate rows (it could be random no..) so how can I set the flag.. thank On Wed, May 12, 2010 at 12:59 PM, Marco Martinez mmarti...@paradigmatecnologico.com wrote: Hi, 2º solution: Not use multiValue fields, instead use two single fields, in your example will be: doc1: dept: student1 city: city1 principalFlag:T doc2: dept: student2 city: city2 principalFlag:F So, if you search without specify any city or dept, you should put princiaplFlag:T for no get duplicate on your response. And if you specify a city or a dept, there is no need to specify the principalFlag because you will only get the result that match with your fields (you dont get duplicates). 3º solution: Do a postprocessing to eleminate the fields in your response that you dont need, i mean, get only the city and the dept that should be in the query response. Hope this will help Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2010/5/12 Jonty Rhods jonty.rh...@gmail.com Hi Marco, I am trying to patch for collapse component support (till now no luck).. In mean time I would like to know the 2nd and 3rd option you mentioned (logic in solrj).. with regards On Thu, May 6, 2010 at 2:36 PM, Marco Martinez mmarti...@paradigmatecnologico.com wrote: Hi Jonty, I think you have three possible solutions: 1. Use the collapse component with your name field for not have any duplicates documents. 2. Create a simple logic in your index with flags, like one flag to determine the first element of the same document (in your example you will have three differents documents and the fist one wiill have this flag=true). If the search only have name, you will have to set this flag to true, if not, the dept or the student will be defined and you will have one document returned. 3. Do a post-processing of your data. Maybe you will have more solutions but these are what i have thought right now. Regards, Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2010/5/6 Jonty Rhods jonty.rh...@gmail.com thanks :General solution is to index 3 different SolrDocument in your example. id and name fields will repeat themselves. All fields will be single-valued. if I am indexing 3 different field then if user is searching by name + dept then it will return duplicate value.. is there any other best possible way..? thanks On Thu, May 6, 2010 at 1:34 PM, Ahmet Arslan iori...@yahoo.com wrote: recently I start to work on solr, So I am still very new to use solr. Sorry if I am logically wrong. I have two table, parent and referenced (child). for that I set multivalue field following is my schema details field name=id type=string indexed=true stored=true required=true / field name=name type=text indexed=true stored=true/ field name=dept type=text indexed=true stored=true multiValued=true/ field name=city type=text indexed=true stored=true multiValued=true/ indexed data details: doc arr name=dept strstudent1/str strstudent2/str strstudent3/str /arr arr name=city strcity1/str strcity2/str strcity3/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc now my question is : When user is searching by city2 then I want to return employee2 and their id (for multi value field). something like: doc arr name=dept strstudent2/str /arr arr name=city strcity2/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc I had a similar need before. AFAIK you cannot do it with multivalued fields. The indexing order is preserved in multivalued field. May be you can post-process returned fields and capture correct position of matched city field, and use this index to display correct dept value. But this is easy if you are using string or integer type for city and dept. General solution is to index 3 different SolrDocument in your example. id and name fields will repeat themselves. All fields will be single-valued.
how to patch solr-236 in mac os
hi all, I am very new to solr. Now I required to patch solr (patch no 236). I download the latest src code and patch, but unable to finde suitable way to patch. I have eclipse installed. please guide me..
Re: how to patch solr-236 in mac os
hi David, thanks for quick reply.. please give me full command. so I can patch. what is meaning of [level]. As I write I had downloaded latest src from trunk.. So please also tell that, in terminal what will be command and from where I can run.. should I try patch -p[level] name_of_patch.patch thanks On Tue, May 11, 2010 at 10:02 PM, David Stuart david.stu...@progressivealliance.co.uk wrote: Hey, In osx you shoud be able to patch in the same way as on liux patch -p[level] name_of_patch.patch. You can do this from the shell including on the mac. David Stuart On 11 May 2010, at 17:15, Jonty Rhods jonty.rh...@gmail.com wrote: hi all, I am very new to solr. Now I required to patch solr (patch no 236). I download the latest src code and patch, but unable to finde suitable way to patch. I have eclipse installed. please guide me..
multivalue fields logic required
Hi, recently I start to work on solr, So I am still very new to use solr. Sorry if I am logically wrong. I have two table, parent and referenced (child). for that I set multivalue field following is my schema details field name=id type=string indexed=true stored=true required=true / field name=name type=text indexed=true stored=true/ field name=dept type=text indexed=true stored=true multiValued=true/ field name=city type=text indexed=true stored=true multiValued=true/ indexed data details: doc arr name=dept strstudent1/str strstudent2/str strstudent3/str /arr arr name=city strcity1/str strcity2/str strcity3/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc now my question is : When user is searching by city2 then I want to return employee2 and their id (for multi value field). something like: doc arr name=dept strstudent2/str /arr arr name=city strcity2/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc please suggest..
Re: multivalue fields logic required
thanks :General solution is to index 3 different SolrDocument in your example. id and name fields will repeat themselves. All fields will be single-valued. if I am indexing 3 different field then if user is searching by name + dept then it will return duplicate value.. is there any other best possible way..? thanks On Thu, May 6, 2010 at 1:34 PM, Ahmet Arslan iori...@yahoo.com wrote: recently I start to work on solr, So I am still very new to use solr. Sorry if I am logically wrong. I have two table, parent and referenced (child). for that I set multivalue field following is my schema details field name=id type=string indexed=true stored=true required=true / field name=name type=text indexed=true stored=true/ field name=dept type=text indexed=true stored=true multiValued=true/ field name=city type=text indexed=true stored=true multiValued=true/ indexed data details: doc arr name=dept strstudent1/str strstudent2/str strstudent3/str /arr arr name=city strcity1/str strcity2/str strcity3/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc now my question is : When user is searching by city2 then I want to return employee2 and their id (for multi value field). something like: doc arr name=dept strstudent2/str /arr arr name=city strcity2/str /arr str name=id1/str arr name=name strname of emp/str /arr /doc I had a similar need before. AFAIK you cannot do it with multivalued fields. The indexing order is preserved in multivalued field. May be you can post-process returned fields and capture correct position of matched city field, and use this index to display correct dept value. But this is easy if you are using string or integer type for city and dept. General solution is to index 3 different SolrDocument in your example. id and name fields will repeat themselves. All fields will be single-valued.
Re: synonym filter problem for string or phrase
On 4/29/10 8:50 PM, Marco Martinez wrote: Hi Ranveer, If you don't specify a field type in the q parameter, the search will be done searching in your default search field defined in the solrconfig.xml, its your default field a text_sync field? Regards, Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2010/4/29 Ranveerranveer.s...@gmail.com ranveer.s...@gmail.com Hi, I am trying to configure synonym filter. my requirement is: when user searching by phrase like what is solr user? then it should be replace with solr user. something like : what is solr user? = solr user My schema for particular field is: fieldType name=text_sync class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.TrimFilterFactory / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=KeywordTokenizerFactory/ /analyzer /fieldType it seems working fine while trying by analysis.jsp but not by url http://localhost:8080/solr/core0/select?q=what is solr user? or http://localhost:8080/solr/core0/select?q=what is solr user? Please guide me for achieve desire result. Hi Marco, thanks. yes my default search field is text_sync. I am getting result now but not as I expect. following is my synonym.txt what is bone cancer=bone cancer what is bone cancer?=bone cancer what is of bone cancer=bone cancer what is symptom of bone cancer=bone cancer what is symptoms of bone cancer=bone cancer in above I am getting result of all synonym but not the last one what is symptoms of bone cancer=bone cancer. I think due to stemming I am not getting expected result. However when I am checking result from the analysis.jsp, its giving expected result. I am confused.. Also I want to know best approach to configure synonym for my requirement. thanks with regards Hi, I am also facing same type of problem.. I am Newbie please help. thanks Jonty
