Re: How to Split Index file.
Thanks Koji, http://wiki.apache.org/solr/MergingSolrIndexes -this will be useful for merge 2 different indexes, am looking for a tool like to Split a Index directory by 2. Kalidoss.m, Koji Sekiguchi wrote: kalidoss wrote: Hi, I would like to split the existing index by 2 index, ( inverse of merge index function). My index directory size around 20G and 10 Million documents. -Kalidoss.m, I think IndexSplitter and/or MultiPassIndexSplitter are what you are looking for: http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/IndexSplitter.html http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/MultiPassIndexSplitter.html Koji Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
How to Split Index file.
Hi, I would like to split the existing index by 2 index, ( inverse of merge index function). My index directory size around 20G and 10 Million documents. -Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Query on Cache size.
Hi, We have enabled the query result cache, its 512 entries, we have calculated the size used for cache : page size about 1000bytes, (1000*512)/1024/1024 = .48MB If we increase the cache count to 10 then the memory used for cache about. (1000*10)/1024/1024 = 96MB Is my calculation is correct? my system has 2.5GB of RAM for Solr and 1GB for OS. If am wrong, let me know how to calculate the memory? Thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Hi. What Configuration we require?
Hi To run a Solr-1.3.0 with Data/index directory size of 11GB, 80 lakhs documents and 11 lakhs read request and 30 thousand writes. Every month 200mb of index directory size getting increase. Please suggest me. What type of configuration(CPU, Ram, hard disk) server require to make the solr as Stable. Thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: Hi. What Configuration we require?
Hi. To run a Solr-1.3.0 with Data/index directory size of 11GB, 80 lakhs documents and 11 lakhs read request and 30 thousand writes. Every month 200mb of index directory size getting increase. Please suggest me. What type of configuration(CPU, Ram, hard disk) server require to make the solr as Stable. Thanks, Kalidoss.m, kalidoss wrote: Hi To run a Solr-1.3.0 with Data/index directory size of 11GB, 80 lakhs documents and 11 lakhs read request and 30 thousand writes. Every month 200mb of index directory size getting increase. Please suggest me. What type of configuration(CPU, Ram, hard disk) server require to make the solr as Stable. Thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: Hi. What Configuration we require?
Thanks Shalin Shekhar. 11 lakh read requests and 30 thousand write requests within how much time? Per day average of 11 lakh read requests and 30 thousand write requests. The system configuration is 4GB RAM and 4 core x 2 CPUs. are you suggesting us to increase the configuration? -Kalidoss.m, Shalin Shekhar Mangar wrote: On Wed, Dec 9, 2009 at 5:36 PM, kalidoss kalidoss.muthuramalin...@sifycorp.com wrote: Hi To run a Solr-1.3.0 with Data/index directory size of 11GB, 80 lakhs documents and 11 lakhs read request and 30 thousand writes. Every month 200mb of index directory size getting increase. 11 lakh read requests and 30 thousand write requests within how much time? Please suggest me. What type of configuration(CPU, Ram, hard disk) server require to make the solr as Stable. In general, having enough RAM for Solr caches as well as the OS for the file caches is good. Fast IO helps too. You'd most likely go for a master/slave deployment in production. We use boxes with quad cores, 16 gig RAM, SCSI disks. YMMV. Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: latency in solr response is observed after index is updated
r u using solr-config for committing the document? bharath venkatesh wrote: Hi, We are observing latency (some times huge latency upto 10-20 secs) in solr response after index is updated . whats the reason of this latency and how can it be minimized ? Note: our index size is pretty large. any help would be appreciated as we largely affected by it Thanks in advance. Bharath This message is intended only for the use of the addressee and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify us immediately by return e-mail and delete this e-mail and all attachments from your system. Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: Solr - Load Increasing.
Thank u all. I have increased the heap size memory from 1gb to 1.5gb. Now its java -Xms512M -Xmx1536M -jar start.jar, My cpu load is normal and solr is not restating frequently, My autocommit maxdoc increased to 200. For last 24 hours no issue on load/restarts. Thanks Guys. Kalidoss.m, Otis Gospodnetic wrote: Your autocommit settings are still pretty aggressive causing very frequent commits, and that is using your CPU. Yes, splitting the servers into a master and slaves tends to be the performant/scalable way to go. There is no real downside to replication, really, just a bit of network traffic. Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: kalidoss kalidoss.muthuramalin...@sifycorp.com To: solr-user@lucene.apache.org Sent: Wed, November 18, 2009 2:25:05 AM Subject: Re: Solr - Load Increasing. There seems to be some improvement. The writes speeds are faster. Server restarts are lower. We changed the configuration to: 50 1 Before the Change: - Server Restarts: 10 times in 12 hours - CPU load: Average:50 and Peak:90 After the Change: - Server Restarts: 4 times in 12 hours. - CPU load: Average:30 and Peak:~70 Our every day writes are around 60k and reads are around 1 million. We are now changing the MaxDocs to 300 and MaxTime will be 1 ms and hoping to some more improvements. The system configuration is 4GB RAM and 4 core x 2 CPUs. We start the solr (1.3) like this: java -Xms512M -Xmx1024M -jar start.jar Is there any other way we can reduce the high CPU load in the system? Do you guys think that upgrading to 1.4 and having the replication in place with reads and writes split into separate solrs will help? How efficient will the replication be with above mentioned scenarios? Is there any place we can look at for info on the disadvantages of replication... Please help. Kalidoss.m, Tom Alt wrote: Nice to learn a new word for the day! But to answer your question, or at least part of it, I don't really think you want a configuration like 1 10 Committing every doc, and every 10 milliseconds? That's just asking for problems. How about starting with 1000 docs, and five minutes for maxTime (5*60*1000) or about 3 laks of milliseconds. That should help performance a lot. Try that, and see how it works. Tom On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant wrote: I think it would be useful for members of this list to realize that not everyone uses the same metrology and terms. It is very easy for Americans to use the imperial system and presume everyone does the same; Europeans to use the metric system etc. Hopefully members on this list would be persuaded to use or at least clarify their terminology. While the apocryphal saying goes the great thing about standards is they are so many choose from, we should all make an effort to communicate across cultures and nations. On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo wrote: On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wrote: Probably lakh: 100,000. So, 900k qpd and 3M docs. http://en.wikipedia.org/wiki/Lakh wunder On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote: Hi, Your autoCommit settings are very aggressive. I'm guessing that's what's causing the CPU load. btw. what is laks? Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: kalidoss To: solr-user@lucene.apache.org Sent: Mon, November 16, 2009 9:11:21 AM Subject: Solr - Load Increasing. Hi All. My server solr box cpu utilization increasing b/w 60 to 90% and some time solr is getting down and we are restarting it manually. No of documents in solr 30 laks. No of add/update requrest solr 30 thousand / day. Avg of every 30 minutes around 500 writes. No of search request 9laks / day. Size of the data directory: 4gb. My system ram is 8gb. System available space 12gb. processor Family: Pentium Pro Our solr data size can be increase in number like 90 laks. and writes per day will be around 1laks. - Hope its possible by solr. For write commit i have configured like 1 10 Is all above can be possible? 90laks datas and 1laks per day writes and 30laks per day read?? - if yes what type of system configuration would require. Please suggest us. thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary
Re: Upgrade to solr 1.4
Even i want to upgrade from v1.3 to 1.4 I did 1.3 index directory replace with 1.4 and associated schema changes in that. Its throwing lot of exception like datatype mismatch with Integer, String, Date, etc. Even the results are coming with some error example: str name=AliasERROR:SCHEMA-INDEX-MISMATCH,stringValue=14903346/str Is there any tool/notes to upgrade from 1.3 to 1.4? on Data and schema data types etc? Please suggest us. -Kalidoss.m, Walter Underwood wrote: We are using the script replication. I have no interest in spending time configuring and QA'ing a different method when the scripts work fine. We are running the nightly from 2009-05-11. wunder On 6/26/09 8:51 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Fri, Jun 26, 2009 at 9:11 PM, Walter Underwood wunderw...@netflix.comwrote: Netflix is running a nightly build from May in production. We did our normal QA on it, then ran it on one of our five servers for two weeks. No problems. It is handling about 10% more traffic with 10% less CPU. Wow, that is good news! Are you also using the java based replication? We deployed 1.4 to all our servers yesterday. Can you tell us which revision you used? Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: Upgrade to solr 1.4
In version 1.3 EventDate field type is date, In 1.4 also its date But we are getting the following error. str name=EventDateERROR:SCHEMA-INDEX-MISMATCH,stringValue=2008-05-16T07:19:28/str -kalidoss.m, kalidoss wrote: Even i want to upgrade from v1.3 to 1.4 I did 1.3 index directory replace with 1.4 and associated schema changes in that. Its throwing lot of exception like datatype mismatch with Integer, String, Date, etc. Even the results are coming with some error example: str name=AliasERROR:SCHEMA-INDEX-MISMATCH,stringValue=14903346/str Is there any tool/notes to upgrade from 1.3 to 1.4? on Data and schema data types etc? Please suggest us. -Kalidoss.m, Walter Underwood wrote: We are using the script replication. I have no interest in spending time configuring and QA'ing a different method when the scripts work fine. We are running the nightly from 2009-05-11. wunder On 6/26/09 8:51 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Fri, Jun 26, 2009 at 9:11 PM, Walter Underwood wunderw...@netflix.comwrote: Netflix is running a nightly build from May in production. We did our normal QA on it, then ran it on one of our five servers for two weeks. No problems. It is handling about 10% more traffic with 10% less CPU. Wow, that is good news! Are you also using the java based replication? We deployed 1.4 to all our servers yesterday. Can you tell us which revision you used? Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Re: Solr - Load Increasing.
There seems to be some improvement. The writes speeds are faster. Server restarts are lower. We changed the configuration to: maxDocs50/maxDocs maxTime1/maxTime Before the Change: - Server Restarts: 10 times in 12 hours - CPU load: Average:50 and Peak:90 After the Change: - Server Restarts: 4 times in 12 hours. - CPU load: Average:30 and Peak:~70 Our every day writes are around 60k and reads are around 1 million. We are now changing the MaxDocs to 300 and MaxTime will be 1 ms and hoping to some more improvements. The system configuration is 4GB RAM and 4 core x 2 CPUs. We start the solr (1.3) like this: java -Xms512M -Xmx1024M -jar start.jar Is there any other way we can reduce the high CPU load in the system? Do you guys think that upgrading to 1.4 and having the replication in place with reads and writes split into separate solrs will help? How efficient will the replication be with above mentioned scenarios? Is there any place we can look at for info on the disadvantages of replication... Please help. Kalidoss.m, Tom Alt wrote: Nice to learn a new word for the day! But to answer your question, or at least part of it, I don't really think you want a configuration like autoCommit maxDocs1/maxDocs maxTime10/maxTime /autoCommit Committing every doc, and every 10 milliseconds? That's just asking for problems. How about starting with 1000 docs, and five minutes for maxTime (5*60*1000) or about 3 laks of milliseconds. That should help performance a lot. Try that, and see how it works. Tom On Mon, Nov 16, 2009 at 2:43 PM, Shashi Kant sk...@sloan.mit.edu wrote: I think it would be useful for members of this list to realize that not everyone uses the same metrology and terms. It is very easy for Americans to use the imperial system and presume everyone does the same; Europeans to use the metric system etc. Hopefully members on this list would be persuaded to use or at least clarify their terminology. While the apocryphal saying goes the great thing about standards is they are so many choose from, we should all make an effort to communicate across cultures and nations. On Mon, Nov 16, 2009 at 5:33 PM, Israel Ekpo israele...@gmail.com wrote: On Mon, Nov 16, 2009 at 5:22 PM, Walter Underwood wun...@wunderwood.org wrote: Probably lakh: 100,000. So, 900k qpd and 3M docs. http://en.wikipedia.org/wiki/Lakh wunder On Nov 16, 2009, at 2:17 PM, Otis Gospodnetic wrote: Hi, Your autoCommit settings are very aggressive. I'm guessing that's what's causing the CPU load. btw. what is laks? Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR - Original Message From: kalidoss kalidoss.muthuramalin...@sifycorp.com To: solr-user@lucene.apache.org Sent: Mon, November 16, 2009 9:11:21 AM Subject: Solr - Load Increasing. Hi All. My server solr box cpu utilization increasing b/w 60 to 90% and some time solr is getting down and we are restarting it manually. No of documents in solr 30 laks. No of add/update requrest solr 30 thousand / day. Avg of every 30 minutes around 500 writes. No of search request 9laks / day. Size of the data directory: 4gb. My system ram is 8gb. System available space 12gb. processor Family: Pentium Pro Our solr data size can be increase in number like 90 laks. and writes per day will be around 1laks. - Hope its possible by solr. For write commit i have configured like 1 10 Is all above can be possible? 90laks datas and 1laks per day writes and 30laks per day read?? - if yes what type of system configuration would require. Please suggest us. thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing
Solr - Load Increasing.
Hi All. My server solr box cpu utilization increasing b/w 60 to 90% and some time solr is getting down and we are restarting it manually. No of documents in solr 30 laks. No of add/update requrest solr 30 thousand / day. Avg of every 30 minutes around 500 writes. No of search request 9laks / day. Size of the data directory: 4gb. My system ram is 8gb. System available space 12gb. processor Family: Pentium Pro Our solr data size can be increase in number like 90 laks. and writes per day will be around 1laks. - Hope its possible by solr. For write commit i have configured like autoCommit maxDocs1/maxDocs maxTime10/maxTime /autoCommit Is all above can be possible? 90laks datas and 1laks per day writes and 30laks per day read?? - if yes what type of system configuration would require. Please suggest us. thanks, Kalidoss.m, Get your world in your inbox! Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id! Log on to http://www.sify.com ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Have query on wildchar.
Hi, In my schema, I have Tags, AdTags and am giving the search like. ?q=(Tags: anton* and AdTags: Funn*) my schema query parser is : solrQueryParser defaultOperator=AND/ So Is it required to give the 'and' in the query?? Is yes is there any difference b/w giving 'AND' 'and' (case sensitive), Please let me know. When am giving 'and' getting result and if am giving 'AND' and not any results... :( let me know. Thanks, Kalidoss.m,
query on defaultSearchField?
Hi, 1) Can i give by default defaultSearchField with multiple field values as like defaultSearchFieldtext, Tag, Category/defaultSearchField Or should i use copyField source=Tag dest=text/ copyField source=Category dest=text/ Thanks, Kalidoss.m,
Re: Unified search of relational data on Solr?
Its for searching with almost all the fields we used for seaching, stats we used for list the most viewd image(gallery). thanks, kalidoss.m, On Thu, Feb 19, 2009 at 12:50 PM, Noble Paul നോബിള് नोब्ळ् noble.p...@gmail.com wrote: do you wish to search on the image names or is it that you only wish to read the image details --Noble On Thu, Feb 19, 2009 at 12:31 PM, Kalidoss MM mm.kalid...@gmail.com wrote: Even in my case, we cant make it flattern, Bcoz we are managing total image gallery information in Solr, So image gallery contains aroung 20 images also with image descrption, thumbnail info, width, height, etc also we want to store/update the stats along with image gallery, If we flatten the xml, for every visit to the image gallery i need to update the whole lengh record again into Solr, we have around 30lacs image gallery also per day around 50K imagegallery stats supposed to update, So we are thinking of spliting of Image gallery And (Stats, comments) as separate xml.. 1) if any body used parallel Reader (lucene) let me know how this will be usefull for us, 2) If any body used multicore let me know how this will be useful for us. 3) Is MultipleIndexes will be useful or not? http://wiki.apache.org/solr/MultipleIndexes Please suggest us, Thanks, kalidoss.m, On Thu, Feb 19, 2009 at 11:24 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Hi, Just flatten it - create a single Person + Address entity (document) and index it. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch From: Senthil Kumar thanjaisent...@gmail.com To: solr-user@lucene.apache.org Sent: Thursday, February 19, 2009 1:20:23 PM Subject: Unified search of relational data on Solr? Hi, How to index relational data in Solr which can not be merged as a single file for some reasons? We have two kinds of XMLs indexed in Solr, Personal id1_persona/id fname\ age\ sex\ /Personal Address Id1_addr/id citywashington/city /Address Our aim to get a list of persons living in Washington. Can anyone suggest what is the best approach for this and to index relational data in general? Senthil Kumar P -- --Noble Paul
Re: Data Directory Sync.
Hi, I think i can use http://wiki.apache.org/solr/MergingSolrIndexes - to index two different solr index directory?? Thanks, kalidoss.m, On Thu, Jan 29, 2009 at 8:57 PM, Noble Paul നോബിള് नोब्ळ् noble.p...@gmail.com wrote: On Thu, Jan 29, 2009 at 7:27 PM, Kalidoss MM mm.kalid...@gmail.com wrote: Hi, I have a requirement like, There is a running solr and having around 10K records indexed in it. Now i have to index another set of 30K records? The 10K data already in live, And i dont have an option to insert that 30K records in live, you can index the 30K data to the live Solr . Is there any way to run the solr in local system and get the 30K records in data directory, and Update/Upgrade the local solr data directoy INTO live data directory? Is there any tools available? Or is there any other method to Sync/combine 2 different data directory and make it to 1 data directory. Thanks, Kalidoss.m, -- --Noble Paul
Re: Unified search of relational data on Solr?
Even in my case, we cant make it flattern, Bcoz we are managing total image gallery information in Solr, So image gallery contains aroung 20 images also with image descrption, thumbnail info, width, height, etc also we want to store/update the stats along with image gallery, If we flatten the xml, for every visit to the image gallery i need to update the whole lengh record again into Solr, we have around 30lacs image gallery also per day around 50K imagegallery stats supposed to update, So we are thinking of spliting of Image gallery And (Stats, comments) as separate xml.. 1) if any body used parallel Reader (lucene) let me know how this will be usefull for us, 2) If any body used multicore let me know how this will be useful for us. 3) Is MultipleIndexes will be useful or not? http://wiki.apache.org/solr/MultipleIndexes Please suggest us, Thanks, kalidoss.m, On Thu, Feb 19, 2009 at 11:24 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Hi, Just flatten it - create a single Person + Address entity (document) and index it. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch From: Senthil Kumar thanjaisent...@gmail.com To: solr-user@lucene.apache.org Sent: Thursday, February 19, 2009 1:20:23 PM Subject: Unified search of relational data on Solr? Hi, How to index relational data in Solr which can not be merged as a single file for some reasons? We have two kinds of XMLs indexed in Solr, Personal id1_persona/id fname\ age\ sex\ /Personal Address Id1_addr/id citywashington/city /Address Our aim to get a list of persons living in Washington. Can anyone suggest what is the best approach for this and to index relational data in general? Senthil Kumar P
Data Normalization in Solr.
Hi, I want to store normalized data into Solr, example am spliting personal information datas(fname, lname, mname) as one solr record, Address (personal, office) as another record in Solr. the id is different 123212_name, 123212_add, Now, some case i require both personal and address records by single xml say( fname, lname, officeaddress only) it self, (with single http request), Is it possible? Thanks, kalidoss.m,
Data Directory Sync.
Hi, I have a requirement like, There is a running solr and having around 10K records indexed in it. Now i have to index another set of 30K records? The 10K data already in live, And i dont have an option to insert that 30K records in live, Is there any way to run the solr in local system and get the 30K records in data directory, and Update/Upgrade the local solr data directoy INTO live data directory? Is there any tools available? Or is there any other method to Sync/combine 2 different data directory and make it to 1 data directory. Thanks, Kalidoss.m,
Datemath Now is UST or IST?
Hi, We use Solr1.3 and indexed some of our date fields in the format '1995-12-31T23:59:59Z' and as we know this is a UTC date. But we do want to index the date in IST which is +05:30hours so that extra conversion from UTC to IST across all our application is avoided. How to do that? And we have some confusion on how the flexible search functions such as (NOW, NOW+1DAY etc) provided by DateMathParser works? Now() is being calculated upon considering the date indexed as UTC or Localtime? Can we have the NOW() results in IST if the date indexed is in IST? Thanks, Kalidoss.m,
Does search query return specific result.?
Hi, We are trying to implement an auto-suggest feature in our application that uses Solr as the core engine for search. The XML is structured as follows: Media IdQLrKnirLDEo9DThnL2h/Id Title/Title DescriptionDescription/Description Categories CategoryCat1/Category CategoryCat2/Category /Categories Tags TagKalidoss/Tag TagKaling/Tag TagSoundoss/Tag /Tags /Media We transform the same in solr understandable format like: field nameTagPicture of the Day/field (which is multivalue=true) ... Now, we want to give an auto-suggest feature on fields like Tag Category. In other words, when we search like: http://localhost:8983/solr/select/?q=(Tag:kali*)fl=Tag,Id We expect it to return: doc str name=Id10001/str arr name=Tag strKalidoss/str strKaling/str /arr /doc But it returns strings that do not match also like the following: doc str name=Id10001/str arr name=Tag strKalidoss/str strKaling/str strsoundoss/str /arr /doc I believe the reason is because, Solr returns the document with all of the Tag field's content. Now, the question is: Is there a way to make it return only Tag that match the criteria from the same document? -Doss.M,
Re: Does search query return specific result.?
Any update on this?? Please let me know? On Thu, Jan 8, 2009 at 3:34 PM, Kalidoss MM mm.kalid...@gmail.com wrote: Hi, We are trying to implement an auto-suggest feature in our application that uses Solr as the core engine for search. The XML is structured as follows: Media IdQLrKnirLDEo9DThnL2h/Id Title/Title DescriptionDescription/Description Categories CategoryCat1/Category CategoryCat2/Category /Categories Tags TagKalidoss/Tag TagKaling/Tag TagSoundoss/Tag /Tags /Media We transform the same in solr understandable format like: field nameTagPicture of the Day/field (which is multivalue=true) ... Now, we want to give an auto-suggest feature on fields like Tag Category. In other words, when we search like: http://localhost:8983/solr/select/?q=(Tag:kali*)fl=Tag,Idhttp://localhost:8983/solr/select/?q=%28Tag:kali*%29fl=Tag,Id We expect it to return: doc str name=Id10001/str arr name=Tag strKalidoss/str strKaling/str /arr /doc But it returns strings that do not match also like the following: doc str name=Id10001/str arr name=Tag strKalidoss/str strKaling/str strsoundoss/str /arr /doc I believe the reason is because, Solr returns the document with all of the Tag field's content. Now, the question is: Is there a way to make it return only Tag that match the criteria from the same document? -Doss.M,
Subscribe Me
Thanks, kalidoss.m, ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to Sify Limited and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If this is a forwarded message, the content of this E-MAIL may not have been sent with the authority of the Company. If you are not the intended recipient, an agent of the intended recipient or a person responsible for delivering the information to the named recipient, you are notified that any use, distribution, transmission, printing, copying or dissemination of this information in any way or in any manner is strictly prohibited. If you have received this communication in error, please delete this mail notify us immediately at ad...@sifycorp.com
Issue with Java Client code
Hi, We have created a Java EmbeddedSolrServer Client Code, I can able to add, delete, update the Solr content - At the same time i cant able to search the updated conente from the Running Solr client(jetty) web interface. My requirement is, All search need to happen from/by running web Solr(jetty, 8983) and all write should happened from Java client code. Both(jeety and javaclient) are using 'Core0' as core name, and both data directory, schema, solrconfig are same. - is there any fix available?? Case1: 1) solr started in 8983 port as Core0, 2) Running a java client(Core0) to add one record say hitest, with commit 3) when i search for hitest, am not getting any result, 4) after i restart the solr(8983) and search for 'hitest' am getting the result. - - both Solr, java client is using the same dada directory, schema.xml, solrconfig.xml fyi: even this java client is working when solr is not started is it possible to handle the read/search by webinterface, and all write from java-client(with out http) Thanks in advance, Kalidoss.m,
