Re: Multi Core - Max Core Count Recommendation
it depends! yes there is overhead to each core -- how much it matters will depend entirely on your setup and typical usage pattern. sorry this is not a particularly useful answer. I think the choice of how many cores will come down to your domain logic needs more then hardware. If you are able to put things into a single index and get the performance you need, it will just be easier to deal with. ryan On Dec 10, 2008, at 3:35 PM, Ryan Peterson wrote: I'm trying to see if anyone has any recommendations on the maximum number of cores that should be used within Solr. Is there significant overhead to each core? Should it be 10 or less, or is 100 or 1,000 cores acceptable. Thanks, Ryan
Re: Multi Core - Max Core Count Recommendation
We are considering a migration to SOLR from a home grown Lucene solution. Currently we have 27,000 seperate lucene indexes that are separated based on business logic. Collectively the indexes are about 1.5 Terrabytes in size. We have some very small indexes and some that are quite large (up to 15GB). My hesitation of grouping all this data across say 4 SOLR instances is that each individual idex will still be about 400GB in size. How bit is too big for a singel Lucene index? Each SOLR instance will be on a dual/dual core xeon box with 6 SAS 15k drives in Raid 5 config and 16GB of RAM. If a 400GB instance is too much, I figured I could reduce the size of each individual index further by using multiple CORES, but again how many would depend on what size index is too big. Any suggestions would be greatly appreciated, thank you for your time. -Ryan From: Ryan McKinley [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, December 10, 2008 1:26:07 PM Subject: Re: Multi Core - Max Core Count Recommendation it depends! yes there is overhead to each core -- how much it matters will depend entirely on your setup and typical usage pattern. sorry this is not a particularly useful answer. I think the choice of how many cores will come down to your domain logic needs more then hardware. If you are able to put things into a single index and get the performance you need, it will just be easier to deal with. ryan On Dec 10, 2008, at 3:35 PM, Ryan Peterson wrote: I'm trying to see if anyone has any recommendations on the maximum number of cores that should be used within Solr. Is there significant overhead to each core? Should it be 10 or less, or is 100 or 1,000 cores acceptable. Thanks, Ryan
RE: Multi Core - Max Core Count Recommendation
1) Our limit is: is how big a file do we want to copy around? We switched to multiple indexes because of the logistics of replicating/backing up giant Lucene index files. 2) Searching takes a little memory, sorting takes a lot of memory, and faceting eats like a black hole. There is an unwritten wiki page of practical experiences. -Original Message- From: Ryan Peterson [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 10, 2008 2:20 PM To: solr-user@lucene.apache.org Subject: Re: Multi Core - Max Core Count Recommendation We are considering a migration to SOLR from a home grown Lucene solution. Currently we have 27,000 seperate lucene indexes that are separated based on business logic. Collectively the indexes are about 1.5 Terrabytes in size. We have some very small indexes and some that are quite large (up to 15GB). My hesitation of grouping all this data across say 4 SOLR instances is that each individual idex will still be about 400GB in size. How bit is too big for a singel Lucene index? Each SOLR instance will be on a dual/dual core xeon box with 6 SAS 15k drives in Raid 5 config and 16GB of RAM. If a 400GB instance is too much, I figured I could reduce the size of each individual index further by using multiple CORES, but again how many would depend on what size index is too big. Any suggestions would be greatly appreciated, thank you for your time. -Ryan From: Ryan McKinley [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, December 10, 2008 1:26:07 PM Subject: Re: Multi Core - Max Core Count Recommendation it depends! yes there is overhead to each core -- how much it matters will depend entirely on your setup and typical usage pattern. sorry this is not a particularly useful answer. I think the choice of how many cores will come down to your domain logic needs more then hardware. If you are able to put things into a single index and get the performance you need, it will just be easier to deal with. ryan On Dec 10, 2008, at 3:35 PM, Ryan Peterson wrote: I'm trying to see if anyone has any recommendations on the maximum number of cores that should be used within Solr. Is there significant overhead to each core? Should it be 10 or less, or is 100 or 1,000 cores acceptable. Thanks, Ryan
