Fot 16 mil docs it may not be necessary. Add the shards when you see that perf is degrading. --Noble
On Wed, Jun 4, 2008 at 4:17 PM, Ritesh Ambastha <[EMAIL PROTECTED]> wrote: > > The number of docs I have indexed till now is : 1,633,570 > I am bit afraid as the number of indexed docs will grow atleast 5-10 times > in very near future. > > Regards, > Ritesh Ambastha > > > > Shalin Shekhar Mangar wrote: >> >> A lot of this also depends on the number of documents. But we have >> successfully used Solr with upto 10-12 million documents. >> >> On Wed, Jun 4, 2008 at 4:10 PM, Ritesh Ambastha <[EMAIL PROTECTED]> >> wrote: >> >>> >>> Thanks Noble, >>> >>> That means, I can go ahead with single Index for long. >>> :) >>> >>> Regards, >>> Ritesh Ambastha >>> >>> Noble Paul നോബിള് नोब्ळ् wrote: >>> > >>> > For the datasize you are proposing , single index should be fine .Just >>> > give the m/c enough RAM >>> > >>> > Distributed search involves multiple requests made between shards >>> > which may be an unncessary overhead. >>> > --Noble >>> > >>> > On Wed, Jun 4, 2008 at 4:02 PM, Ritesh Ambastha >>> <[EMAIL PROTECTED]> >>> > wrote: >>> >> >>> >> Thanks Noble, >>> >> >>> >> I maintain two separate indexes on my disk for two different search >>> >> services. >>> >> The index size of two are: 91MB and 615MB. I am pretty sure that these >>> >> index >>> >> size will grow in future, and may reach 10GB. >>> >> >>> >> My doubts : >>> >> >>> >> 1. When should I start partitioning my index? >>> >> 2. Is there any performance issue with partitioning? For eg: A query >>> on >>> >> 1GB >>> >> and 500MB indexed data will take same time to give the result? Or >>> lesser >>> >> the >>> >> index size, lesser the response time? >>> >> >>> >> >>> >> Regards, >>> >> Ritesh Ambastha >>> >> >>> >> Noble Paul നോബിള് नोब्ळ् wrote: >>> >>> >>> >>> You could have been more specific on the dataset size. >>> >>> >>> >>> If your data volumes are growing you can partition your index into >>> >>> multiple shards. >>> >>> http://wiki.apache.org/solr/DistributedSearch >>> >>> --Noble >>> >>> >>> >>> On Sat, May 31, 2008 at 9:02 PM, Ritesh Ambastha >>> >>> <[EMAIL PROTECTED]> >>> >>> wrote: >>> >>>> >>> >>>> Dear Readers, >>> >>>> >>> >>>> I am a newbie in solr world. I have successfully deployed solr on my >>> >>>> machine, and I am able to index a large DB table. I am pretty sure >>> that >>> >>>> internal index structure of solr is much capable to handle large >>> data >>> >>>> sets. >>> >>>> >>> >>>> But, say my data size keeps growing at jet speed, then what should >>> be >>> >>>> the >>> >>>> index structure? Do I need to follow some specific index structuring >>> >>>> patterns/algos for handling such massive data? >>> >>>> >>> >>>> I am sorry as I may be sounding novice in this area. I would >>> appreciate >>> >>>> your >>> >>>> thoughts/suggestions. >>> >>>> >>> >>>> Regards, >>> >>>> Ritesh Ambastha >>> >>>> -- >>> >>>> View this message in context: >>> >>>> http://www.nabble.com/Index-structuring-tp17576449p17576449.html >>> >>>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>>> >>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> --Noble Paul >>> >>> >>> >>> >>> >> >>> >> -- >>> >> View this message in context: >>> >> http://www.nabble.com/Index-structuring-tp17576449p17643690.html >>> >> Sent from the Solr - User mailing list archive at Nabble.com. >>> >> >>> >> >>> > >>> > >>> > >>> > -- >>> > --Noble Paul >>> > >>> > >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Index-structuring-tp17576449p17643798.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- > View this message in context: > http://www.nabble.com/Index-structuring-tp17576449p17643909.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
