What you have below is not really what we call "Distributed Search", but more of "Query Load Balancing". Yes, the diagram below will work IF a single Solr box (A or B) can really handle a full 50M doc index. Of course "handle" can be fuzzy. That is, you could have a large index on a Solr box and it will "handle" it - nothing will crash, nothing will die, it's just that it may not be able to handle it "well enough" - that is, the queries may take longer than you'd like.
NFS mounting an index directory is a separate story and very often a bad idea, again because of performance. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Antonio Eggberg <antonio_eggb...@yahoo.se> > To: solr-user@lucene.apache.org > Sent: Monday, December 29, 2008 4:19:23 PM > Subject: Re: how large can the index be? > > Thanks you very much for your answer. > > I was afraid of that the each document has about 20 fields.. As you pointed > out > it will slow down. Anyway I am thinking is it not possible to do the > following: > > Load Balancer > | > Solr A, Solr B, ... > | > one index > > So I send 50% query to Solr A, 50% to Solr B and so forth.. is this not good? > Also to add The index will be like a mounted drive to the solr boxes... On > the > above do I really need to worry about Solr Master, Solr Slave? It probably > solve > my load but I think query speed will be slow... > > Just curious anyone using distributed search in production? > > Cheers > > > > --- Den mån 2008-12-29 skrev Otis Gospodnetic : > > > Från: Otis Gospodnetic > > Ämne: Re: how large can the index be? > > Till: solr-user@lucene.apache.org > > Datum: måndag 29 december 2008 21.53 > > Hi Antonio, > > > > Besides thinking in terms of documents, you also need to > > think in terms of index size on the file system vs. the > > amount of RAM your search application/server can use. 50M > > documents may be doable on a single server if those > > documents are not too large and you have sufficient RAM. It > > gets even better if your index doesn't change very often > > and if you can get decent hit ratios on the various Solr > > caches. > > > > If you are indexing largish documents, or even something as > > small as an average web page, 50M docs may be too much on a > > "commodity box" (say dual core 8 GB RAM box) > > > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > > > > > ----- Original Message ---- > > > From: Antonio Eggberg > > > To: solr-user@lucene.apache.org > > > Sent: Monday, December 29, 2008 3:41:48 PM > > > Subject: how large can the index be? > > > > > > Hi, > > > > > > We are running successfully a solr index of 3 million > > docs. I have just been > > > informed that our index size will increase to 50 > > million. I been going through > > > the doc > > > > > > http://wiki.apache.org/solr/DistributedSearch > > > > > > Seems like we will loose out on the date facet and > > some more other stuff that we > > > use. which is important to us. So far we been using 1 > > index and 1 machine. > > > > > > Can I still stick with my 1 index but have many query > > servers? We don't update > > > our index so often this are rather static data. Over > > the past year we have > > > updated the index data a total of 3 times and about > > 300 records :) > > > > > > Can someone provide some idea how/what should I do to > > deal with new datasets?. > > > > > > Thanks for your help. > > > > > > > > > > > __________________________________________________________ > > > Går det långsamt? Skaffa dig en snabbare > > bredbandsuppkoppling. > > > Sök och jämför priser hos Kelkoo. > > > > > http://www.kelkoo.se/c-100015813-bredband.html?partnerId=96914325 > > > __________________________________________________________ > Låna pengar utan säkerhet. Jämför vilkor online hos Kelkoo. > http://www.kelkoo.se/c-100390123-lan-utan-sakerhet.html?partnerId=96915014
