Subject: Re: Solr server requirements for 100+ million documents
Hi Susheel,
Like Erick said, it's impossible to give precise recommendations, but making a
few assumptions and combining them with experience (+ a licked finger in the
air):
* 3 servers
* 32 GB
* 2+ CPU cores
* Linux
Assuming docs
requirements for 100+ million documents
Hi Susheel,
Like Erick said, it's impossible to give precise recommendations, but
making a few assumptions and combining them with experience (+ a licked
finger in the air):
* 3 servers
* 32 GB
* 2+ CPU cores
* Linux
Assuming docs are not bigger than
Subject: Re: Solr server requirements for 100+ million documents
Hi Susheel,
No, we wouldn't want to go with just 1 ZK. :)
Otis
--
Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch
Support * http://sematext.com/
On Tue, Feb 11, 2014 at 5:18 PM, Susheel Kumar
ZK needs a quorum to keep functional so 3 servers handles one failure. 5
handles 2 node failures. If you Solr with 1 replica per shard then stick to
3 ZK. If you use 2 replicas use 5 ZK
Whether you use the same machines as Solr or separate machines is a matter
suited to taste.
If you are the CTO, then you should make this decision. If not, inform
management that risk conditions are greater when you share function and control
on a single piece of hardware. A single failure
On 2/11/2014 3:28 PM, Susheel Kumar wrote:
Thanks, Otis for quick reply. So for ZK do you recommend separate servers and
if so how many for initial Solr cloud cluster setup.
In a minimal 3-server setup, all servers would run zookeeper and two of
them would also run Solr.With this setup, you
they are on different machines.)
-- Jack Krupansky
-Original Message-
From: Susheel Kumar
Sent: Sunday, January 26, 2014 10:54 AM
To: solr-user@lucene.apache.org
Subject: RE: Solr server requirements for 100+ million documents
Thank you Erick for your valuable inputs. Yes, we have to re-index
Thanks, Jack. That helps.
-Original Message-
From: Jack Krupansky [mailto:j...@basetechnology.com]
Sent: Tuesday, January 28, 2014 8:01 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Lucene and Solr work best if the full index can
]
Sent: Tuesday, January 28, 2014 8:01 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Lucene and Solr work best if the full index can be cached in OS memory.
Sure, Lucene/Solr does work properly once the index no longer fits, but
performance
@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
can you post the complete solrconfig.xml file and schema.xml files to review
all of your settings that would impact your indexing performance.
Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa
On Sat
@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
can you post the complete solrconfig.xml file and schema.xml files to review
all of your settings that would impact your indexing performance.
Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa
:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Dumping the raw data would probably be a good idea. I guarantee you'll be
re-indexing the data several times as you change the schema to accommodate
different requirements...
But it may
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Dumping the raw data would probably be a good idea. I guarantee you'll
be re-indexing the data several times as you change the schema to
accommodate different requirements...
But it may also
...@gmail.com] On Behalf Of
svante karlsson
Sent: Friday, January 24, 2014 5:05 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
I just indexed 100 million db docs (records) with 22 fields (4
multivalued) in 9524 sec using libcurl.
11
-
From: saka.csi...@gmail.com [mailto:saka.csi...@gmail.com] On Behalf Of
svante karlsson
Sent: Friday, January 24, 2014 5:05 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
I just indexed 100 million db docs (records) with 22 fields (4
:05 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
I just indexed 100 million db docs (records) with 22 fields (4
multivalued) in 9524 sec using libcurl.
11 million took 763 seconds so the speed drops somewhat with increasing
dbsize
-
From: saka.csi...@gmail.com [mailto:saka.csi...@gmail.com] On
Behalf Of
svante karlsson
Sent: Friday, January 24, 2014 5:05 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
I just indexed 100 million db docs (records) with 22
]
Sent: Saturday, January 25, 2014 12:08 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
can you post the complete solrconfig.xml file and schema.xml files to review
all of your settings that would impact your indexing performance.
Thanks,
Kranti
Hi,
Currently we are indexing 10 million document from database (10 db data
entities) index size is around 8 GB on windows virtual box. Indexing in one
shot taking 12+ hours while indexing parallel in separate cores merging them
together taking 4+ hours.
We are looking to scale to 100+
Can't be done with the information you provided, and can only
be guessed at even with more comprehensive information.
Here's why:
http://searchhub.org/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
Also, at a guess, your indexing speed is so slow due to data
. Do you recommend to use SolrJ or should we
try to automate using Curl?
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Friday, January 24, 2014 2:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Can't
to automate using Curl?
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Friday, January 24, 2014 2:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Can't be done with the information you provided
Hi Susheel,
Like Erick said, it's impossible to give precise recommendations, but
making a few assumptions and combining them with experience (+ a licked
finger in the air):
* 3 servers
* 32 GB
* 2+ CPU cores
* Linux
Assuming docs are not bigger than a few KB, that they are not being
reindexed
SolrJ or should we try to automate using Curl?
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Friday, January 24, 2014 2:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
Can't be done
recommend to use SolrJ or should we try to automate using Curl?
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Friday, January 24, 2014 2:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr server requirements for 100+ million documents
25 matches
Mail list logo
