Re: concept and choice: custom sharding or auto sharding?

: Erick Erickson To: solr-user Date: 2015-09-04, 01:47:23 Subject: Re: Re: Re: Re: Re: concept and choice: custom sharding or auto sharding? Ah, that may make my suggestions unworkable re: just reindexing. Still, how much time are we talking about here? I've very often found that indexing

Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

MMSeg algorithm or 1-ngram+query-preprocessor). - Original Message - From: Erick Erickson To: solr-user Date: 2015-09-04, 00:07:43 Subject: Re: Re: Re: concept and choice: custom sharding or auto sharding? bq: If you switch to SolrCloud, will you still keep numShards parameter to 1

Re: Re: Re: concept and choice: custom sharding or auto sharding?

al Message - From: Toke Eskildsen To: solr-user Date: 2015-09-03, 18:33:39 Subject: Re: Re: concept and choice: custom sharding or auto sharding? On Thu, 2015-09-03 at 18:24 +0800, Scott Chu wrote: > Do you use master-slave or SolrCloud for that single shard? Due to legacy reasons we are just

Re: Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

: Re: Re: Re: concept and choice: custom sharding or auto sharding? scott chu <scott@udngroup.com> wrote: ? > I keep forgeting to mention one thing along the discussion session. > Our data is Chinese news articles and we use CJK tokenizer > (i.e. 2-gram) currently. The time sp

Re: Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

Ah, that may make my suggestions unworkable re: just reindexing. Still, how much time are we talking about here? I've very often found that indexing performance isn't gated by the Solr processing, but by whatever is feeding Solr. A quick test is to fire up your indexing and see if the CPU

Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

scott chu wrote:   > I keep forgeting to mention one thing along the discussion session. > Our data is Chinese news articles and we use CJK tokenizer > (i.e. 2-gram) currently. The time spent to indexing is quite slow, > compared to indexing english articles. That's why I

Re: Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

scott chu wrote: > No, both. But first I have to face the indexing performance problem. > Where can I see information about concurrent/parallel indexing on Solr? Depends on how you index. If you use a Java program,

Re: Re: Re: concept and choice: custom sharding or auto sharding?

bly because I want to take advange of the Zookeeper > functionality for HA/DR. > > - Original Message - > From: Toke Eskildsen > To: solr-user > Date: 2015-09-03, 18:33:39 > Subject: Re: Re: concept and choice: custom sharding or auto sharding? > > On Thu, 2015-09-03

Re: concept and choice: custom sharding or auto sharding?

On 02/09/2015 15:47, scott chu wrote: > I post a question on Stackoverflow > http://stackoverflow.com/questions/32343813/custom-sharding-or-auto-sharding-on-solrcloud: > However, since this is a mail-list, I repost the question below to request > for suggestion and more subtle concept of

Re: Re: concept and choice: custom sharding or auto sharding?

On Wed, 2015-09-02 at 08:30 -0700, Erick Erickson wrote: > Because I routinely see 50M docs on a single node and I've seen over 300M docs > on a single node with sub-second responses. For what it's worth, we also do article-based search of newspaper based material (old OCR'ed papers). We use a

Re: Re: concept and choice: custom sharding or auto sharding?

- Original Message - From: Toke Eskildsen To: solr-user Date: 2015-09-03, 17:46:22 Subject: Re: Re: concept and choice: custom sharding or auto sharding? On Wed, 2015-09-02 at 08:30 -0700, Erick Erickson wrote: > Because I routinely see 50M docs on a single node and I've seen over 300M d

Re: Re: concept and choice: custom sharding or auto sharding?

On Thu, 2015-09-03 at 18:24 +0800, Scott Chu wrote: > Do you use master-slave or SolrCloud for that single shard? Due to legacy reasons we are just using 2 fully independent Solrs, each indexing independently, with an Apache load balancer in front for the searches. It does give us the occasional

Re: Re: Re: Re: concept and choice: custom sharding or auto sharding?

solr-user，妳好 Sorry ,wrong again. Auto sharding is not implicit router. - Original Message - From: scott chu To: solr-user Date: 2015-09-02, 23:50:20 Subject: Re: Re: Re: concept and choice: custom sharding or auto sharding? solr-user，妳好 Thanks! I'll go back to check my old

Re: Re: concept and choice: custom sharding or auto sharding?

- Original Message - From: Erick Erickson To: solr-user Date: 2015-09-02, 23:00:29 Subject: Re: concept and choice: custom sharding or auto sharding? Frankly, at 10M documents there's rarely a need to shard at all. Why do you think you need to? This seems like adding complexity for no g

Re: Re: concept and choice: custom sharding or auto sharding?

ormance is kinda sad. Why do you say: "at 10M documents there's rarely a > need to shard at all?" Do I provide too few slaves? What amount of documents > is suitable for a need for shard in SolrCloud? > > - Original Message - > > From: Erick Erickson > To: sol

Re: Re: Re: concept and choice: custom sharding or auto sharding?

afte all, I may have to use auto sharding (i.e. implicit router). - Original Message - From: Erick Erickson To: solr-user Date: 2015-09-02, 23:30:53 Subject: Re: Re: concept and choice: custom sharding or auto sharding? bq: Why do you say: "at 10M documents there's rarely a need to

Re: concept and choice: custom sharding or auto sharding?

On 9/2/2015 9:19 AM, scott chu wrote: > Mail > Do you mean I only have to put 10M documents in one index and copy > it to many slaves in a classic Solr master-slave architecture to > provide querying serivce on internet, and it won't have obvious > downgrade of query performance? But I did have

concept and choice: custom sharding or auto sharding?

I post a question on Stackoverflow http://stackoverflow.com/questions/32343813/custom-sharding-or-auto-sharding-on-solrcloud: However, since this is a mail-list, I repost the question below to request for suggestion and more subtle concept of SolrCloud's behavior on document routing. I want to

Re: concept and choice: custom sharding or auto sharding?

Frankly, at 10M documents there's rarely a need to shard at all. Why do you think you need to? This seems like adding complexity for no good reason. Sharding should only really be used when you have too many documents to fit on a single shard as it adds some overhead, restricts some possibilities