Re: How to split index more than 2GB in size

2018-06-21 Thread Michael Kuhlmann
Hi Sushant, while this is true in general, it won't hold here. If you split your index, searching on each splitted shard might be a bit faster, but you'll increase search time much more because Solr needs to send your search queries to all shards and then combine the results. So instead of having

Re: How to split index more than 2GB in size

2018-06-20 Thread Sushant Vengurlekar
Thank you for the detailed response Eric. Very much appreciated. The reason I am looking into splitting the index into two is because it’s much faster to search across a smaller index than a larger one. On Wed, Jun 20, 2018 at 10:46 AM Erick Erickson wrote: > You still haven't answered _why_

Re: How to split index more than 2GB in size

2018-06-20 Thread Erick Erickson
You still haven't answered _why_ you think splitting even a 20G index is desirable. We regularly see 200G+ indexes per replica in the field, so what's the point? Have you measured different setups to see if it's a good idea? A 200G index needs some beefy hardware admittedly If you have

Re: How to split index more than 2GB in size

2018-06-20 Thread Sushant Vengurlekar
The index size is small because this is my local development copy. The production index is more than 20GB. So I am working on getting the index split and replicated on different nodes. Our current instance on prod is single instance solr 6 which we are working on moving towards solrcloud 7 On

Re: How to split index more than 2GB in size

2018-06-20 Thread Erick Erickson
Use the indexupgrader tool or optimize your index before using splitshard. Since this is a small index (< 5G), optimizing will not create an overly-large segment, so that pitfall is avoided. You haven't yet explained why you think splitting the index would be beneficial. Splitting an index this

Re: How to split index more than 2GB in size

2018-06-20 Thread Sushant Vengurlekar
How can I resolve this error? On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch wrote: > This seems more related to an old index upgraded to latest Solr rather than > the split itself. > > Regards, > Alex > > On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, < >

Re: How to split index more than 2GB in size

2018-06-20 Thread Sushant Vengurlekar
My old solr instance was 6.6.3 and the current solrcloud I am building is 7.3.1. Are there any issues there? On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch wrote: > This seems more related to an old index upgraded to latest Solr rather than > the split itself. > > Regards, > Alex >

Re: How to split index more than 2GB in size

2018-06-20 Thread Alexandre Rafalovitch
This seems more related to an old index upgraded to latest Solr rather than the split itself. Regards, Alex On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, < svengurle...@curvolabs.com> wrote: > Thanks for the reply Alessandro! Appreciate it. > > Below is the full request and the error

Re: How to split index more than 2GB in size

2018-06-20 Thread Sushant Vengurlekar
Thanks for the reply Alessandro! Appreciate it. Below is the full request and the error received curl ' http://localhost:8081/solr/admin/collections?action=SPLITSHARD=dev-transactions=shard1 ' { "responseHeader":{ "status":500, "QTime":7920}, "success":{

Re: How to split index more than 2GB in size

2018-06-20 Thread Alessandro Benedetti
Hi, in the first place, why do you want to split 2 Gb indexes ? Nowadays is a fairly small index. Secondly what you reported is incomplete. I would expect a Caused By section in the stacktrace. This are generic recommendations, always spend time in analysing the problem you had scrupulously. -

How to split index more than 2GB in size

2018-06-19 Thread Sushant Vengurlekar
How do I split indexes which are more than 2GB in size. I get this error when I try to use SPLITSHARD on a collection of size more than 2GB 2018-06-20 02:25:49.810 ERROR (qtp1025799482-19) [ ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: SPLITSHARD failed to invoke SPLIT