You need not to configure primary shards, the automatic allocation of shards is ok.
If you index, the indexing load will be distributed over primary and replica. Because you only have two nodes and one replica, Elasticsearch skips the replica by default, unless you configure the write consistency from "quorum" to "all". If you use three nodes, this will work better, because the quorum fomula is (n / 2 + 1) for n > 2 and ES writes to at least two nodes. See also https://github.com/elasticsearch/elasticsearch/issues/6482 Quoting kimchy: "I will add, that the number 2 is just a tricky number when it comes to distributed systems. I would argue that either quorum in this case set to 2, or it being set to 1 can be debatable..., since 2 in this case also means all. The reason we went with the default mentioned is because many times people run Elasticsearch using 1 node, or 2 (as a search platform for their database), on top of the just getting started aspect, and they are ok with potentially needing to reindex the data with the downsides that come with 1 or 2 nodes." Jörg On Thu, Aug 7, 2014 at 11:33 AM, Pavel P <[email protected]> wrote: > @Jörg > > If I have the next situation: > [image: Inline image 1] > > Does this mean the all the primary shards are allocated on the one node, > then this one node is indexing all the queries? > > If it's true - how could I configure that the primary shards would be > allocated through the cluster equally? > > If I should configure those values: > > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html > > Then which configuration values should I set? > > Assuming that currently I have the default values set - why the primary > shards are not distributed among the cluster? > > Regards, > > > On Thu, Aug 7, 2014 at 12:56 AM, [email protected] < > [email protected]> wrote: > >> 1. Yes, it is spread automatically >> >> 2. No >> >> The bulk queue up is where the shards are. So check your shard >> distribution. They should be equal on each node for an index. Otherwise >> your system load is unbalanced. >> >> Jörg >> >> >> On Wed, Aug 6, 2014 at 10:36 PM, Pavel P <[email protected]> wrote: >> >>> Still interested to know your view on the issue. >>> >>> On Wednesday, August 6, 2014 5:12:41 PM UTC+3, Pavel P wrote: >>>> >>>> Hi, >>>> >>>> Could someone clarify me the next: >>>> >>>> When I have the ES cluster, consisting from 2 machines, how should I >>>> send the bulk index requests to them. >>>> >>>> 1. Do I understand right that I can send everything to any node I have, >>>> then it would be spreaded for indexing among the cluster automatically? >>>> 2. Do I need to cover the cluster with the load balancer so each node >>>> would receive some portion of the indexing pressure? >>>> >>>> How it supposed to work by design? >>>> >>>> Currently I use the load balancer over my two instances, and as I see >>>> with the Bigdesk - the bulk queue is growing on the master node, while the >>>> slave node feels itself quite relaxed. >>>> >>>> Master node: >>>> >>>> >>>> <https://lh6.googleusercontent.com/-muM2VK-lKwM/U-I2iLbXh3I/AAAAAAAAAH0/RkYg6DVf7fI/s1600/master_node_queue.png> >>>> >>>> >>>> Slave node: >>>> >>>> >>>> <https://lh5.googleusercontent.com/-Ed4nY_7nrw4/U-I2Yzdb76I/AAAAAAAAAHs/ozL7MnZR1lA/s1600/Screen+Shot+2014-08-06+at+5.05.55+PM.png> >>>> >>>> >>>> Is that ok, that my ES cluster from 2 machines, which are c3.large (4 >>>> CPU, 8Gb memory) is only able to index 13k small documents per 10 seconds >>>> (I use it as output for the logstash)? >>>> Which performance should I expect? >>>> >>>> Regards, >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/2fa3fe97-ddfd-403c-98f3-22dc0bd1c70b%40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/2fa3fe97-ddfd-403c-98f3-22dc0bd1c70b%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/7XHQjAoKPfw/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHOe-0ppbhZ_%2ByROzww7YprMBh8RjrM8WYRiYhExopBAQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHOe-0ppbhZ_%2ByROzww7YprMBh8RjrM8WYRiYhExopBAQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > *Pavel Polyakov* > > Software Engineer - PHP team > > E-mail: [email protected] > Skype: pavel.polyakov.x1 > > <https://www.facebook.com/kreditech> > Kreditech Holding SSL GmbH > Am Sandtorkai 50, 20457 Hamburg, Germany > Office phone: +49 (0)40 - 605905-60 > Authorized representatives: Sebastian Diemer, Alexander Graubner-Müller > Company registration: Hamburg HRB122027 > > www.kreditech.com > facebook.com/kreditech <https://www.facebook.com/kreditech> > > <https://www.facebook.com/kreditech> > > This e-mail contains confidential and/or legally protected information. If > you are not the intended recipient or if you have received this e-mail by > error please notify the sender immediately and destroy this e-mail. Any > unauthorized review, copying, disclosure or distribution of the material in > this e-mail is strictly forbidden. The contents of this e-mail is legally > binding only if it is confirmed by letter or fax. The sending of e-mails to > us does not have any period-protecting effect. Thank you for your > cooperation. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAFVUaqM4M89ey_Ktfc9xvULxW%3DY8otT2_1iuOpomniJmrfWehg%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAFVUaqM4M89ey_Ktfc9xvULxW%3DY8otT2_1iuOpomniJmrfWehg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGn4OGO6OVDu1PQWattP3O2AO8crKKySB%2BLx-c%2Bg_zDbQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
