Hi Danny! I don't think that it's doable out of a box for now. I believe the issue with the wannabe random distribution will be fixed soon, or an additional deployment strategy will be introduced.
But I may suggest you trying to simulate the desired behavior by hands: 1. Assign each node in the cluster group a unique id or name 2. Deploy your services manually in the way that every node gets 900/20 services 3. Listen for node leave events for this cluster group to process such cases properly, for example redeploying services on other nodes or restarting the left node and deploying services on it Kind regards, Alex. On Fri, May 19, 2017 at 9:42 AM, dany74q [via Apache Ignite Users] < ml+s70518n13020...@n6.nabble.com> wrote: > Hey Alex, > > I can't really paste the source, but I can help with pseudo-code: > > - We have 3 categories of nodes, let's call them Workers, Scheduler, and > Seeds (There are NodeAttributes that set each node in a particular group) > - Most nodes are Workers and roughly 20 nodes are Scheduler - and Seeds > has only 3 nodes and is a bit irrelevant in this case > - We have roughly 1,000 services deployed across this cluster - most of > them are cluster singletons, few are node singletons > - Within those 1,000 services, most go to the Schedulers group, via a node > attribute filter (.forAttribute) > - We would like a solutions that could round-robin the service deployment > for us (So that each Scheduler node would get 900 / 20 services ~), if > possible, because when services congregate in a particular node, it gets > overloaded and crashes > - Using affinity key is problematic because the cache could rebalance and > redeploy services with no real reason > > Would love your assistance ! > - Danny > > On Tue, May 16, 2017 at 4:01 PM afedotov [via Apache Ignite Users] <[hidden > email] <http:///user/SendEmail.jtp?type=node&node=13020&i=0>> wrote: > >> Danny, just one more question. >> Are all the services deployed as cluster singletons? >> >> Kind regards, >> Alex. >> >> On Tue, May 16, 2017 at 1:10 PM, afedotov <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=12888&i=0>> wrote: >> Hi Danny, >> >> Could you please share your node filter/cluster group pedicate? >> >> Kind regards, >> Alex. >> >> On Mon, May 15, 2017 at 10:49 PM, dany74q [via Apache Ignite Users] <[hidden >> email] <http:///user/SendEmail.jtp?type=node&node=12883&i=0>> wrote: >> Hey Alex. >> Thanks for the prompt response ! >> We actually do employ a node filter - lets say we have N nodes, we deploy >> our services to a group B, being roughly N/10 of size (20~ nodes). >> >> Our services are quite extensive, and so deploying them unevenly across B >> causes GCs, stalls and other unfortunate results. >> >> We have implemented a round robin approach, where we create a cluster >> group consisting of one of the nodes in B (node at position counter % >> B_size). >> But it has many shortcomings - >> their deployment predicate is always fixed on that specific node, and it >> doesn't scale when adding additional nodes with the same node attribute. >> >> We're looking for a different solution. >> Question - is it possible to have an affinity key deployment strategy, >> but not having the services "rebalance" when a node leaves / joins ? >> Any other solutions ? >> On Mon, May 15, 2017 at 10:12 PM afedotov [via Apache Ignite Users] <[hidden >> email] <http:///user/SendEmail.jtp?type=node&node=12861&i=0>> wrote: >> Danny, in this case, it looks like you are right and the random don't >> provide even distribution. >> To avoid this you could devise a solution based on a node filter if >> affinity deployment doesn't do the trick for you >> https://apacheignite.readme.io/v1.8/docs/service-grid# >> section-node-filter-based-deployment >> >> Kind regards, >> Alex. >> >> On Mon, May 15, 2017 at 8:34 PM, dany74q <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=12860&i=0>> wrote: >> Hey Alex, >> >> The services are deployed dynamically after all nodes are started. >> It could be the case that the random distribution isn't "random enough" - >> and most of the services are concentrated on specific nodes. >> >> Could we somehow control the way those services are deployed within their >> cluster group, but without an affinity key ? >> >> Would love that ability ! >> >> On Mon, May 15, 2017 at 4:08 PM afedotov [via Apache Ignite Users] <[hidden > email] <http:///user/SendEmail.jtp?type=node&node=12858&i=0>> wrote: > Hi Danny. > > At the moment, cluster singleton services are deployed on a random basis. > How many Ignite nodes do you run? > How did you deploy services: after all the nodes had been started or you > have the services specified in configuration? > > Kind regards, > Alex. > > On Sat, May 13, 2017 at 1:06 AM, dany74q [via Apache Ignite Users] <[hidden > email] <http:///user/SendEmail.jtp?type=node&node=12845&i=0>> wrote: > Hi everyone, > > We use ignite's (1.7.4) service grid to deploy hundreds of (different) > services in our cluster. > There's a behavior we've noticed, once deploying that many services, that > they tend do deploy unevenly in the cluster, > that means that we can get to a point where a great number of services are > deployed on a single node, instead of spreading them out more evenly. > > Is there any way to guarantee a random or even distribution, or a > round-robin deployment policy on our nodes, without using affinity keys > (Partitions may shift, and so services can get cancelled and redeployed > with no reason) ? > > Thanks ! > - Danny > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674.html > To start a new topic under Apache Ignite Users, email [hidden email] > <http:///user/SendEmail.jtp?type=node&node=12845&i=1> > To unsubscribe from Apache Ignite Users, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674p12845.html > To unsubscribe from How to deploy ignite service evenly across the cluster > ?, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > ------------------------------ > View this message in context: Re: How to deploy ignite service evenly > across the cluster ? > <http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12858.html> > > Sent from the Apache Ignite Users mailing list archive > <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674p12860.html > To unsubscribe from How to deploy ignite service evenly across the cluster > ?, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674p12861.html > To start a new topic under Apache Ignite Users, email [hidden email] > <http:///user/SendEmail.jtp?type=node&node=12883&i=1> > To unsubscribe from Apache Ignite Users, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > View this message in context: Re: How to deploy ignite service evenly > across the cluster ? > <http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p12883.html> > Sent from the Apache Ignite Users mailing list archive > <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com. > > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674p12888.html > To unsubscribe from How to deploy ignite service evenly across the cluster > ?, click here. > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-ignite-users.70518.x6.nabble.com/How-to- > deploy-ignite-service-evenly-across-the-cluster-tp12674p13020.html > To start a new topic under Apache Ignite Users, email > ml+s70518n1...@n6.nabble.com > To unsubscribe from Apache Ignite Users, click here > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=YWxleGFuZGVyLmZlZG90b2ZmQGdtYWlsLmNvbXwxfC0xMzYxNTU0NTg=> > . > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-deploy-ignite-service-evenly-across-the-cluster-tp12674p13092.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
