The relay copies all data it receives to all configured backends. The basic flow should be something like this:
client -> load balancer -> relays -> influxdb, kapacitor (can have multiple instances of each) Kapacitor should be configured to disable subscriptions and use the load balancer when querying InfluxDB. On Wednesday, November 30, 2016 at 7:41:08 AM UTC-7, jaya...@gmail.com wrote: > > Hi Nathan > > While reading this discussion, I got a question on your statement > "I would look at option 2. This is where my questions comes in. If you > are running multiple relays, which I recommend you do, then it should not > be too difficult to update the config on each relay one at a time, making > use of the load balancer. Thus enabling you to add/remove Kapacitor > instances dynamically without downtime. " > > My Questions here are > 1. If Relays are configured with Load balancer to sends the data points to > Kapacitor, the load balancer will send to only kapacitor instance right? > The load balance will not distribute the data points to multiple kapacitors > instances which are behind the load balancer. > > 2. How to configure the relays to send the data points to Kapacitor and > what is the configuration required in Kapacitor? > > Thanks > Jayanna > > > > > On Friday, November 18, 2016 at 1:13:04 PM UTC-5, nath...@influxdb.com > wrote: > > Kirtimaan, > > > > > > There are a few things at play here. Here are the questions I had while > reading about your setup: > > > > > > 1) Do you want multiple Kapacitor instances because of > redundancy/isolation or because you are worried about capacity? > > 2) Are you running multiple relays? If so how easy is it to reconfigure > them? > > 3) How often do you expect you will need a new Kapacitor instance? > Meaning, once every team is using it, are new teams created frequently or > would the need to create new Kapacitor instances flatten out? > > > > > > I don't need the answer to these question, but they should point you in > the right direction. > > > > > > Now to answer your questions: > > > > > > 1) I don't forsee any issues having multiple Kapacitor instances > connecting to the same InfluxDB instance. The Kapacitor instances will be > isolated and unaware of each other, InfluxDB will handle the subscriptions > well. We have tested this use case. > > 2) This is the tricky part. Since each InfluxDB server is receiving all > the data you can't have Kapacitor subscribe to both InfluxDB instances > without duplicating all your data to Kapacitor. As a result if you are > going to use subscriptions you have to pick only one InfluxDB server, which > means your HA Kapacitor story now has two points of failure, Kapacitor > itself and the InfluxDB server. Obviously not ideal > > 3) This depends on the workload each task adds, but we have tested > Kapacitor for use cases of 1000+ tasks. In other words as long as you have > the cpu/ram you need, which is sounds like you do, a single Kapacitor > instance should be efficient at handling many tasks. > > > > > > Since the answer to #2 is not ideal I would recommend the best > architecture would be to use the relay and not subscriptions. That leaves > two options: > > > > > > 1. Use a single Kapacitor instance > > 2. Run multiple Kapacitor instances behind the relay. > > > > > > Option 1 is the simplest but you risk one team creating a heavy task and > breaking alerting for all other teams. If this is a serious concern then I > would look at option 2. This is where my questions comes in. If you are > running multiple relays, which I recommend you do, then it should not be > too difficult to update the config on each relay one at a time, making use > of the load balancer. Thus enabling you to add/remove Kapacitor instances > dynamically without downtime. Depending on the frequency of adding and > removing Kapacitor instances and the importance of isolation/redundancy > you can invest in making that process as flexible subscriptions. > > > > > > Does that help give you a direction? Do you have any further questions? > > > > > > As a final note, if you do run into performance issues please let us > know. Kapacitor is still a young product and when new use cases pop up, we > want to learn how we can improve Kapacitor for those types of workloads. > That said what you are doing here seems reasonable and within the scope of > what we have tested. > > > > > > Thanks > > > > On Friday, November 18, 2016 at 8:23:04 AM UTC-7, kraj...@gmail.com > wrote:I am setting up high availability influxdb setup as mentioned on > influxdb site. > > loadbalancer-->relay-->influxdbs(pri/sec) and kapacitor > > Though there are obvious advantages to sending real time traffic via > influxdbrelay to Kapacitor together with influxdbs,it takes away > flexibility to spin up multiple kapacitors in the environment without > updating relay configuration. > > Our usecase has several teams who want to control their own alerting > configuration in kapacitor so we are looking for spinning up multiple > kapacitors in the environment (one controlled by each team). I have couple > questions around that: > > 1) Do you forsee any issues with having multiple kapacitors in the > environment talking to same influxdb servers (if they subscribe themselves > to influxdb servers instead of relay pushing traffic to kapacitors for > which it would need to be updated). > > 2) If i have two influxdb servers which are identical (primary and > secondary), where should my kapacitor point to? (load balancer DNS or > influxdb-primary and influxdb-secondary in the list [host1, host2]? > > 3) How many tasks can one kapacitor handle ( I can throw as many > resources (cpu/ram) on it as i want)? > > > > Thanks > > Kirtimaan > > -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to influxdb+unsubscr...@googlegroups.com. To post to this group, send email to influxdb@googlegroups.com. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/4de03696-97cf-4073-80c3-b4eafab3bee4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
