Re: Server-template and randomized DNS responses

2018-02-12 Thread Baptiste
To share the solution with everyone, the problem was fixed by a configuration update. Mike added a "accepted_payload_size 1024" into his resolvers section. HAProxy announces by an accepted payload of 512 bytes, which let the place for only 3 records reported by consul. With a payload of 1024, up t

Re: Server-template and randomized DNS responses

2018-02-12 Thread Baptiste
Replying to myself :) I think I spotted a bug in HAProxy as well. For some reasons, when I run HAProxy in debug more, I never ever have the issue (all my servers are properly populated and maintained). I did a strace of the process running in daemon mode in the container, and I can confirm the fo

Re: Server-template and randomized DNS responses

2018-02-12 Thread Baptiste
Continuing on my investigation I found an other interesting piece of information: I run haproxy and my consul environment in a docker host, through docker-compose and I can reproduce the same issue as you. Basically, I have a service delivered by 20 containers, and HAProxy in docker can see only 10

Re: Server-template and randomized DNS responses

2018-02-12 Thread Baptiste
First, I confirm the following bug in consul 1.0.5: - start a X instances of a service - scale the service to X+Y (with Y > 1) ==> then consul crashes... >From time to time, I also saw HAProxy getting only 10 servers from 20 for a given service. I'll revert to 1.0.2 for now. The order of the retu

Re: Server-template and randomized DNS responses

2018-02-11 Thread Чепайкин Михаил
Im on Consul 1.0.2. Why do you think this issue is about serving SRV over UDP, rather than about different order of SRV or A records returned by Consul DNS with consecutive requests? On 11 February 2018 at 18:46, Baptiste wrote: > Hi, > > What consul version are you using? > I'm facing the same

Re: Server-template and randomized DNS responses

2018-02-11 Thread Baptiste
Hi, What consul version are you using? I'm facing the same issue in my consul lab. That said, it seems to be a bug in consul, not able to serve too many SRV records over UDP. I even triggered a consul crash (using 1.0.5 version). I'm still investigating this issue and will come back to you as soon

Re: Server-template and randomized DNS responses

2018-02-07 Thread Чепайкин Михаил
Hi I’ve changed configuration as you suggested: backend tsdb_backend_query server-template tsdb_query 5 _mfm-monitor-opentsdb._tcp.service.mfmconsul:4242 check resolvers dns inter 1000 Logs are kinda different - backend servers now go UP and DOWN, but seems the same - ip addresses changing in

Re: Server-template and randomized DNS responses

2018-02-07 Thread Baptiste
Hi You're not using SRV records and that may be the root cause of your issue. Please try something like this: backend tsdb_backend_query server-template tsdb_query 5 _mfm-monitor-opentsdb._tcp.service.mfmconsul:4242 check resolvers dns inter 1000 if "mfm-monitor-opentsdb" is your service name

Server-template and randomized DNS responses

2018-02-07 Thread Чепайкин Михаил
Hi! I have a Consul as service discovery tool and HAProxy as load balancer. In Consul registered a service running on a number of servers, and this service can be scaled by adding and removing nodes and by moving nodes from one server to another. Consul has DNS service which randomizes responses