Re: Varnish won't start because backend host resolves to too many addresses, but they are all identical IPs

[Batanun B]

> https://github.com/nigoroll/libvmod-dynamic/blob/master/src/vmod_dynamic.vcc#L538-L583
> maybe?
>

> I'm sure Nils will pipe up here if you need help, and if you want more
> synchronous assistance, there's always the discord channel
> .

Thanks! :)

___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Varnish won't start because backend host resolves to too many addresses, but they are all identical IPs

[Guillaume Quintard]

> The documentation seems a bit lacking (no full VCL example), but I guess
I could use their test cases as examples.

https://github.com/nigoroll/libvmod-dynamic/blob/master/src/vmod_dynamic.vcc#L538-L583
maybe?
I'm sure Nils will pipe up here if you need help, and if you want more
synchronous assistance, there's always the discord channel
<https://varnish-cache.org/support/>.

> The dynamic one seems like the only one that supports community edition
LTS 6.0.

Yes, of the three, that's the only one that will support that one (VS is
focused on the Enterprise version, and I lack the time to port vmods on 6.0
(but I'll welcome the help)).

Cheers,

-- 
Guillaume Quintard


On Wed, Apr 19, 2023 at 9:02 AM Batanun B  wrote:

> > Shouldn't your DNS entries be clean? ;-)
>
> Preferably, but I blame Microsoft here 
>
> The problem went away by itself when I tried starting again like half an
> hour later or so, so I guess it was a temporary glitch in the matrix.
>
> As far as I understand it, the IPs of these machines only change if they
> are deleted and created again. We do it occasionally in test/staging, and
> there we can live with Varnish needing to be restarted. In production we
> don't really delete them once they are properly setup, unless there is some
> major problem and then a restart of the load balanced varnish servers
> should not be a concern.
>
> Thanks for your vmod suggestions! I will check them out. The dynamic one
> seems like the only one that supports community edition LTS 6.0. The
> documentation seems a bit lacking (no full VCL example), but I guess I
> could use their test cases as examples.
>
> --
> *From:* Guillaume Quintard 
> *Sent:* Wednesday, April 19, 2023 4:42 PM
> *To:* Batanun B 
> *Cc:* varnish-misc@varnish-cache.org 
> *Subject:* Re: Varnish won't start because backend host resolves to too
> many addresses, but they are all identical IPs
>
> The fact the IPs are identical is weird, but I wouldn't be surprised if
> the dns entry actually contained 3 identical IPs.
>
> > Shouldn't Varnish be able to figure out that in that case it can just
> choose any one and it will work as expected?
>
> Shouldn't your DNS entries be clean? ;-)
>
> Honestly, if the IP(s) behind the service name is liable to change, you
> shouldn't use a dynamic backend because Varnish resolves the IP when the
> VCL is loaded, so if the IP changes behind your back, Varnish won't follow
> it, and you'll be screwed.
> Instead, you should use dynamic backends, of which there are a handful:
> - dynamic <https://github.com/nigoroll/libvmod-dynamic>, by UPLEX: it's
> been around for ages, it's battle-tested, and it's included in the oficial
> Varnish Docker image <https://hub.docker.com/_/varnish>
> - udo+activedns
> <https://docs.varnish-software.com/varnish-enterprise/vmods/udo/#subscribe>,
> by Varnish Software: the design is slightly different and allows you to
> specify pretty much any load-balancing policy you might need. You'll need a
> subscription but you'll get excellent support (disclaimer, I'm an ex
> employee)
> - reqwest
> <https://github.com/gquintard/vmod_reqwest#backend-https-following-up-to-5-redirect-hops-and-brotli-auto-decompression>,
> by yours truly: the interface focuses on providing a simple experience and
> a few bells and whistles (HTTPS, HTTP2, brotli, following redirects)
>
> As you can see, the static backend's reluctance to fully handle DNS has
> been a fertile ground for vmods :-)
>
> --
> Guillaume Quintard
>
>
> On Wed, Apr 19, 2023 at 1:49 AM Batanun B  wrote:
>
> All of the sudden Varnish fails to start in my development environment,
> and gives me the following error message:
>
> Message from VCC-compiler:
> Backend host "redacted-hostname": resolves to too many addresses.
> Only one IPv4 and one IPv6 are allowed.
> Please specify which exact address you want to use, we found all of these:
>  555.123.123.3:80
>  555.123.123.3:80
>  555.123.123.3:80
>
> I have changed the hostname and the IP above to not expose our server, but
> all three IP numbers are 100% identical. Shouldn't Varnish be able to
> figure out that in that case it can just choose any one and it will work as
> expected? It really should remove duplicates, and only if there are more
> than one non-duplicate IP then it should fail.
>
> The problem is that the backend host is a so called "app service" in
> Microsoft Azure, which is basically a platform as a service (PaaS), where
> Microsoft handles the networking including the domain name (no user access
> it directly). I have no idea why it suddenly resolves to multiple duplicate
> IPs.
> __

Re: Varnish won't start because backend host resolves to too many addresses, but they are all identical IPs

[Batanun B]

> Shouldn't your DNS entries be clean? ;-)

Preferably, but I blame Microsoft here 

The problem went away by itself when I tried starting again like half an hour 
later or so, so I guess it was a temporary glitch in the matrix.

As far as I understand it, the IPs of these machines only change if they are 
deleted and created again. We do it occasionally in test/staging, and there we 
can live with Varnish needing to be restarted. In production we don't really 
delete them once they are properly setup, unless there is some major problem 
and then a restart of the load balanced varnish servers should not be a concern.

Thanks for your vmod suggestions! I will check them out. The dynamic one seems 
like the only one that supports community edition LTS 6.0. The documentation 
seems a bit lacking (no full VCL example), but I guess I could use their test 
cases as examples.


From: Guillaume Quintard 
Sent: Wednesday, April 19, 2023 4:42 PM
To: Batanun B 
Cc: varnish-misc@varnish-cache.org 
Subject: Re: Varnish won't start because backend host resolves to too many 
addresses, but they are all identical IPs

The fact the IPs are identical is weird, but I wouldn't be surprised if the dns 
entry actually contained 3 identical IPs.

> Shouldn't Varnish be able to figure out that in that case it can just choose 
> any one and it will work as expected?

Shouldn't your DNS entries be clean? ;-)

Honestly, if the IP(s) behind the service name is liable to change, you 
shouldn't use a dynamic backend because Varnish resolves the IP when the VCL is 
loaded, so if the IP changes behind your back, Varnish won't follow it, and 
you'll be screwed.
Instead, you should use dynamic backends, of which there are a handful:
- dynamic<https://github.com/nigoroll/libvmod-dynamic>, by UPLEX: it's been 
around for ages, it's battle-tested, and it's included in the oficial Varnish 
Docker image<https://hub.docker.com/_/varnish>
- 
udo+activedns<https://docs.varnish-software.com/varnish-enterprise/vmods/udo/#subscribe>,
 by Varnish Software: the design is slightly different and allows you to 
specify pretty much any load-balancing policy you might need. You'll need a 
subscription but you'll get excellent support (disclaimer, I'm an ex employee)
- 
reqwest<https://github.com/gquintard/vmod_reqwest#backend-https-following-up-to-5-redirect-hops-and-brotli-auto-decompression>,
 by yours truly: the interface focuses on providing a simple experience and a 
few bells and whistles (HTTPS, HTTP2, brotli, following redirects)

As you can see, the static backend's reluctance to fully handle DNS has been a 
fertile ground for vmods :-)

--
Guillaume Quintard


On Wed, Apr 19, 2023 at 1:49 AM Batanun B 
mailto:bata...@hotmail.com>> wrote:
All of the sudden Varnish fails to start in my development environment, and 
gives me the following error message:

Message from VCC-compiler:
Backend host "redacted-hostname": resolves to too many addresses.
Only one IPv4 and one IPv6 are allowed.
Please specify which exact address you want to use, we found all of these:
 555.123.123.3:80
 555.123.123.3:80
 555.123.123.3:80

I have changed the hostname and the IP above to not expose our server, but all 
three IP numbers are 100% identical. Shouldn't Varnish be able to figure out 
that in that case it can just choose any one and it will work as expected? It 
really should remove duplicates, and only if there are more than one 
non-duplicate IP then it should fail.

The problem is that the backend host is a so called "app service" in Microsoft 
Azure, which is basically a platform as a service (PaaS), where Microsoft 
handles the networking including the domain name (no user access it directly). 
I have no idea why it suddenly resolves to multiple duplicate IPs.
___
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Varnish won't start because backend host resolves to too many addresses, but they are all identical IPs

[Dridi Boukelmoune]

On Wed, Apr 19, 2023 at 2:44 PM Guillaume Quintard
 wrote:
>
> The fact the IPs are identical is weird, but I wouldn't be surprised if the 
> dns entry actually contained 3 identical IPs.
>
> > Shouldn't Varnish be able to figure out that in that case it can just 
> > choose any one and it will work as expected?
>
> Shouldn't your DNS entries be clean? ;-)

It should, but Varnish could probably make an effort here.
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Varnish won't start because backend host resolves to too many addresses, but they are all identical IPs

[Guillaume Quintard]

The fact the IPs are identical is weird, but I wouldn't be surprised if the
dns entry actually contained 3 identical IPs.

> Shouldn't Varnish be able to figure out that in that case it can just
choose any one and it will work as expected?

Shouldn't your DNS entries be clean? ;-)

Honestly, if the IP(s) behind the service name is liable to change, you
shouldn't use a dynamic backend because Varnish resolves the IP when the
VCL is loaded, so if the IP changes behind your back, Varnish won't follow
it, and you'll be screwed.
Instead, you should use dynamic backends, of which there are a handful:
- dynamic , by UPLEX: it's
been around for ages, it's battle-tested, and it's included in the oficial
Varnish Docker image 
- udo+activedns
,
by Varnish Software: the design is slightly different and allows you to
specify pretty much any load-balancing policy you might need. You'll need a
subscription but you'll get excellent support (disclaimer, I'm an ex
employee)
- reqwest
,
by yours truly: the interface focuses on providing a simple experience and
a few bells and whistles (HTTPS, HTTP2, brotli, following redirects)

As you can see, the static backend's reluctance to fully handle DNS has
been a fertile ground for vmods :-)

-- 
Guillaume Quintard


On Wed, Apr 19, 2023 at 1:49 AM Batanun B  wrote:

> All of the sudden Varnish fails to start in my development environment,
> and gives me the following error message:
>
> Message from VCC-compiler:
> Backend host "redacted-hostname": resolves to too many addresses.
> Only one IPv4 and one IPv6 are allowed.
> Please specify which exact address you want to use, we found all of these:
>  555.123.123.3:80
>  555.123.123.3:80
>  555.123.123.3:80
>
> I have changed the hostname and the IP above to not expose our server, but
> all three IP numbers are 100% identical. Shouldn't Varnish be able to
> figure out that in that case it can just choose any one and it will work as
> expected? It really should remove duplicates, and only if there are more
> than one non-duplicate IP then it should fail.
>
> The problem is that the backend host is a so called "app service" in
> Microsoft Azure, which is basically a platform as a service (PaaS), where
> Microsoft handles the networking including the domain name (no user access
> it directly). I have no idea why it suddenly resolves to multiple duplicate
> IPs.
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc