Re: [squid-users] High-Availability in Squid

[Amos Jeffries]

On 4/09/2015 3:56 a.m., Imaginovskiy wrote:
> Thanks for this will about to start some testing in a test environment to see
> the behaviour of the cache_peer method listed earlier. 
> 
> Sorry to be a pain (will create a new thread for this question if needed),
> but would I need to recompile with Squid v4.0 to get SNI and ECDHE support
> for PFS? Have had to make do with normal Diffie-Hellman in 3.5.5 

You would need to be using Squid-4 yes.

Eliezer has some experimental RPM packages pre-built, but since its not
had its first formal beta quite yet the distros have not officially
picked it up.


Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Imaginovskiy]

Thanks for this will about to start some testing in a test environment to see
the behaviour of the cache_peer method listed earlier. 

Sorry to be a pain (will create a new thread for this question if needed),
but would I need to recompile with Squid v4.0 to get SNI and ECDHE support
for PFS? Have had to make do with normal Diffie-Hellman in 3.5.5 





--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/High-Availability-in-Squid-tp4672899p4673072.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Ashish Mukherjee]

Hi,

Agree that Squid is a specialized proxy and more optimal architecture for
the purpose and trying to achieve HA on the Browser side is certainly a bad
idea.

Talking specifically of a reverse proxy scenario, whether one uses Squid or
Apache mod_proxy or something else may well depend upon what other features
are needed and the traffic volumes. In many reverse proxy environments
where more complex control is needed, mod_proxy seems to be often used with
modules like mod_rewrite. My understanding of Squid is that it does great
as a proxy but does not provide these features, as that is not its
purpose.  Does Squid have its own production level extensions for some
scenarios which may be typically addressed by Apache modules?

Thanks!

- Ashish

On Sun, Aug 30, 2015 at 8:02 PM, Amos Jeffries  wrote:

> On 30/08/2015 3:58 p.m., Ashish Mukherjee wrote:
> > Why should Squid take care of HA though? Isn't it the responsibility of
> the
> > Website to ensure its uptime? Even if used as a reverse proxy, Apache may
> > do a better job at this, since one can do more sophisticated
> configurations
> > with mod_proxy.
>
> The hint is in the name of that module: mod_*proxy*.
>
> HA is an outcome of using any one or more of several _proxy features_.
>
> Squid is a proxy. The purpose of mod_proxy is to hack/squash/ proxy
> gateway capabilities (ie Squid functionality) into a piece of origin
> server software (Apache).
>
> Would you expect a dedicated/designed piece of proxy software or an
> 'other' software design with add-on tacked into it to actually be better
> at the proxying functionality?
>
> Sure you can use the other softwares add-on *if* it performs the
> function correctly and sufficiently for your needs. But that does not
> make it the function any less a proxy mechanism.
>
> On the other end Browsers also have HA functionality they call "Happy
> Eyeballs". Varuous downloaders and app update serices try to do IP-based
> filovers and recovery. And it call causes no end of trouble to the real
> proxies in the middle just doing their job properly.
>
> Amos
>
> >
> > On Sun, Aug 30, 2015 at 3:56 AM, Kinkie wrote:
> >
> >> Hi,
> >>   please see http://wiki.squid-cache.org/Features/MonitorUrl.
> >> It's available in squid 2.6 , and is one of the last few features who
> >> haven't yet made it to Squid 3.X. If anyone is interested, code and
> >> sponsorships are always welcome :)
> >>
> >> On Thu, Aug 27, 2015 at 12:10 PM, Imaginovskiy wrote:
> >>
> >>> Hi all,
> >>>
> >>> Bit of a strange one but I'm wondering if it's possible to have squid
> >>> redirect a site to a secondary backend server if the primary is down.
>
> Yes it is. This is one of the core functionalities of proxying and so
> well integrated there is *no* special configuration needed. In fact you
> have to explicitly configure alternative if you want to avoid it.
>
> >>> Have
> >>> been looking into this but haven't seen much similar to this.
> Currently I
> >>> have my setup along the lines of this;
> >>>
> >>> Client -> Squid -> Backend1
> >>>
> >>> but in the event that Backend1 is down, the following should be done;
> >>>
> >>> Client -> Squid -> Backend2
> >>>
> >>> Is squid capable of monitoring connections to peer or redirecting
> based on
> >>> an ACL looking for some HTTP error code?
>
> I would almost bet you already have it setup in your squid.conf:
>
>  cache_peer Backend1.example.com ...
>  cache_peer Backend2.example.com ...
>
>
> Backend2 will be used only if Backend1 is detected as unavailable, or
> starts getting overloaded/slow (yes the HA includes latency optimization
> too), or starts returning some retriable status responses (most common
> is 502).
>
>
> Yes, simply the line order itself. Everything else is just tuning of the
> HA parameters.
>
> To prevent flapping there is a default of 10 requests that need to have
> failed (in a row) before the 'dead' detection kicks in to send 100% to
> Backend2. Those 10 will make TCP connect attempts to Backend1 before
> retrying with Backend2 (if enough time remains in the forward_timeout).
> That is configurable, see cache_peer documentation for timeouts and
> failure options. background-ping may also be of interest to you.
>
>
> You can improve further by using DNS-based HA availability. Pointing
> each Backend1 and Backend2 domain names at multiple IPs. But be wary of
> those Browsers and their (overly) "Happy eyeballs".
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Amos Jeffries]

On 31/08/2015 6:23 p.m., Ashish Mukherjee wrote:
> Hi,
> 
> Agree that Squid is a specialized proxy and more optimal architecture for
> the purpose and trying to achieve HA on the Browser side is certainly a bad
> idea.
> 
> Talking specifically of a reverse proxy scenario, whether one uses Squid or
> Apache mod_proxy or something else may well depend upon what other features
> are needed and the traffic volumes. In many reverse proxy environments
> where more complex control is needed, mod_proxy seems to be often used with
> modules like mod_rewrite.

Does it not strike you as somehow wrong that "flexibility" is gained by
mangling the original request URL in ways such that the script engines
do not see what the original actually is?

A very large portion of that "complexity" in the backend applications
and CGI is having to deal with the way the URL was (or might have been)
mangled by the server itself. Then guessing what URIs to output that the
client would understand in the public context.


> My understanding of Squid is that it does great
> as a proxy but does not provide these features, as that is not its
> purpose.  Does Squid have its own production level extensions for some
> scenarios which may be typically addressed by Apache modules?


The key is again in the middle word of the phrase "reverse proxy
scenario". If it is *proxy* related Squid does it.


extensions? everything relevant is core functionality to a proxy. But
yes, there are addons and extensions for integrating to particular
network situations. We call them "helpers".

"production level"? Squid is the de-facto benchmark all the other
proxying software is compared against. Including mod_proxy. We usually
see them crowing about how fast they are at one particular little
targeted feature while glossing over the things they traded away to get
that speed. Squid goes somewhat slower overall, but fast enough and
"does everything".


virtual hosting?
 http_port 80 accel vhost
 https_port 80 accel vhost

mod_rewrite?
 url_rewrite_program (a helper interface, script your own poison)

mod_proxy?
 cache_peer

authentication?
 auth_param (a helper interface)

security policies?
 acl (including a helper interface ACL)
 many *_access directives

message payload transcoding?
 ESI
 icap_service
 ecap_service


Okay that last one is not internal to Squid exactly (except ESI is), but
that is because of the line between proxy and origin: touching the
message content is not a proxy functionality.


The one scenario where using Apache modules makes complete sense is when
dealing with FastCGI and/or a mix of FastCGI and static content on the
same server. Thats where Apache came from, and it does it well.

Once you start getting into using HTTP to pull from other servers and/or
ports on one server you are moving well into territory where a proper
proxy is the better tool (not just Squid, there are others). Apache
simply wont scale. Squid scales both horizontally and vertically. Our
poster child installations are Wikimedia (~200 Squid serving up
Wikipedia on a scale of TB/sec), and FrontierNET at CERN (a mesh layout
pumping Petabytes of science data around, where the small files are
measured in GB).

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Amos Jeffries]

On 30/08/2015 3:58 p.m., Ashish Mukherjee wrote:
 Why should Squid take care of HA though? Isn't it the responsibility of the
 Website to ensure its uptime? Even if used as a reverse proxy, Apache may
 do a better job at this, since one can do more sophisticated configurations
 with mod_proxy.

The hint is in the name of that module: mod_*proxy*.

HA is an outcome of using any one or more of several _proxy features_.

Squid is a proxy. The purpose of mod_proxy is to hack/squash/ proxy
gateway capabilities (ie Squid functionality) into a piece of origin
server software (Apache).

Would you expect a dedicated/designed piece of proxy software or an
'other' software design with add-on tacked into it to actually be better
at the proxying functionality?

Sure you can use the other softwares add-on *if* it performs the
function correctly and sufficiently for your needs. But that does not
make it the function any less a proxy mechanism.

On the other end Browsers also have HA functionality they call Happy
Eyeballs. Varuous downloaders and app update serices try to do IP-based
filovers and recovery. And it call causes no end of trouble to the real
proxies in the middle just doing their job properly.

Amos

 
 On Sun, Aug 30, 2015 at 3:56 AM, Kinkie wrote:
 
 Hi,
   please see http://wiki.squid-cache.org/Features/MonitorUrl.
 It's available in squid 2.6 , and is one of the last few features who
 haven't yet made it to Squid 3.X. If anyone is interested, code and
 sponsorships are always welcome :)

 On Thu, Aug 27, 2015 at 12:10 PM, Imaginovskiy wrote:

 Hi all,

 Bit of a strange one but I'm wondering if it's possible to have squid
 redirect a site to a secondary backend server if the primary is down.

Yes it is. This is one of the core functionalities of proxying and so
well integrated there is *no* special configuration needed. In fact you
have to explicitly configure alternative if you want to avoid it.

 Have
 been looking into this but haven't seen much similar to this. Currently I
 have my setup along the lines of this;

 Client - Squid - Backend1

 but in the event that Backend1 is down, the following should be done;

 Client - Squid - Backend2

 Is squid capable of monitoring connections to peer or redirecting based on
 an ACL looking for some HTTP error code?

I would almost bet you already have it setup in your squid.conf:

 cache_peer Backend1.example.com ...
 cache_peer Backend2.example.com ...


Backend2 will be used only if Backend1 is detected as unavailable, or
starts getting overloaded/slow (yes the HA includes latency optimization
too), or starts returning some retriable status responses (most common
is 502).


Yes, simply the line order itself. Everything else is just tuning of the
HA parameters.

To prevent flapping there is a default of 10 requests that need to have
failed (in a row) before the 'dead' detection kicks in to send 100% to
Backend2. Those 10 will make TCP connect attempts to Backend1 before
retrying with Backend2 (if enough time remains in the forward_timeout).
That is configurable, see cache_peer documentation for timeouts and
failure options. background-ping may also be of interest to you.


You can improve further by using DNS-based HA availability. Pointing
each Backend1 and Backend2 domain names at multiple IPs. But be wary of
those Browsers and their (overly) Happy eyeballs.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Kinkie]

Hi,
  please see http://wiki.squid-cache.org/Features/MonitorUrl.
It's available in squid 2.6 , and is one of the last few features who
haven't yet made it to Squid 3.X. If anyone is interested, code and
sponsorships are always welcome :)

On Thu, Aug 27, 2015 at 12:10 PM, Imaginovskiy m...@stellarise.com wrote:

 Hi all,

 Bit of a strange one but I'm wondering if it's possible to have squid
 redirect a site to a secondary backend server if the primary is down. Have
 been looking into this but haven't seen much similar to this. Currently I
 have my setup along the lines of this;

 Client - Squid - Backend1

 but in the event that Backend1 is down, the following should be done;

 Client - Squid - Backend2

 Is squid capable of monitoring connections to peer or redirecting based on
 an ACL looking for some HTTP error code?

 Thanks.





 --
 View this message in context:
 http://squid-web-proxy-cache.1019090.n4.nabble.com/High-Availability-in-Squid-tp4672899.html
 Sent from the Squid - Users mailing list archive at Nabble.com.
 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users




-- 
Francesco
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Luis Daniel Lucio Quiroz]

You may want to play with PEN
On Aug 29, 2015 6:27 PM, Kinkie gkin...@gmail.com wrote:

 Hi,
   please see http://wiki.squid-cache.org/Features/MonitorUrl.
 It's available in squid 2.6 , and is one of the last few features who
 haven't yet made it to Squid 3.X. If anyone is interested, code and
 sponsorships are always welcome :)

 On Thu, Aug 27, 2015 at 12:10 PM, Imaginovskiy m...@stellarise.com wrote:

 Hi all,

 Bit of a strange one but I'm wondering if it's possible to have squid
 redirect a site to a secondary backend server if the primary is down. Have
 been looking into this but haven't seen much similar to this. Currently I
 have my setup along the lines of this;

 Client - Squid - Backend1

 but in the event that Backend1 is down, the following should be done;

 Client - Squid - Backend2

 Is squid capable of monitoring connections to peer or redirecting based on
 an ACL looking for some HTTP error code?

 Thanks.





 --
 View this message in context:
 http://squid-web-proxy-cache.1019090.n4.nabble.com/High-Availability-in-Squid-tp4672899.html
 Sent from the Squid - Users mailing list archive at Nabble.com.
 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users




 --
 Francesco

 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] High-Availability in Squid

[Ashish Mukherjee]

Why should Squid take care of HA though? Isn't it the responsibility of the
Website to ensure its uptime? Even if used as a reverse proxy, Apache may
do a better job at this, since one can do more sophisticated configurations
with mod_proxy.

On Sun, Aug 30, 2015 at 3:56 AM, Kinkie gkin...@gmail.com wrote:

 Hi,
   please see http://wiki.squid-cache.org/Features/MonitorUrl.
 It's available in squid 2.6 , and is one of the last few features who
 haven't yet made it to Squid 3.X. If anyone is interested, code and
 sponsorships are always welcome :)

 On Thu, Aug 27, 2015 at 12:10 PM, Imaginovskiy m...@stellarise.com wrote:

 Hi all,

 Bit of a strange one but I'm wondering if it's possible to have squid
 redirect a site to a secondary backend server if the primary is down. Have
 been looking into this but haven't seen much similar to this. Currently I
 have my setup along the lines of this;

 Client - Squid - Backend1

 but in the event that Backend1 is down, the following should be done;

 Client - Squid - Backend2

 Is squid capable of monitoring connections to peer or redirecting based on
 an ACL looking for some HTTP error code?

 Thanks.





 --
 View this message in context:
 http://squid-web-proxy-cache.1019090.n4.nabble.com/High-Availability-in-Squid-tp4672899.html
 Sent from the Squid - Users mailing list archive at Nabble.com.
 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users




 --
 Francesco

 ___
 squid-users mailing list
 squid-users@lists.squid-cache.org
 http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] High-Availability in Squid

[Imaginovskiy]

Hi all, 

Bit of a strange one but I'm wondering if it's possible to have squid
redirect a site to a secondary backend server if the primary is down. Have
been looking into this but haven't seen much similar to this. Currently I
have my setup along the lines of this;

Client - Squid - Backend1

but in the event that Backend1 is down, the following should be done;

Client - Squid - Backend2 

Is squid capable of monitoring connections to peer or redirecting based on
an ACL looking for some HTTP error code?

Thanks.





--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/High-Availability-in-Squid-tp4672899.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users