Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Alex Rousskov]

On 2/11/19 5:01 PM, Amos Jeffries wrote:
> On 12/02/19 6:11 am, Alex Rousskov wrote:
>> On 2/2/19 12:37 PM, elie...@ngtech.co.il wrote:
>>> Can we change the default from "startup=0" to "startup=1" ?
>>
>> We obviously can. The real question is whether we should. AFAICT, the
>> default changed to zero in commit 48d54e4. In that commit message, I did
>> not find an explanation of _why_ the default was changed

> The default being 0 was extra performance tuning

When there is a trade-off (e.g., with detecting misconfigurations), the
choice of the default should not be driven by performance optimizations
or special deployment environments IMO -- those who need to optimize
performance or accommodate special environments can and should tune
their squid.conf settings explicitly instead of relying on defaults.


>> Before we restart changing defaults, we should agree on some principles
>> that should guide us in selecting the right default. Please feel free to
>> propose/defend them if you want to work on this change. Here is an
>> example of a possible principle we could use for situations where the
>> default option value is not clear/obvious:

>> * The default should maximize the chance that a misconfiguration is
>> discovered at startup time (rather than at runtime).

> * the default should not induce overly much RAM usage.

> * the default should not cause unnecessary processes to run.

The last two are too obvious to be practically useful AFAICT: Clearly,
we do not want "overly much" or "unnecessary" of anything.


>  ** Default 0 (current status-quo) assumption is that the admin might
> configure a helper that is never used.

>  ** Default of 1 that all helpers are needed, but maybe fast enough not
> to need many forks().

>  ** Default 2+ that traffic load and helper usage is going to be high
> with all helpers handling a lot of I/O.

Yes, but those use cases are not principles that can guide us towards
selecting the right default. Clearly, any reasonable default value will
match some use case or another.

Also, "configure a helper that is never used" is arguably a
misconfiguration (that we should, to the extent possible, highlight
rather than conceal).

There is another general principle that says "Admin should only pay for
the features they enable", but it does not help in this particular
situation AFAICT because the admin _is_ configuring the helper
explicitly, so we have the right to charge the admin for that (by
increasing startup costs).

Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Amos Jeffries]

On 12/02/19 6:11 am, Alex Rousskov wrote:
> On 2/2/19 12:37 PM, elie...@ngtech.co.il wrote:
>> Can we change the default from "startup=0" to "startup=1" ?
> 
> We obviously can. The real question is whether we should. AFAICT, the
> default changed to zero in commit 48d54e4. In that commit message, I did
> not find an explanation of _why_ the default was changed, but I could
> have missed it. I only saw references to why the new default may cause
> problems.

This feature was added with a focus on improving efficiency for small
integrated systems (OpenWRT, RaspberryPi, Android etc.) with some
additional benefits for larger systems.

The small limited-resource systems lack of RAM meant the default of 10
always running helpers of each type consumed sometimes considerably more
memory than was available in total or necessary.

Even larger resource-rick systems were having issues with admin
(mis)configuring hundreds of NTLM helpers in attempts to avoid helpers
all being busy at peak login times.

Most of that was solved by going dynamic. The default being 0 was extra
performance tuning - in hindsight perhapse not the best choice but
suited the use-case for limited memory devices and we have not had many
issues reported about it. A default of 1 would still solve most of the
issues as well as detecting helper crashes on startup. It would mean a
somewhat slower (few seconds) startup on some devices though.


> 
> Before we restart changing defaults, we should agree on some principles
> that should guide us in selecting the right default. Please feel free to
> propose/defend them if you want to work on this change. Here is an
> example of a possible principle we could use for situations where the
> default option value is not clear/obvious:
> 
> * The default should maximize the chance that a misconfiguration is
> discovered at startup time (rather than at runtime).
> 

* the default should not induce overly much RAM usage.

* the default should not cause unnecessary processes to run.

This last is the trickiest because it is a bit fuzzy and relies on
assumptions about admin behaviours - which also vary over time as
experience is gained or forgotten.

 ** Default 0 (current status-quo) assumption is that the admin might
configure a helper that is never used.

 ** Default of 1 that all helpers are needed, but maybe fast enough not
to need many forks().

 ** Default 2+ that traffic load and helper usage is going to be high
with all helpers handling a lot of I/O.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Alex Rousskov]

On 2/2/19 12:37 PM, elie...@ngtech.co.il wrote:
> Can we change the default from "startup=0" to "startup=1" ?

We obviously can. The real question is whether we should. AFAICT, the
default changed to zero in commit 48d54e4. In that commit message, I did
not find an explanation of _why_ the default was changed, but I could
have missed it. I only saw references to why the new default may cause
problems.

Before we restart changing defaults, we should agree on some principles
that should guide us in selecting the right default. Please feel free to
propose/defend them if you want to work on this change. Here is an
example of a possible principle we could use for situations where the
default option value is not clear/obvious:

* The default should maximize the chance that a misconfiguration is
discovered at startup time (rather than at runtime).

Alex.


> -Original Message-
> From: squid-users  On Behalf Of 
> Amos Jeffries
> Sent: Saturday, February 2, 2019 14:33
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] Squid doesn't execute url_rewrite_program 
> /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf
> 
> On 2/02/19 7:56 am, Roberto Carna wrote:
>> Dear Amos, thanks for your comments.
>>
>> I realized that I have some clues in cache.log:
>>
>> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/20 'squidGuard'
>> processes
>> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squidGuard' processes
>> needed.
>> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/5
>> 'squid_ldap_auth' processes
>> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squid_ldap_auth'
>> processes needed.
>>
>> These lines appears after I execute "systemctl reload squid".
>>
>> Users and rights are OK.
>>
>> Please can you help me one more time?
>>
> 
> The above log lines indicate that Squid is waiting for traffic before
> going to the trouble of starting helpers. This is the default since
> Squid-3.2.
> 
> If you want to change that the relevant directives for these two helpers
> are:
>  <http://www.squid-cache.org/Doc/config/url_rewrite_children/>
>  <http://www.squid-cache.org/Doc/config/auth_param/> under "children"
> 
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[eliezer]

Can we change the default from "startup=0" to "startup=1" ?

Thanks,
Eliezer


Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: elie...@ngtech.co.il


-Original Message-
From: squid-users  On Behalf Of Amos 
Jeffries
Sent: Saturday, February 2, 2019 14:33
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Squid doesn't execute url_rewrite_program 
/usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

On 2/02/19 7:56 am, Roberto Carna wrote:
> Dear Amos, thanks for your comments.
> 
> I realized that I have some clues in cache.log:
> 
> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/20 'squidGuard'
> processes
> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squidGuard' processes
> needed.
> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/5
> 'squid_ldap_auth' processes
> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squid_ldap_auth'
> processes needed.
> 
> These lines appears after I execute "systemctl reload squid".
> 
> Users and rights are OK.
> 
> Please can you help me one more time?
> 

The above log lines indicate that Squid is waiting for traffic before
going to the trouble of starting helpers. This is the default since
Squid-3.2.

If you want to change that the relevant directives for these two helpers
are:
 <http://www.squid-cache.org/Doc/config/url_rewrite_children/>
 <http://www.squid-cache.org/Doc/config/auth_param/> under "children"

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Amos Jeffries]

On 2/02/19 7:56 am, Roberto Carna wrote:
> Dear Amos, thanks for your comments.
> 
> I realized that I have some clues in cache.log:
> 
> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/20 'squidGuard'
> processes
> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squidGuard' processes
> needed.
> 2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/5
> 'squid_ldap_auth' processes
> 2019/02/01 15:51:44 kid1| helperOpenServers: No 'squid_ldap_auth'
> processes needed.
> 
> These lines appears after I execute "systemctl reload squid".
> 
> Users and rights are OK.
> 
> Please can you help me one more time?
> 

The above log lines indicate that Squid is waiting for traffic before
going to the trouble of starting helpers. This is the default since
Squid-3.2.

If you want to change that the relevant directives for these two helpers
are:
 
  under "children"

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Eliezer Croitoru]

Share your full squid.conf removing the confidential details and we might be 
able to understand the issue.

If you insist on using SquidGuard please use the latest version as an external 
ACL helper and not as a url_rewrite_program.

If you need instructions how to implement this we can try to help you.

 

Eliezer

 



  Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: elie...@ngtech.co.il



 

From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On Behalf 
Of Roberto Carna
Sent: Thursday, January 31, 2019 21:48
To: squid-users@lists.squid-cache.org
Subject: [squid-users] Squid doesn't execute url_rewrite_program 
/usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

 

Dear, I have Squid 3.5.23 and I use Squidguard for URL and domain filtering.

 

In squid.conf I have this line:

url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

 

but in this proxy server, the line is not executed by Squid, so Squidguard 
doesn't work at all.

 

Same configuration in another proxy server works OK.

 

Please can you tell me how I can force the execution of url_rewrite_program 
line ???

 

Thanks a lot !!!

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Roberto Carna]

Dear Amos, thanks for your comments.

I realized that I have some clues in cache.log:

2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/20 'squidGuard'
processes
2019/02/01 15:51:44 kid1| helperOpenServers: No 'squidGuard' processes
needed.
2019/02/01 15:51:44 kid1| helperOpenServers: Starting 0/5 'squid_ldap_auth'
processes
2019/02/01 15:51:44 kid1| helperOpenServers: No 'squid_ldap_auth' processes
needed.

These lines appears after I execute "systemctl reload squid".

Users and rights are OK.

Please can you help me one more time?

Because I have compared squid.conf and squidGuard.conf between this server
and the other running OK, and both files are similar.

Thanking in advance.

Robert



El vie., 1 feb. 2019 a las 3:45, Amos Jeffries ()
escribió:

> On 1/02/19 8:48 am, Roberto Carna wrote:
> > Dear, I have Squid 3.5.23 and I use Squidguard for URL and domain
> filtering.
> >
> > In squid.conf I have this line:
> >
> > url_rewrite_program /usr/bin/squidGuard -c
> /etc/squidguard/squidGuard.conf
> >
> > but in this proxy server, the line is not executed by Squid, so
> > Squidguard doesn't work at all.
> >
> > Same configuration in another proxy server works OK.
> >
> > Please can you tell me how I can force the execution of
> > url_rewrite_program line ???
>
>
> If the helper is not even being started:
>
> Check cache.log
>
> Check that the Squid low-privileges user account is allowed to run that
> helper.
>
> Check that there are not other copies of the line replacing the helper
> with another later in the config. That includes the
> backward-compatibility alias of this directive: redirector_program.
>
> Check what startup=N option to the url_rewrite_children (and alias
> redirector_children) are using. If it is set to '0' the helper will not
> be started until it is necessary to handle a URL.
>
>
> If the helper is starting but crashing or exiting immediately (see
> cache.log):
>
> Check that your version of SquidGuard has been patched to comply with
> the Squid-3.4+ helper protocol.
>
> Check that the Squid low-privileges user account is allowed to run that
> helper.
>
>
> If the helper is running but appears not to be doing anything:
>
> Check your url_rewrite_access lines (and alias redirector_access) to
> ensure that the traffic you want to re-write is allowed to be passed to
> the helper.
>
>
> PS. Please consider using ufdbguard instead of SquidGuard which has not
> been maintained in many years.
>
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid doesn't execute url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf

[Amos Jeffries]

On 1/02/19 8:48 am, Roberto Carna wrote:
> Dear, I have Squid 3.5.23 and I use Squidguard for URL and domain filtering.
> 
> In squid.conf I have this line:
> 
> url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf
> 
> but in this proxy server, the line is not executed by Squid, so
> Squidguard doesn't work at all.
> 
> Same configuration in another proxy server works OK.
> 
> Please can you tell me how I can force the execution of
> url_rewrite_program line ???


If the helper is not even being started:

Check cache.log

Check that the Squid low-privileges user account is allowed to run that
helper.

Check that there are not other copies of the line replacing the helper
with another later in the config. That includes the
backward-compatibility alias of this directive: redirector_program.

Check what startup=N option to the url_rewrite_children (and alias
redirector_children) are using. If it is set to '0' the helper will not
be started until it is necessary to handle a URL.


If the helper is starting but crashing or exiting immediately (see
cache.log):

Check that your version of SquidGuard has been patched to comply with
the Squid-3.4+ helper protocol.

Check that the Squid low-privileges user account is allowed to run that
helper.


If the helper is running but appears not to be doing anything:

Check your url_rewrite_access lines (and alias redirector_access) to
ensure that the traffic you want to re-write is allowed to be passed to
the helper.


PS. Please consider using ufdbguard instead of SquidGuard which has not
been maintained in many years.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users