Hi Andre,
I did not receive your new reply, but I did find it in the mailing
list archives in time to download the logs. Thank you for sharing them!
FWIW, you may want to CC me in future emails (but I do not know whether
that will help).
Something really strange is going on. Squid runtime logs (both versions)
do not match the two identical "squid -k parse" logs you have shared
earlier. For example, at runtime, Squid v5 checks the following rules,
in this order:
1. http_access ... ArticaTSTSite ...
2. http_access ... NoCacheMGR ...
3. http_access ... LocalClient ...
4. http_access ... MyIPSrc ...
5. http_access ... internal_icons ...
6. http_access ... MgrInfoUri ...
The first 10 rules in "squid -k parse" output are different! I pasted
them below and marked those that are missing in runtime output:
1. http_access deny ArticaTSTSite ArticaTSTSiteAll
2. http_access allow NoCacheMGR ToArticaWWW
3. http_access allow LocalClient NoCacheMGR
4. http_access allow MyIPSrc NoCacheMGR
5. http_access allow internal_icons
6. Missing: http_access allow internal_icons
7. Missing: http_access deny NormalPorts to_localhost CVEFix all
8. Missing: http_access deny NormalPorts manager ManagerDeny all
9. Missing: http_access allow MgRPort manager all
10. http_access allow MgRPort MgrInfoUri all
The differences continue...
N.B. Rules 5 and 6 in parsing logs are the same, but Squid does not know
that (yet?). Squid will parse, consume, and evaluate both (if they are
reached and do not match). In runtime logs, Squid v5 only evaluates the
first one (#5) and then goes to a MgrInfoUri rule that is listed as #6
in Squid v5 runtime logs and as #10 in parse logs, skipping four rules.
Runtime logs do not contain just the ACL names. They also contain rule
numbers assigned at parsing time. The number should be incremented for
every http_access rule Squid parses. Those numbers do not match "squid
-k parse" output, reflecting the same "missing rules" problem, and
elevating my confidence that other rules are missing at runtime.
Squid v6 runtime logs show the same problem, but the differences start
later (and are more profound). For example, Squid v6 logs do not show
Group212 ACL being evaluated (rule #151 in parse logs) but do show
AnnotateFinalAllow ACL evaluated (rule #168 in parse logs and rule #43
in v6 runtime logs). Group98 (rule #116 in parse logs) is also missing
from v6 runtime logs.
Parsing logs show 168 http_access rules.
Running Squid v5 appears to have 163 http_access rules.
Running Squid v6 appears to have 43 http_access rules!
Bugs notwithstanding, Squid does not "skip" http_access rules that it
successfully parsed/consumed at startup or reconfiguration. Squid does
not evaluate higher-numbered rules after a rule matches, but that is not
what it happening here: For example, I see sequences where Squid v6
checks 43 rules from the first (in all logs) ArticaTSTSite rule to the
last (in all logs) AnnotateFinalAllow rule; the other 125 rules (seen in
"squid -k parse" logs) appear to be "missing" from v6 runtime logs.
I speculate that one of these things have happened (in no particular order):
A. I am confused by these partial logs.
B. Squid failed to parse/consume some http_access rules in your
configuration files. The set of rules Squid failed to parse differs
across v5 and v6. The logs do not detail parsing, so I cannot validate
this theory, but it would explain the missing rules. In the vast
majority of cases, there ought to be at least some level-0/1 cache.log
errors or warnings about failing to parse/consume an http_access rule!
One way to validate this theory is to compare "squid -k parse" output
with http_access lines in mgr:config cache manager report received from
a running Squid instance. You may need to modify squid.conf to allow
that cache manager query (see cachemgr_passwd). Squids, especially older
ones, have various bugs in mgr:config output so a manual comparison may
be needed.
Another way is to study "squid -k parse -X -d9" output that should have
more details for Squid v6. This is similar to the trick in (D) below,
but without runtime debugging.
C. Your "squid -k parse" commands do not match the actual configurations
that Squids are parsing and then using at runtime. The same mgr:config
trick from (B) above can be used to validate this theory.
D. Both Squids are suffering from some serious bug. The effects of that
bug result in different actual/used http_access rule sets in each version.
One way to confirm theory D is to study ALL,9 logs collected from Squid
v6 with "squid ... -X -d9" command line option (or a similar trick that
will include initial parsing in the log). The logs should detail at
least one transaction that should evaluate most http_access rules. Such
logs will address concern (A) as well, but you will probably have to
share them privately if you are using production configuration/instance.
HTH,
Alex.
On 2024-04-15 19:49, Andre Bolinhas wrote:
Hi Alex,
Thnks for your reply.
Logs uploaded again, you can find it here.
https://we.tl/t-QiSKMgclOb
Best regards
On 15/04/2024 14:12, Alex Rousskov wrote:
On 2024-04-14 17:23, Andre Bolinhas wrote:
Any tip on this matter? I want to upgrade to squid 6.9 but due to
this issue, i'm stuck.
Hi Andre,
Please note that I did _not_ receive your email quoted below. It
is in the email archive, so the problem is not on your end, but I just
wanted to mention that I was not (knowingly) ignoring you.
> I have re-uploaded the cache.log files.
The files have expired again. I have reviewed the diff you shared, but
cannot make further progress without those test logs. Hopefully, your
next list post reaches me.
Alex.
On 01/04/2024 11:53, Andre Bolinhas wrote:
Hi Alex
Thanks for your help on the matter.
The logs archive you shared previously has expired, so I cannot
double check, but from what I remember, the shared logs did not
support the above assertion, so there may be more to the story
here. However, to make progress, let's assume that v5 configuration
files are identical to v6 configuration files.
If you want, I can run the same test with in a different debug
parameters, just tell which ones.
I have re-uploaded the cache.log files.
https://we.tl/t-AB4XuUwuf7
One way to answer all of the above questions is to look at the
following output:
squid -k parse ... |& grep Processing:.http_access
There is no diff between both squid version, you can check it here
DiffNow - Compare Files, URLs, and Clipboard Contents Online
<https://www.diffnow.com/report/jsrva>
The logs archive you shared previously has expired, so I cannot
double check, but from what I remember, the shared logs did not
support the above assertion, so there may be more to the story
here. However, to make progress, let's assume that v5 configuration
files are identical to v6 configuration files.
The configuration files / folder are the same, the server is the
same, the only thing that changes is the Squid version
On 29/03/2024 17:40, Alex Rousskov wrote:
On 2024-03-25 15:13, Bolinhas André wrote:
Yes, the configuration is the same for both versions.
The logs archive you shared previously has expired, so I cannot
double check, but from what I remember, the shared logs did not
support the above assertion, so there may be more to the story
here. However, to make progress, let's assume that v5 configuration
files are identical to v6 configuration files.
1. Is there an "http_access allow all AnnotateFinalAllow" rule?
2. Is there an "http_access deny HTTP Group38 AnnotateRule28" rule?
3. Assuming the answers are "yes" and "yes", which rule comes
first? If you use include files, this question applies to the
imaginary preprocessed squid.conf file with all the include files
inlined (recursively if needed). That kind of preprocessed
configuration is what Squid effectively sees when compiling
http_access rules, one by one. Which of the two rules will Squid
see first?
One way to answer all of the above questions is to look at the
following output:
squid -k parse ... |& grep Processing:.http_access
Replace "..." with your regular squid startup command line options
and adjust standard error redirection (|&) as needed for your
shell. Run the above command for both Squid v5 and v6 binaries. You
should see output like this:
2024/03/29 13:31:05| Processing: http_access allow manager
2024/03/29 13:31:05| Processing: http_access deny all
HTH,
Alex.
------------------------------------------------------------------------
*De:* Alex Rousskov <rouss...@measurement-factory.com>
*Enviado:* segunda-feira, 25 de março de 2024 19:12
*Para:* squid-users@lists.squid-cache.org
*Assunto* Re: [squid-users] ACL / http_access rules stop work
using Squid 6+
On 2024-03-22 09:38, Andre Bolinhas wrote:
> In previous versions of squid, from 3 to 5.9, I use this kind
of deny
> rules and they work like charm
>
> acl AnnotateRule28 annotate_transaction accessrule=Rule28
> http_access deny HTTP Group38 AnnotateRule28
>
> This allows me to deny objects without bump / show the error page
> (deny_info)
>
> But using squid 6+ this rules stop to work and everything is
allowed.
>
> Example:
> Squid 5.9 (OK)
> https://ibb.co/YdKgL1Y
>
> Squid 6.8 (NOK)
> https://ibb.co/tbyY2GV
>
> Sample of both cache.log in debug mode
>
> https://we.tl/t-T7Nz1rVbVu
In you v6 logs, most logged transactions are allowed because a rule
similar to the one reconstructed below is matching:
http_access allow all AnnotateFinalAllow
There are similar cases in v5 logs as well, but most denied v5
transactions match the following rule instead (i.e. the one you
shared
above):
http_access deny HTTP Group38 AnnotateRule28
In your Squid configuration, v6 allow rule is listed much higher
than v5
deny rule (#43 vs #149). I do not see any signs of Group38 or
AnnotateRule28 ACL evaluation in v6 logs, as if the rule sets are
different for two different Squid instances. Are you using the
same set
of http_access rules for both Squid versions?
Alex.
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-users
