Re: [squid-users] logformat odd values
On 9/14/21 3:04 PM, Moti Berger wrote:
> I have the followings in squid.conf:
>
> logformat metrics %icap::tt %adapt::all_trs %adapt::sum_trs
> %{service_req_a}adapt::sum_trs %{service_resp_a}adapt::sum_trs
> %{service_req_b}adapt::sum_trs %{service_resp_b}adapt::sum_trs
> access_log daemon:/var/log/squid/metrics.log metrics
>
>
>
> icap_service service_req_a reqmod_precache bypass=1 on-overload=wait
> routing=1 icap://a.y:12345/request
> icap_service service_req_b reqmod_precache bypass=1 on-overload=wait
> icap://b.y:10101/request
> adaptation_service_chain svcRequest service_req_a service_req_b
> adaptation_access svcRequest deny manager
> adaptation_access svcRequest allow all
> icap_service service_resp_a respmod_precache bypass=1
> on-overload=wait routing=1 icap://a.y:12345/response
> icap_service service_resp_b respmod_precache bypass=1
> on-overload=wait icap://b.y:10101/response
> adaptation_service_chain svcResponse service_resp_a service_resp_b
> adaptation_access svcResponse deny manager
> adaptation_access svcResponse allow all
>
>
> I see in metrics.log lines like this:
>
> 4 4,180 4,180 4 180 - -
>
>
> Now I wonder how come the value of %icap:tt isn't at least as the sum of
> all the numbers appear on %adapt::all_trs or %adapt::sum_trs (assuming
> no failed transactions)?
There is probably a bug somewhere, but please note that %icap:tt may not
be the sum of individual transaction response times (in _some_ cases)
even after that bug is fixed because those individual transactions may
run _concurrently_ (i.e. partially overlap in time).
> If %icap:tt isn't at least the sum of all ICAPs processing time, what is?
Bugs notwithstanding, it is approximate time a master transaction spent
doing adaptation (including checking whether adaptation is necessary).
This stopwatch ticks when adaptation_access ACLs are checked and also
when at least one adaptation transaction associated with that master
transaction is in progress.
Please note that a master transaction can do a lot of different things
at once or in parallel. For example, it can communicate with an HTTP
client while communicating with an FTP server while communicating with
an eCAP REQMOD adaptation service while communicating with a DNS server
to decide whether to start communicating with an ICAP RESPMOD service.
HTH,
Alex.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Compile/Rebuilding on debian bullseye (or buster)
On 15/09/21 10:28 pm, L.P.H. van Belle wrote: Hai Amos, Thanks on the reply, ive missed the change from db to tdb, thanks on that. What i notice in the builds is, I see this one.. config.status: creating test-suite/Makefile And then i see these, then it failes. cp ../../src/tes ts/stub_fd.cc tests/stub_fd.cc cp: cannot create regular file 'tests/stub_fd.cc': No such file or directory make[4]: *** [Makefile:1445: tests/stub_fd.cc] Error 1 Is it possible that the "tests" folder (keep in mind above uses the old /debian setup), is changed to "test-suite".. No, the folders are not related. The issue is that without dependency tracking make does not detect that tools/squidclient/tests/ directory needs to exist before stub_fd.cc is copied to build tools/squidclient/tests/stub_fd.o Im not really that into the code but, it "looks" like there is a folder missing "tests" Based on above, if can disable all tests, it should build. Suggestion on disabling the tests? I have spent a while working on it today and have pushed an update to Debian packaging repo. Please pull a new copy of that latest. It should fix all the issues you have. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Compile/Rebuilding on debian bullseye (or buster)
Hai Amos, Thanks on the reply, ive missed the change from db to tdb, thanks on that. What i notice in the builds is, I see this one.. config.status: creating test-suite/Makefile And then i see these, then it failes. cp ../../src/tests/stub_fd.cc tests/stub_fd.cc cp: cannot create regular file 'tests/stub_fd.cc': No such file or directory make[4]: *** [Makefile:1445: tests/stub_fd.cc] Error 1 Is it possible that the "tests" folder (keep in mind above uses the old /debian setup), is changed to "test-suite".. Im not really that into the code but, it "looks" like there is a folder missing "tests" Based on above, if can disable all tests, it should build. Suggestion on disabling the tests? Greetz, Louis > -Oorspronkelijk bericht- > Van: squid-users > [mailto:squid-users-boun...@lists.squid-cache.org] Namens > Amos Jeffries > Verzonden: woensdag 15 september 2021 1:14 > Aan: squid-users@lists.squid-cache.org > Onderwerp: Re: [squid-users] Compile/Rebuilding on debian > bullseye (or buster) > > On 14/09/21 9:22 pm, L.P.H. van Belle wrote: > > Hai Amos, > > > > Im attempting to make a squid 5.1 build based on the > bullseye squid/debian folder. > > ( ps. Im building with sbuilder ) > > > > Now, this "normaly" worked since squid 3.2 for me, copy the > debian folder, make minor adjustments if needed, > > Just with latest adjustments, well, i cant make it work. > > > > This was my last adjustmated.. > > > >* Used build : squid-5.1-20210804-r1f9e52827 of 04 Aug 2021 > >* Refreshed patches, removed patches already included. > >* d/control lower debhelper to 12 to allow building > > > > > > Changed d/rules, added. -srcdir=. --disable-dependency-tracking > > Any suggestions what i can do here? > > I am a bit stuck here myself with this build style, that is > part of why > the package is not already updated. The Debian auto-build system adds > those options and AFAICT does not provide an easy way to avoid. > > > > > > And i tested also with my last settings from my backport to > Debian buster and squid 4.16(ssl enabled) > > > > Resulting in : > > configure: Samba TrivialDB library support: no > > configure: error: external acl helper time_quota ... found > but cannot be built > > make: *** [/usr/share/cdbs/1/class/autotools.mk:46: > debian/stamp-autotools] Error 1 > > dpkg-buildpackage: error: debian/rules build subprocess > returned exit status 2 > > > -- > -- > > Debian used to use the BerkeleyDB, so the normal Squid build > dependencies pulls the library in for that. > > If libdb-dev is no longer available you will need to install > libtdb-dev > package. > > Also, (for now) if you are using the ext_time_quote_acl or > ext_session_acl helpers you will need to manually purge their > databases > on install/upgrade. > > Amos > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users > > ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] [squid-announce] Squid 4.16 is available
Hey Amos and Alex, I have tested the 4.16 version and it seems to work steady on basic loads. Eliezer -Original Message- From: squid-announce On Behalf Of Amos Jeffries Sent: Thursday, July 22, 2021 7:24 AM To: squid-annou...@lists.squid-cache.org Subject: [squid-announce] Squid 4.16 is available The Squid HTTP Proxy team is very pleased to announce the availability of the Squid-4.16 release! This release is a bug fix release resolving several issues found in the prior Squid releases. The major changes to be aware of since 4.15: * Regression Fix: --with-valgrind-debug build Squid-4.15 changes caused a build failure linking with valgrind memory tracking tool. This release fixes that to allow memory leak tracing again. * Bug 4528: ICAP transactions quit on async DNS lookups Squid has never reliably been able to resolve hostnames configured for ICAP services. They might work most of the time when added to /etc/hosts, but not always - and would rarely work if relying on remote DNS servers. This release adds full support for DNS remote resolution of service names in icap_service directive. Regardless of where the hostname is resolved from it can now be expected to resolve and also properly obey DNS TTL expiry for IP address changes. * Bug 5128: Translation: Fix '% i' typo in es/ERR_FORWARDING_DENIED Spanish translation of the ERR_FORWARDING_DENIED template have for some time omitted the URL which was having issues being fetched. The template published with this release and current squid-langpack downloads will now display the URL identically to other error pages. All users of Squid are encouraged to upgrade as soon as possible. See the ChangeLog for the full list of changes in this and earlier releases. Please refer to the release notes at http://www.squid-cache.org/Versions/v4/RELEASENOTES.html when you are ready to make the switch to Squid-4 This new release can be downloaded from our HTTP or FTP servers http://www.squid-cache.org/Versions/v4/ ftp://ftp.squid-cache.org/pub/squid/ ftp://ftp.squid-cache.org/pub/archive/4/ or the mirrors. For a list of mirror sites see http://www.squid-cache.org/Download/http-mirrors.html http://www.squid-cache.org/Download/mirrors.html If you encounter any issues with this release please file a bug report. http://bugs.squid-cache.org/ Amos Jeffries ___ squid-announce mailing list squid-annou...@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-announce ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] squid 5.1/Debian WARNING: no_suid: setuid(0): (1) Operation not permitted
Many thanks It fix the issue ! Le 15/09/2021 à 13:08, Graham Wharton a écrit : You see this when starting as non rootuser. Squid should be started as root and then it changes identity to cache effective user as defined in config when it forks. Graham Wharton Lube Finder Tel (UK) : 0800 955 0922 Tel (Intl) : +44 1305 898033 https://www.lubefinder.com *From:* squid-users on behalf of David Touzeau *Sent:* Wednesday, September 15, 2021 11:40:04 AM *To:* squid-users@lists.squid-cache.org *Subject:* [squid-users] squid 5.1/Debian WARNING: no_suid: setuid(0): (1) Operation not permitted On Debian 10 64bits with squid 5.1 we have thousand warning as this: 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted When squid try to load external acls binaries add chmod 04755 in binaries did not resolve the issue. No issue with same configuration with squid 3.5x branch Any tips ? ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] squid 5.1/Debian WARNING: no_suid: setuid(0): (1) Operation not permitted
How do you build and start it, init.d/squid or systemd start squid
In case of last, what i suspect, I seen more if these messages on previous
version..
But all my version dont show this on Debian 10.
This is my latest startup for systemd
# /lib/systemd/system/squid.service
## Copyright (C) 1996-2021 The Squid Software Foundation and contributors
##
## Squid software is distributed under GPLv2+ license and includes
## contributions from numerous individuals and organizations.
## Please see the COPYING and CONTRIBUTORS files for details.
##
[Unit]
Description=Squid Web Proxy Server
Documentation=man:squid(8)
After=network.target network-online.target nss-lookup.target
[Service]
Type=notify
PIDFile=/run/squid.pid
ExecStartPre=/usr/sbin/squid --foreground -z
ExecStart=/usr/sbin/squid --foreground -sYC
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
NotifyAccess=all
[Install]
WantedBy=multi-user.target
---
these are the settings from a debian (own) build setup with squid 4.16. ( with
ssl enabled)
squid -v
Squid Cache: Version 4.16
Service Name: squid
Debian linux
This binary uses OpenSSL 1.1.1d 10 Sep 2019. For legal restrictions on
distribution see https://www.openssl.org/source/license.html
configure options: '--build=x86_64-linux-gnu' '--prefix=/usr'
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
'--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var'
'--libexecdir=${prefix}/lib/squid' '--srcdir=.' '--disable-maintainer-mode'
'--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2
-fdebug-prefix-map=/build/squid-4.16=. -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wl,-z,now
-Wl,--as-needed' 'BUILDCXX=x86_64-linux-gnu-g++'
'--with-build-environment=default' '--enable-build-info=Debian linux'
'--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
'--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline'
'--disable-arch-native' '--enable-async-io=8'
'--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap'
'--enable-delay-pools' '--enable-cache-digests' '--enable-ica
p-client' '--enable-follow-x-forwarded-for'
'--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB'
'--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper'
'--enable-auth-ntlm=fake,SMB_LM'
'--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group'
'--enable-security-cert-validators=fake'
'--enable-storeid-rewrite-helpers=file' '--enable-url-rewrite-helpers=fake'
'--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos'
'--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy'
'--with-gnutls' '--enable-ssl' '--enable-ssl-crtd' '--with-openssl'
'--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu'
'CC=x86_64-linux-gnu-gcc' 'CFLAGS=-g -O2 -fdebug-prefix-map=/buil
d/squid-4.16=. -fstack-protector-strong -Wformat -Werror=format-security
-Wall' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed' 'CPPFLAGS=-Wdate-time
-D_FORTIFY_SOURCE=2' 'CXX=x86_64-linux-gnu-g++' 'CXXFLAGS=-g -O2
-fdebug-prefix-map=/build/squid-4.16=. -fstack-protector-strong -Wformat
-Werror=format-security'
look if you also see : '--with-default-user=proxy'
and if its self compiled.
sudo adduser --system proxy
And when thats done verify the needed folders and there rights/ownerships.
The "debian" folder if you want it have a look of what i use currently in
production.
https://apt.van-belle.nl/debian/pool/main/s/squid/squid_4.16-0.1ssl1buster1.debian.tar.xz
As soon i can make debian packages of 5.1, im making a buster and bullseye
version.
I hope this helps you a bit.
Greetz,
Louis
Van: squid-users [mailto:squid-users-boun...@lists.squid-cache.org]
Namens David Touzeau
Verzonden: woensdag 15 september 2021 12:40
Aan: squid-users@lists.squid-cache.org
Onderwerp: [squid-users] squid 5.1/Debian WARNING: no_suid: setuid(0):
(1) Operation not permitted
On Debian 10 64bits with squid 5.1 we have thousand warning as this:
2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation
not permitted
2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation
not permitted
2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation
not permitted
2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation
not permitted
2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation
not permitted
[squid-users] squid 5.1/Debian WARNING: no_suid: setuid(0): (1) Operation not permitted
On Debian 10 64bits with squid 5.1 we have thousand warning as this: 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid2| WARNING: no_suid: setuid(0): (1) Operation not permitted 2021/09/15 08:00:18 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted When squid try to load external acls binaries add chmod 04755 in binaries did not resolve the issue. No issue with same configuration with squid 3.5x branch Any tips ? ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] SSL Terminating Reverse Proxy with Referral Tracking
On 15/09/21 1:21 pm, Grant Taylor wrote: On 9/14/21 6:09 PM, Amos Jeffries wrote: b) If those upstream servers are embedding URLs for clients to directly contact the XaaS services. Then your desire is not possible without redesigning the upstream service(s) such that they stop exposing their use of the XaaS. Which often also means redesigning the XaaS service itself too. I don't know about Squid, but I do know that it's possible to manipulate traffic with Apache in a similar role. I've done so a number of times using the mod_proxy and associated mod_proxy_html modules. This allows Apache to re-write content as it's passing through the Apache proxy. I wonder if Squid's ICAP support might allow something to modify traffic as it passes through the Squid proxy. Squid also has ability to pass traffic to adaptors. That is quite a complicated system, the OP is looking for something elegant (aka simple). That is not possible for a reverse-proxy to do. It will never see the third-party traffic, as mentioned by (b) above. Sure it is. }:-) I am talking here about traffic that does not go near a reverse-proxy. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
