Re: [squid-users] ext_ldap_group_acl is returned ERR when LDAP bind was fail.

[asakura]

Hello,

I posted a question last month below. However, I haven't receivedany replies.
Does anyone want to respond to this?

Thanks in advance for any comments you might have.

I investigating source code of ext_ldap_group_acl.cc below.

helpers/external_acl/LDAP_group/ext_ldap_group_acl.cc
571 rc = ldap_simple_bind_s(ld, binddn, bindpasswd);
572 if (rc != LDAP_SUCCESS) {
573 fprintf(stderr, PROGRAM_NAME ": WARNING: could not 
bind to binddn '%s'\n", ldap_err2string(rc));
574 ldap_unbind(ld);
575 ld = NULL;
576 break;
577 }
snip...

593 if (found)
594 SEND_OK("");
595 else {
596 SEND_ERR("");
597 }

Regards,
Kazuhiro

From: asak...@ioc.dnp.co.jp
Subject: [squid-users] ext_ldap_group_acl is returned ERR when LDAP bind was 
fail.
Date: Wed, 23 Mar 2016 15:08:50 +0900 (JST)

> Hello,
> 
> Thank you always for your kind support.
> 
> I would like to ask you about SEND_ERR reply of ext_ldap_group_acl.
> In our environment, squid fail ldap_bind to LDAP server sometimes.
> Then, ext_ldap_group_acl replies "ERR". So, username is registered
> in the negative_cache.
> 
> I don't want to register in the negative_cache when external_acl
> failed ldap_bind.
> I guess that to solve if ext_ldap_group_acl reply SEND_BH instead of
> SEND_ERR.
> 
> I would appreciate it if you could investigate this.
> 
> Regards,
> Kazuhiro
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] query ?

[Luis Abdon Diaz Parra]

Hello, good day to ask quisera
there any way atra instead of a domain allows me to upload all your files
I mean a group of users only allow you to access a specific domain dstdomain 
say "facebook.com" but does not load your images that refer to other URLs and 
domains

Luis Abdon Diaz ParraMovil: 951159908“La posibilidad de realizar un sueño es lo 
que hace que la vida sea interesante”.
  ___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Alex Rousskov]

On 04/07/2016 12:21 PM, Odhiambo Washington wrote:
> 
> 
> On 7 April 2016 at 19:35, Alex Rousskov
>  > wrote:
> 
> On 04/07/2016 08:21 AM, Odhiambo Washington wrote:
> 
> > On 7 April 2016 at 17:16, Amos Jeffries wrote:
> >
> > On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
> > > I am getting the following error in cache.log:
> > >
> > > Squid Cache (Version 3.5.16): Terminated abnormally.
> > > FATAL: Ipc::Mem::Segment::create failed to
> > > shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) 
> File exists
> > >
> > > However, that file doesn't exist.
> 
> > This can happen if you have a startup script the runs 'squid -z' or
> > similar just prior to starting the main proxy, and not waiting
> > sufficiently long for the -z run to finish.
> 
> 
> > I am gonna check this out again tonight although I doubt if that is the
> > cause.
> >
> > I run squid using daemontools, invoked as:
> >
> > exec setuidgid root /opt/squid-3.5/sbin/squid -f
> > /opt/squid-3.5/etc/squid.conf -N
> >
> > /opt/squid-3.5/var/run/squid/ is actually empty when I get this error.
> 
> 
> I see two possibilities:
> 
> 1. The file was there at the time the error was triggered but was not
> there at the time you checked the directory. This would mean that
> something is starting a second Squid while the first Squid has not
> removed the shared memory segment file (yet). Amos mentioned one such
> common scenario (not waiting for background squid-z) but there are
> others, possibly including handling of Squid crashes. Do you see any
> other errors, assertions, or FATAL messages in your cache.log?
> 
> 2. Squid code that is trying to open the shared segment is broken or,
> more likely, not compatible with your FreeBSD environment. For example,
> it tries to exclusively create a shared segment using the wrong name.
> 
> If you can reproduce this, I recommend starting Squid via strace (or
> equivalent) to see the system calls that Squid is making when calling
> shm_open() and the exact call parameters. This can confirm or eliminate
> #2 as the suspect.


3. The error reported by Squid is bogus. Squid has lots of code that
corrupts/overwrites the error number before actually reporting the
system call error...



> All I get from running strace -ff -vvv -o /tmp/squid-strace.txt
> /opt/squid-3.5/sbin/squid -f /opt/squid-3.5/etc/squid.conf:
> ..
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> pread: Device busy
> PIOCRUN: Input/output error
> trouble opening proc file 

The above does not look like strace output I am used to [on Linux], but
I cannot help you with making strace work. There may be a better tool on
FreeBSD. Hopefully, somebody else will tell you how to get the necessary
info on FreeBSD.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
https://vgy.me/GnhuDD.png

07.04.16 20:16, Amos Jeffries пишет:
> On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
>> I am getting the following error in cache.log:
>>
>> Squid Cache (Version 3.5.16): Terminated abnormally.
>> CPU Usage: 0.082 seconds = 0.052 user + 0.030 sys
>> Maximum Resident Size: 54992 KB
>> Page faults with physical i/o: 0
>> FATAL: Ipc::Mem::Segment::create failed to
>> shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) File exists
>>
>>
>> However, that file doesn't exist.
>>
>
> This can happen if you have a startup script the runs 'squid -z' or
> similar just prior to starting the main proxy, and not waiting
> sufficiently long for the -z run to finish.
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXBrIDAAoJENNXIZxhPexGhukH/i2RaUjjkbK5XQ7pgBmvfJgQ
+SktSTKKpRWNMqferypXWfoLr44ojXLELj6KB/cHTfY1fhCmKQy0pUSQxsiB8QFp
Yz3R8xxw/2rJPJDcZKQzxazDalVs7ihfTUAcMs8gA9FaebjOIs1kPNjy99t0Veow
QL0dZe3GYoL6eIaK+jeo6cRwckGsp6009s92icXTrGasIiDELMlBwGLz8VrdTjBP
QppjdSlV2EZA5q+8+WMvHxsWbJ7oZocnO+YW8LfunoJfmkC0kh0WUjORZjUiQ1/R
9MA69WmhdREKtnMmB2aKyuYtnSYPy4UXZRXlD/a9i9faOdVQR6hthCLNWDe3KtU=
=tD45
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
http://bugs.squid-cache.org/show_bug.cgi?id=4486

07.04.16 20:16, Amos Jeffries пишет:
> On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
>> I am getting the following error in cache.log:
>>
>> Squid Cache (Version 3.5.16): Terminated abnormally.
>> CPU Usage: 0.082 seconds = 0.052 user + 0.030 sys
>> Maximum Resident Size: 54992 KB
>> Page faults with physical i/o: 0
>> FATAL: Ipc::Mem::Segment::create failed to
>> shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) File exists
>>
>>
>> However, that file doesn't exist.
>>
>
> This can happen if you have a startup script the runs 'squid -z' or
> similar just prior to starting the main proxy, and not waiting
> sufficiently long for the -z run to finish.
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXBrHTAAoJENNXIZxhPexGyn8IALAK0K2WLF3NfRX/fszUr/X/
6syPQWbGXUkw/ktauWB4HqegDEoDHMdRI5+EDUKNS8eFdfMgC8HY0EOVMowsw2RL
6hyKSwWzmVV0p+OzC77dzeAPC2MqlLa5kb8yCTHC1ZDtQv5ZJmgaHsMixzTHCCfj
fAme3vRG/HCJnQ4BbdybTz1XhYduB8aF91cRTQtQyGCYhkuRKYdVWpxDn1t2/+3D
lQrqymKFS34C5eqcm1HiaIoXLdiPUUcmLyY8QjXfaAQbrRR6yKLa613pjU6XFYP+
Aj7WthR4zHJgGPx65QLyEyHvyzAdgYCbjhRnyT7K4yonFvrEBC4a3DV6tePEzZg=
=+50D
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Odhiambo Washington]

On 7 April 2016 at 19:35, Alex Rousskov 
wrote:

> On 04/07/2016 08:21 AM, Odhiambo Washington wrote:
>
> > On 7 April 2016 at 17:16, Amos Jeffries wrote:
> >
> > On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
> > > I am getting the following error in cache.log:
> > >
> > > Squid Cache (Version 3.5.16): Terminated abnormally.
> > > FATAL: Ipc::Mem::Segment::create failed to
> > > shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) File
> exists
> > >
> > > However, that file doesn't exist.
>
> > This can happen if you have a startup script the runs 'squid -z' or
> > similar just prior to starting the main proxy, and not waiting
> > sufficiently long for the -z run to finish.
>
>
> > I am gonna check this out again tonight although I doubt if that is the
> > cause.
> >
> > I run squid using daemontools, invoked as:
> >
> > exec setuidgid root /opt/squid-3.5/sbin/squid -f
> > /opt/squid-3.5/etc/squid.conf -N
> >
> > /opt/squid-3.5/var/run/squid/ is actually empty when I get this error.
>
>
> I see two possibilities:
>
> 1. The file was there at the time the error was triggered but was not
> there at the time you checked the directory. This would mean that
> something is starting a second Squid while the first Squid has not
> removed the shared memory segment file (yet). Amos mentioned one such
> common scenario (not waiting for background squid-z) but there are
> others, possibly including handling of Squid crashes. Do you see any
> other errors, assertions, or FATAL messages in your cache.log?
>
> 2. Squid code that is trying to open the shared segment is broken or,
> more likely, not compatible with your FreeBSD environment. For example,
> it tries to exclusively create a shared segment using the wrong name.
>
> If you can reproduce this, I recommend starting Squid via strace (or
> equivalent) to see the system calls that Squid is making when calling
> shm_open() and the exact call parameters. This can confirm or eliminate
> #2 as the suspect.
>
>
> HTH,
>
> Alex.
>

All I get from running strace -ff -vvv -o /tmp/squid-strace.txt
/opt/squid-3.5/sbin/squid -f /opt/squid-3.5/etc/squid.conf:
..
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
pread: Device busy
PIOCRUN: Input/output error
trouble opening proc file



-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Sending intermediate certificate with SSL-Bumped Certificate. (V3.5.1516-3-2-r14000)

[Jok Thuau]

with 3.5.15, I have this config:

---8<---
https_port 8443 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=64MB \
cert=/etc/squid/ssl/proxy.pem \
key=/etc/squid/ssl/proxy.key \
cafile=/etc/squid/ssl/proxy.pem
--->8---

proxy.pem is the concatenation of both the CA cert (intermediate) followed
by the root cert (my offline CA). Best i can tell, all of it is sent back
to the client (generated cert, intermediate and root CA).

HTH
Jok




On Thu, Apr 7, 2016 at 10:59 AM, Amos Jeffries  wrote:

> On 7/04/2016 5:25 a.m., Nicolaas Hyatt wrote:
> > Amos,
> > Thanks for your quick response and your time. I have not yet messed with
> > 4.0. Is this something that may find its way into the 3.x stable branch
> > at some point?
> >
>
> Maybe. I am reliant on the guys doing OpenSSL code (aka. Christos) to
> test the backporting though. So it will depend on whether he thinks its
> important enough.
>
> I'm hopeful, but no guarantees.
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Sending intermediate certificate with SSL-Bumped Certificate. (V3.5.1516-3-2-r14000)

[Amos Jeffries]

On 7/04/2016 5:25 a.m., Nicolaas Hyatt wrote:
> Amos,
> Thanks for your quick response and your time. I have not yet messed with
> 4.0. Is this something that may find its way into the 3.x stable branch
> at some point?
> 

Maybe. I am reliant on the guys doing OpenSSL code (aka. Christos) to
test the backporting though. So it will depend on whether he thinks its
important enough.

I'm hopeful, but no guarantees.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.16 and vary loop objects (bug ?)

[Amos Jeffries]

On 8/04/2016 2:39 a.m., joe wrote:
> some of my post you miss reading or
> wen i ask for water  and i get water  same size same glas  but i was offered
> somthing else i was refuse that

In terms of the protocol, you cant refuse exactly. It is what was
responded with. The server/provider 'owns' the resource and is the
control authority about what should be provided to any request.


> and my friend ask for the same he should get same glas of water same size
> so what i was trying to enplane is
>  
> 
> i ask  for gzip  the the md5 vary calculate with string

Asking for gzip would be sending the header exact and full value of:
  Accept-Ecoding: gzip\r\n

... not the below multiple list of values:

> accept-encoding="gzip,%20deflate,%20sdch" 
> so the gzip dose not have %20  on front of it so we can use only that and

%20 is a space character when URL-encoded for adding to the URL hash key.

The client sending a request with that header value(s) is asking for
gzip OR deflate OR sdch encoding (in that order of preference). The
server decides whether one of those can be produced. And identity
encoding (unencoded) is guaranteed to always be acceptible.


> filter out those ,%20deflate,%20sdch
> it will be string accept-encoding="gzip" use  that in md5 calc only
> if other browser has in vary string accept-encoding="gzip,%20deflate" 
> without examle  %20sdch
> md5 calc use string accept-encoding="gzip,%20deflate"
> so we also filter out -- > ,%20deflate
> and keep only to use md5 with  string accept-encoding="gzip"
> 
> since  gzip dose not have %20 on front of it

What if the sdch was the type that server actually wanted to produce?
If you are using Google Docs sdch objects actually *are* more often than
not what gets sent. Its a type of patch/diff format for collaborative
tools updating an object in bits and pieces.

> 
> that will make a beter hit without any problem on all browser
> 

*Maybe*. You are free to do so if you like. We cannot do it in the
general case though for Squid has to work according to proper HTTP
requirements in networks other than yours (and there are several others
also wanting this same thing have configured their proxies to do it).


> as i says and i test the link i provide before   
> on firefox   first time after the cache ar empty  i get HIT  alwes just by
> using firefox
> until now perfect
> 
> but  wen i use chrome since it send and resive in very  string
> accept-encoding="gzip,%20deflate,%20sdch"
> the cached file ar deleted and resolt  = MISS  then it re creat new file
> with HIT alwes just by using chrome
> 
> if i return to firefox its  MISS  then file got purged from cache then its
> HIT again

Two things:

1) MISS does not mean any content was thrown away. It just means what
was already in the cache was not able to be used.
  - it can happen because of a max-age=0 or no-cache from the client.
Chrome and other Google products seem to like sending those. Very
unfriendly to caches.
  - it can happen if Vary exists and the particular object this request
needs is not one of the current set of variants in the cache.

2) You can have the appearance of stuff being "thrown away" if the Vary
marker object used to store the pattern actually was thrown away for
some reason. Without it there is no way to reach the variant objects
which are still in cache.
 - A MISS is needed in order to re-fetch a response with Vary pattern to
create the marker, and bingo all the variants that pattern describes
which were still in the cache can be fetched as HITs again.
 - BUT, if the Vary pattern changed they stay 'lost'.

This #2 is a common problem seen with Apache servers which emit
different Vary patterns depending on which modules were run on the request.


> 
> if other browser has  vary  with string
> accept-encoding="%20gzip,deflate,%20sdch"
> watch this  deflate   has no --->> %20 in front of it so md5 calc should
> use only deflate   not encluding the one with  %20   string
> this how i understand it   and i monitor those prossess i prove that is
> waste

Whitespace normalization (or lack of it rather) does cause wastage. The
only reason that is not done is nobody did it yet. Patches fixing that
are welcome.

> 
> so anoter example cause i dont know if im trying to explain it in right
> manner 
>  another browser get vary  string accept-encoding="%20gzip,%20deflate,sdch"
> as you see  sdch  its the one used  so we must use that filter out the rest 
> 
> variety with string confuse the correct match match lookup it it purge the
> same file just becaus somthing in vary has extra string and its not used
> 

A)
I know what you are trying to say. I'm trying to say why its not
necessarily such a great idea (for Squid). There are subtle things in
the protocol design that prevent it and/or would be broken by it.


B)
You are also focused on The Browser(s). Some of the smart guys in IETF
HTTP Working Group did some research a few months ago. Its a funny story
really:

* the Chrome guys did 

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Alex Rousskov]

On 04/07/2016 08:21 AM, Odhiambo Washington wrote:

> On 7 April 2016 at 17:16, Amos Jeffries wrote:
> 
> On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
> > I am getting the following error in cache.log:
> >
> > Squid Cache (Version 3.5.16): Terminated abnormally.
> > FATAL: Ipc::Mem::Segment::create failed to
> > shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) File 
> exists
> >
> > However, that file doesn't exist.

> This can happen if you have a startup script the runs 'squid -z' or
> similar just prior to starting the main proxy, and not waiting
> sufficiently long for the -z run to finish.


> I am gonna check this out again tonight although I doubt if that is the
> cause.
> 
> I run squid using daemontools, invoked as:
> 
> exec setuidgid root /opt/squid-3.5/sbin/squid -f
> /opt/squid-3.5/etc/squid.conf -N
> 
> /opt/squid-3.5/var/run/squid/ is actually empty when I get this error.


I see two possibilities:

1. The file was there at the time the error was triggered but was not
there at the time you checked the directory. This would mean that
something is starting a second Squid while the first Squid has not
removed the shared memory segment file (yet). Amos mentioned one such
common scenario (not waiting for background squid-z) but there are
others, possibly including handling of Squid crashes. Do you see any
other errors, assertions, or FATAL messages in your cache.log?

2. Squid code that is trying to open the shared segment is broken or,
more likely, not compatible with your FreeBSD environment. For example,
it tries to exclusively create a shared segment using the wrong name.

If you can reproduce this, I recommend starting Squid via strace (or
equivalent) to see the system calls that Squid is making when calling
shm_open() and the exact call parameters. This can confirm or eliminate
#2 as the suspect.


HTH,

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Logging of https

[Markey, Bruce]

Perfect.  I'm totally fine without the full url.

Thanks
Bruce

On Apr 7, 2016 11:34 AM, James Lay  wrote:
That's correctpeek/stare don't require a cert on the client end.
Just keep in mind you won't get a full URL in the logs with https
sites...just the host/ip:

Apr  7 09:30:31 gateway (squid-1): 192.168.1.106 - -
[07/Apr/2016:09:30:31 -0600] "CONNECT 216.58.193.78:443 HTTP/1.1"
safebrowsing.google.com - 200 871538 TCP_TUNNEL:ORIGINAL_DST

James

On 2016-04-07 07:11, Markey, Bruce wrote:
> Ok thanks for that.  I think I have a slightly better understanding of
> what is going on.That being said this is what I've come up with.
>
> No caching.  All sites allowed, peeking at all.
>
> I'm hoping this config will simply give me the logging that I'm
> looking for and nothing else.  And from that link you sent I don't
> have to install the client side cert?
>
> Thanks
>
>   1 #Access Lists
>   2 acl internal src 192.168.200.0/21
>   3 acl wireless src 192.168.100.0/23
>   4
>   5 #Ports allowed through Squid
>   6 acl Safe_ports port 80
>   7 acl Safe_ports port 443
>   8 acl SSL_ports port 443
>   9 acl CONNECT method CONNECT
>  10
>  11 #allow/deny
>  12 http_access allow internal
>  13 http_access allow wireless
>  14 http_access deny !Safe_ports
>  15 http_access deny CONNECT !SSL_ports
>  16 http_access deny all
>  17
>  18 #Bumping
>  19 acl step1 at_step SslBump1
>  20 acl step2 at_step SslBump2
>  21 acl step3 at_step SslBump3
>  22
>  23 ssl_bump peek all
>  24 ssl_bump splice all
>  25
>  26 sslproxy_capath /etc/ssl/certs
>  27
>  28 sslcrtd_program /usr/lib/squid3/ssl_crtd -s /opt/var/ssl_db -M 6MB
>  29 sslcrtd_children 5
>  30
>  31 #certs
>  32 cert=/etc/squid3/certs/squid.pem
>  33 cafile=/etc/squid3/certs/squid.pem
>  34 key=/etc/squid3/certs/squid.pem generate-host-certificates=on
> dynamic_cert_mem_cache_size=6MB sslflags=NO_SESSION_REUSE
>  35
>  36 logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni
> %ssl::>cert_subject %>Hs %  37
>  38 access_log syslog:daemon.info mine
>  39
>  40 #intercept
>  41 http_port 3128 intercept
>  42 https_port 3129 intercept ssl-bump
>  43
>  44 #nameservers
>  45 dns_nameservers 192.168.201.1 8.8.8.8
>  46
>  47 #WCCPv2 items
>  48 wccp_version 2
>  49 wccp2_router 192.168.200.73
>  50 wccp2_forwarding_method gre
>  51 wccp2_return_method gre
>  52 wccp2_service standard 0 password=LNP1
>  53 wccp2_service dynamic 70 password=LNP1
>  54 wccp2_service_info 70 protocol=tcp flags=dst_ip_hash priority=240
> ports=443
>  55
>
> Bruce Markey | Network Security Analyst
> STEINMAN COMMUNICATIONS
> 717.291.8758 (o) | bmar...@steinmancommunications.com
> 8 West King St | PO Box 1328, Lancaster, PA 17608-1328
>
> -Original Message-
> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org]
> On Behalf Of James Lay
> Sent: Thursday, March 24, 2016 4:14 PM
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] Logging of https
>
> On 2016-03-24 13:41, Markey, Bruce wrote:
>> I'm hoping this is a simple question, I've gotten/seen differing
>> answers and I'd just like a final answer.
>>
>> With squid setup as a transparent proxy via wccp will there be any log
>> entries for https sites, even just the ip?  Just the initial get
>> request is what I'd expect.
>>
>> ( I have no interest in breaking https, I'd simply like to get any
>> data I can without having to go down that road)
>>
>> If yes then what needs to be done to make that happen. Currently
>> everything is working on the http side perfectly.  Oh the https side
>> as soon as I enable wccp redirection of 443 to squid it breaks https.
>>  ( I'll add here that I've read all the peek and splice info and I
>> don't really understand it.)
>>
>> Thanks
>>
>> BRUCE MARKEY | Network Security Analyst
>>
>> STEINMAN COMMUNICATIONS
>>
>> 717.291.8758 (o) | bmar...@steinmancommunications.com
>>
>> 8 West King St | PO Box 1328, Lancaster, PA 17608-1328
>>
>>
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>
>
> Read this:
>
> http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389
>
> Sample messages:
>
> allowed https:
> Mar 24 14:02:11 gateway (squid-1): 192.168.1.101 - -
> [24/Mar/2016:14:02:11 -0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - -
> 200 5511 TCP_TUNNEL:ORIGINAL_DST
>
> note the size, 5511, and the TCP_TUNNEL, this has no SNI
>
> denied https:
> Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
> [24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.35.38:443 HTTP/1.1" - -
> 200
> 0 TAG_NONE:ORIGINAL_DST
>
> note the size, 0, and the TAG_NONE, and this also has no SNI
>
> Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
> [24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.177.121:443 HTTP/1.1"
> track.appsflyer.com - 200 0 TAG_NONE:ORIGINAL_DST
>
> again, size, and TAG_NONE, but we saw SNI for this one.
>
> 

[squid-users] Debian jessie + squid 3.5.16 - Will not start.

[Markey, Bruce]

I'm running debian Jessie.
Squid 3.5.16 compiled from source with the following:

./configure --build=x86_64-linux-gnu \
--prefix=/usr \
--includedir=${prefix}/include \
--mandir=${prefix}/share/man \
--infodir=${prefix}/share/info \
--sysconfdir=/etc \
--localstatedir=/var \
--libexecdir=${prefix}/lib/squid3 \
--srcdir=. \
--disable-maintainer-mode \
--disable-dependency-tracking \
--disable-silent-rules \
--datadir=/usr/share/squid3 \
--sysconfdir=/etc/squid3 \
--mandir=/usr/share/man \
--enable-inline \
--enable-gnuregex \
--enable-xmalloc-statistics \
--enable-useragent-log \
--enable-kill-parent-hack \
--enable-htpc \
--enable-forw-via-db \
--enable-dl-malloc \
--enable-time-hack \
--enable-err-language=English \
--disable-arch-native \
--enable-async-io=8 \
--enable-storeio=ufs,aufs,diskd,rock \
--enable-removal-policies=lru,heap \
--enable-delay-pools \
--enable-cache-digests \
--enable-icap-client \
--enable-follow-x-forwarded-for \
--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB \
--enable-auth-digest=file,LDAP \
--enable-auth-negotiate=kerberos,wrapper \
--enable-auth-ntlm=fake,smb_lm \
--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group
 \
--enable-url-rewrite-helpers=fake \
--enable-eui \
--enable-esi \
--enable-icmp \
--enable-zph-qos \
--enable-ecap \
--disable-translation \
--with-swapdir=/var/spool/squid3 \
--with-logdir=/var/log/squid3 \
--with-pidfile=/var/run/squid3.pid \
--with-filedescriptors=65536 \
--with-large-files \
--with-default-user=proxy \
--enable-ssl \
--enable-ssl-crtd \
--enable-wccpv2 \
--with-openssl \
--enable-linux-netfilter \
'CFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security 
-Wall' \
'LDFLAGS=-fPIE -pie -Wl,-z,relro -Wl,-z,now' \
'CPPFLAGS=-D_FORTIFY_SOURCE=2' \
'CXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat 
-Werror=format-security'

Here is my squid.conf

#Access Lists
acl internal src 192.168.200.0/21
acl wireless src 192.168.100.0/23

#Ports allowed through Squid
acl Safe_ports port 80
acl Safe_ports port 443
acl SSL_ports port 443
acl CONNECT method CONNECT

#allow/deny
http_access allow internal
http_access allow wireless
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

#Bumping
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

ssl_bump peek all
ssl_bump splice all

sslproxy_capath /etc/ssl/certs

sslcrtd_program /usr/lib/squid3/ssl_crtd -s /etc/squid3/ssl_db -M 4MB
sslcrtd_children 5


logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni 
%ssl::>cert_subject %>Hs %___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.16 and vary loop objects (bug ?)

[joe]

some of my post you miss reading or
wen i ask for water  and i get water  same size same glas  but i was offered
somthing else i was refuse that
and my friend ask for the same he should get same glas of water same size
so what i was trying to enplane is
 

i ask  for gzip  the the md5 vary calculate with string
accept-encoding="gzip,%20deflate,%20sdch" 
so the gzip dose not have %20  on front of it so we can use only that and
filter out those ,%20deflate,%20sdch
it will be string accept-encoding="gzip" use  that in md5 calc only
if other browser has in vary string accept-encoding="gzip,%20deflate" 
without examle  %20sdch
md5 calc use string accept-encoding="gzip,%20deflate"
so we also filter out -- > ,%20deflate
and keep only to use md5 with  string accept-encoding="gzip"

since  gzip dose not have %20 on front of it

that will make a beter hit without any problem on all browser

as i says and i test the link i provide before   
on firefox   first time after the cache ar empty  i get HIT  alwes just by
using firefox
until now perfect

but  wen i use chrome since it send and resive in very  string
accept-encoding="gzip,%20deflate,%20sdch"
the cached file ar deleted and resolt  = MISS  then it re creat new file
with HIT alwes just by using chrome

if i return to firefox its  MISS  then file got purged from cache then its
HIT again

if other browser has  vary  with string
accept-encoding="%20gzip,deflate,%20sdch"
watch this  deflate   has no --->> %20 in front of it so md5 calc should
use only deflate   not encluding the one with  %20   string
this how i understand it   and i monitor those prossess i prove that is
waste

so anoter example cause i dont know if im trying to explain it in right
manner 
 another browser get vary  string accept-encoding="%20gzip,%20deflate,sdch"
as you see  sdch  its the one used  so we must use that filter out the rest 

variety with string confuse the correct match match lookup it it purge the
same file just becaus somthing in vary has extra string and its not used






--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-3-5-16-and-vary-loop-objects-bug-tp4676901p4676990.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] External ACL Lookup

[Craddock, Tommy]

My replies are interspersed below, in between lines of

___


On 6/04/2016 9:16 a.m., Craddock, Tommy wrote:
> Hello,
> 
> Trying to use an external ACL helper to do a lookup of my user in a group in 
> a Windows AD.  I can test from the command line:
> 
> 
> /usr/lib64/squid/squid_ldap_group -R -K -S -b "dc=example,dc=com" -D 
> sq...@example.com -W /etc/squid/password -f 
> "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=%g,ou=Some 
> Group,dc=EXAMPLE,dc=COM))" -h dc01.example.com tcradd...@example.com 
> Full.Access OK
> 

I'm always a little suspicious about whitespace in the LDAP parameters.
Such as you have for "ou=Some Group" in the -f filter.

It does depend on how new vs old your Squid is whether that will be treated as 
two parameters or one passed to the helper by Squid. The commmad line test will 
always pass it as one parameter.

If you can rework your ou= parameter to avoid the whitespace it might work 
better (just a maybe, but you do have Squid 3.1).

___
MY REPLY:


Amos, I moved my group into a different OU, called Some.OU, and it still gives 
me the same result in the cache.log:


In squid.conf:

external_acl_type memberof %LOGIN /usr/lib64/squid/squid_ldap_group -R -K -S -b 
"dc=example,dc=com" -D sq...@example.com -W /etc/squid/password -f 
"(&(objectclass=person)(sAMAccountName=$)(memberof=cn=%g,ou=Some.OU,dc=example,dc=com))"
 -h dc01.example.com


In cache.log:

2016/04/07 09:26:55.123| aclMatchExternal: memberof("tcradd...@example.com 
Full.Access") = lookup needed
2016/04/07 09:26:55.123| aclMatchExternal: "tcradd...@example.com Full.Access": 
entry=@0, age=0
2016/04/07 09:26:55.123| aclMatchExternal: "tcradd...@example.com Full.Access": 
queueing a call.
2016/04/07 09:26:55.123| aclMatchExternal: "tcradd...@example.com Full.Access": 
return -1.
2016/04/07 09:26:55.123| externalAclLookup: lookup in 'memberof' for 
'tcradd...@example.com Full.Access'
2016/04/07 09:26:55.128| externalAclHandleReply: reply="ERR"
2016/04/07 09:26:55.128| external_acl_cache_add: Adding 'tcradd...@example.com 
Full.Access' = 0
2016/04/07 09:26:55.128| aclMatchExternal: memberof = 0

__
> 
> In the file referenced in the ACLs:
> 
> acl RestrictedAccessexternal memberof "/etc/squid/restricted_access.txt"
> acl FullAccess  external memberof "/etc/squid/full_access.txt"
> 
> 
> it has:
> 
> cat /etc/squid/full_access.txt
> Full.Access
> 
> cat /etc/squid/restricted_access.txt
> Restricted.Access
> 

Speaking of white spaces. The only reason for using files there is when the 
group name contains a whitespace character. TO avoid a squid.conf parser bug 
(Sorry). If those dots are in fact dots and not spaces, then you dont need the 
extra files.


__
MY REPLY:

Understood, changed this to:

acl RestrictedAccessexternal memberof Restricted.Access
acl FullAccess external memberof Full.Access

___
> 
> ### provide basic authentication via ldap for clients not 
> authenticated via kerberos/ntlm auth_param basic program 
> /usr/lib64/squid/squid_ldap_auth -R -b "dc=example,dc=com" -D 
> sq...@example.com -W /etc/squid/password -f sAMAccountName=%s -h 
> DC01.EXAMPLE.COM auth_param basic children 10 auth_param basic realm 
> Internet Proxy auth_param basic credentialsttl 1 minute
> 


Your NTLM and Negotiate authenticators have a parameter requiring membership of 
the Ful.Access group as part of the auth process.

That means you should be able to use the auth type to tell what group they are 
a member of.


___

How?  Ive tried to find out how to use that in a ACL but I haven’t been able to 
find an example. Plus, since Im using negotiate with NTLM and Kerberos, if the 
client uses Kerberos, does the NTLM group membership parameter even get used?
___
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> #http_access deny !memberof
> http_access allow localhost
> http_access allow HEAD
> http_access deny !our_networks
> http_access allow Smartconnect
> http_access deny 

Re: [squid-users] compiling 4.0.8 on FreeBSD-10.1

[Amos Jeffries]

On 7/04/2016 3:57 a.m., Odhiambo Washington wrote:
> Hi,
> 
> My configure options:
> 
> 
> setenv CC  clang
> setenv CXX clang++
> 

Something is not right with the above. I dont think its doing what you
think it does.

The Squid ./configure found that a compiler called "c++" was the only
working one, and the libtool ./configure found that gcc was available
and used that. That can only happen if they both see the CC/CXX
variables as being empty.


Normally we setup the environment using shell parameters like this:
 ./configure CC=clang CXX=clang++ ...


Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Error starting 3.5.16 on FreeBSD-8.4

[Amos Jeffries]

On 7/04/2016 3:16 a.m., Odhiambo Washington wrote:
> I am getting the following error in cache.log:
> 
> Squid Cache (Version 3.5.16): Terminated abnormally.
> CPU Usage: 0.082 seconds = 0.052 user + 0.030 sys
> Maximum Resident Size: 54992 KB
> Page faults with physical i/o: 0
> FATAL: Ipc::Mem::Segment::create failed to
> shm_open(/opt/squid-3.5/var/run/squid/cf__metadata.shm): (17) File exists
> 
> 
> However, that file doesn't exist.
> 

This can happen if you have a startup script the runs 'squid -z' or
similar just prior to starting the main proxy, and not waiting
sufficiently long for the -z run to finish.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.16 and vary loop objects (bug ?)

[Amos Jeffries]

On 7/04/2016 5:05 a.m., joe wrote:
> amos question ?
> 
> off topic sorry for that 
> 
> vary is the second key store right ?

I'm not sure what you mean by "second key store". Its a bit ambiguous.

Squid looks up the URL and finds an object saying there is Vary header
details also needed.
Squid does a second lookup (on the same cache) for a string containing
URL + extras.
Squid then verifies that the object found by that second lookup actually
matches the URL *and* that it has a correct Vary header for the pattern
being looked up (to make sure its from the current Vary set and not some
older obsolete set).

> 
> same object with 2 browser  firefox
> accept-encoding="gzip,%20deflate"
> 
> chrome
> accept-encoding="gzip,%20deflate,%20sdch"
> 
> so if they ar 2*1 object stored is that right   ? 

 (2 +1) objects. Add not multiply.

> and other browser might have more compression 
> accept-encoding="gzip,%20deflate,%xxx
> that will be 3*1   right  if yes

That would be a third variant, yes.

>   why not filtering only the one has (not
> %) = accept-encoding="gzip"
> and use that only it will be beter hit and save more storage
> just idea what you think  ??  since "%"  meaning those compression not used
> for the object only
> the one without it 

The object encoding is not relevant. Surprising maybe, but true.

Think of it like this:

 If you request from me a drink, indicating that you will take Coffee or
Tea. I hand you a Coffee.
 *  Does that mean I dont have plain Water? of course not.

 If you asked me for a drink and indicated you will take Coffee, Tea or
Water. I might hand you Water.

 If you asked for Coffee or Tea a second time. I might hand you Tea.

 If you asked for Beer or Coffee. I might hand you Water. (Even though I
did have Coffee earlier, and I might have Beer but not be willing to
deliver it yet)


The one thing in common with all the requests above was Coffee. But
notice how the most common thing delivered was actually Water, and
sometimes Tea.



Back to caching.

The variant object stored at cache location:
   URL + 'accept-encoding="gzip,%20deflate"'

... is *only* guaranteed to be whatever object needs to be delivered
when the client sends "Accept-Encoding: gzip, deflate". That is all.


When the server response contains "Vary: Accept-Encoding" then even a
single-character difference in two client requests Accept-Encoding
header means a different cache variant object. Because that small
difference does mean it could be a different type coming back. Including
the *absence* of that header being one possible variant.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.16 and vary loop objects (bug ?)

[Amos Jeffries]

On 7/04/2016 1:42 a.m., joe wrote:
> yes
> 
> FredB wrote
>>>
>>> Attached is a patch which I think will fix 3.5.16 (should apply fine
>>> on
>>> 4.0.8 too) without needing the cache reset. Anyone able to test it
>>> please?
>>>
>>
>> Reset the cache still needed, at least in my case 
>>

Hmm. I'm not sure why that reset would be needed. I just ran a series of
tests with detailed debugging of the vary details being loaded from disk
and it seems the last patch was correctly erasing the \0 terminators
(and they were wrongly being stored).

So at least this new regression is fixed. Anything else seen in Vary is
a separate bug.

FYI Ralf reported bug 4481 to track it. I have updated the bug with an
explanation of the issue and applied that last patch to Squid-4 now. It
should be in 3.5 in a day or two.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Logging of https

[Markey, Bruce]

Ok thanks for that.  I think I have a slightly better understanding of what is 
going on.That being said this is what I've come up with.  

No caching.  All sites allowed, peeking at all. 

I'm hoping this config will simply give me the logging that I'm looking for and 
nothing else.  And from that link you sent I don't have to install the client 
side cert?

Thanks

  1 #Access Lists
  2 acl internal src 192.168.200.0/21
  3 acl wireless src 192.168.100.0/23
  4 
  5 #Ports allowed through Squid
  6 acl Safe_ports port 80
  7 acl Safe_ports port 443
  8 acl SSL_ports port 443
  9 acl CONNECT method CONNECT
 10 
 11 #allow/deny
 12 http_access allow internal
 13 http_access allow wireless
 14 http_access deny !Safe_ports
 15 http_access deny CONNECT !SSL_ports
 16 http_access deny all
 17 
 18 #Bumping 
 19 acl step1 at_step SslBump1
 20 acl step2 at_step SslBump2
 21 acl step3 at_step SslBump3
 22 
 23 ssl_bump peek all
 24 ssl_bump splice all
 25 
 26 sslproxy_capath /etc/ssl/certs
 27 
 28 sslcrtd_program /usr/lib/squid3/ssl_crtd -s /opt/var/ssl_db -M 6MB
 29 sslcrtd_children 5
 30 
 31 #certs
 32 cert=/etc/squid3/certs/squid.pem
 33 cafile=/etc/squid3/certs/squid.pem
 34 key=/etc/squid3/certs/squid.pem generate-host-certificates=on 
dynamic_cert_mem_cache_size=6MB sslflags=NO_SESSION_REUSE
 35 
 36 logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni 
%ssl::>cert_subject %>Hs %mailto:squid-users-boun...@lists.squid-cache.org] On Behalf 
Of James Lay
Sent: Thursday, March 24, 2016 4:14 PM
To: squid-users@lists.squid-cache.org
Subject: Re: [squid-users] Logging of https

On 2016-03-24 13:41, Markey, Bruce wrote:
> I'm hoping this is a simple question, I've gotten/seen differing 
> answers and I'd just like a final answer.
> 
> With squid setup as a transparent proxy via wccp will there be any log 
> entries for https sites, even just the ip?  Just the initial get 
> request is what I'd expect.
> 
> ( I have no interest in breaking https, I'd simply like to get any 
> data I can without having to go down that road)
> 
> If yes then what needs to be done to make that happen. Currently 
> everything is working on the http side perfectly.  Oh the https side 
> as soon as I enable wccp redirection of 443 to squid it breaks https.
>  ( I'll add here that I've read all the peek and splice info and I 
> don't really understand it.)
> 
> Thanks
> 
> BRUCE MARKEY | Network Security Analyst
> 
> STEINMAN COMMUNICATIONS
> 
> 717.291.8758 (o) | bmar...@steinmancommunications.com
> 
> 8 West King St | PO Box 1328, Lancaster, PA 17608-1328
> 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


Read this:

http://thread.gmane.org/gmane.comp.web.squid.general/114384/focus=114389

Sample messages:

allowed https:
Mar 24 14:02:11 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:14:02:11 -0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - -
200 5511 TCP_TUNNEL:ORIGINAL_DST

note the size, 5511, and the TCP_TUNNEL, this has no SNI

denied https:
Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.35.38:443 HTTP/1.1" - - 200
0 TAG_NONE:ORIGINAL_DST

note the size, 0, and the TAG_NONE, and this also has no SNI

Mar 24 13:36:01 gateway (squid-1): 192.168.1.101 - -
[24/Mar/2016:13:36:01 -0600] "CONNECT 54.171.177.121:443 HTTP/1.1" 
track.appsflyer.com - 200 0 TAG_NONE:ORIGINAL_DST

again, size, and TAG_NONE, but we saw SNI for this one.

the above are the output when using the config info in the link.  Hope that 
helps.

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users