[squid-users] NAT/TPROXY lookup failed to locate original IPs

[dbrb2]

Hi, 

I'm not sure if this is a Mint issue, a Squid issue, a bit of both, or
neitherbut here goes:
 
I am trying to build squid on Mint 17.3 
kernel 3.19.0-32 geeric 
Squid 3.5.12 

Alls seems to have built OK, Squid launches without errors, and the proxy
works OK for HTTP requests.However when I try to proxy an SSL connection,
the squid logs show:
 
ERROR: NAT/TPROXY lookup failed to locate original IPs on local=
remote=yyy
 
I'm not having much luck deciphering this...any ideas? 



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/NAT-TPROXY-lookup-failed-to-locate-original-IPs-tp4675464.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] NAT/TPROXY lookup failed to locate original IPs

[Ben Barker]

Thanks Amos - good points - thanks. Both now fixed - thought I still seem
to be getting errors...sorry to be a bit inept here!

squid -v
Squid Cache: Version 3.5.12
Service Name: squid
configure options:
 '--prefix=/usr' '--localstatedir=/var' '--libexecdir=/lib/squid'
'--datadir=/share/squid' '--sysconfdir=/etc/squid'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--enable-icap-client' '--enable-linux-netfilter' '--enable-ssl-crtd'
'--with-default-user=squid' '--with-openssl'

cctv@bridgebox ~/squid-3.5.12 $ 2016/01/06 11:56:58 kid1| Current Directory
is /home/cctv/squid-3.5.12
2016/01/06 11:56:58 kid1| Starting Squid Cache version 3.5.12 for
i686-pc-linux-gnu...
2016/01/06 11:56:58 kid1| Service Name: squid
2016/01/06 11:56:58 kid1| Process ID 1721
2016/01/06 11:56:58 kid1| Process Roles: worker
2016/01/06 11:56:58 kid1| With 1024 file descriptors available
2016/01/06 11:56:58 kid1| Initializing IP Cache...
2016/01/06 11:56:58 kid1| DNS Socket created at [::], FD 6
2016/01/06 11:56:58 kid1| DNS Socket created at 0.0.0.0, FD 7
2016/01/06 11:56:58 kid1| Adding nameserver 208.67.222.222 from
/etc/resolv.conf
2016/01/06 11:56:58 kid1| Adding nameserver 208.67.220.220 from
/etc/resolv.conf
2016/01/06 11:56:58 kid1| helperOpenServers: Starting 5/5 'ssl_crtd'
processes
2016/01/06 11:56:58 kid1| helperOpenServers: Starting 0/20
'basic_ncsa_auth' processes
2016/01/06 11:56:58 kid1| helperOpenServers: No 'basic_ncsa_auth' processes
needed.
2016/01/06 11:56:58 kid1| Logfile: opening log
daemon:/var/log/squid/access.log
2016/01/06 11:56:58 kid1| Logfile Daemon: opening log
/var/log/squid/access.log
2016/01/06 11:56:58 kid1| Store logging disabled
2016/01/06 11:56:58 kid1| Swap maxSize 0 + 262144 KB, estimated 20164
objects
2016/01/06 11:56:58 kid1| Target number of buckets: 1008
2016/01/06 11:56:58 kid1| Using 8192 Store buckets
2016/01/06 11:56:58 kid1| Max Mem  size: 262144 KB
2016/01/06 11:56:58 kid1| Max Swap size: 0 KB
2016/01/06 11:56:58 kid1| Using Least Load store dir selection
2016/01/06 11:56:58 kid1| Current Directory is /home/cctv/squid-3.5.12
2016/01/06 11:56:58 kid1| Finished loading MIME types and icons.
2016/01/06 11:56:58 kid1| HTCP Disabled.
2016/01/06 11:56:58 kid1| Squid plugin modules loaded: 0
2016/01/06 11:56:58 kid1| Adaptation support is off.
2016/01/06 11:56:58 kid1| Accepting HTTP Socket connections at
local=[::]:13128 remote=[::] FD 22 flags=9
2016/01/06 11:56:58 kid1| Accepting NAT intercepted SSL bumped HTTPS Socket
connections at local=[::]:13129 remote=[::] FD 23 flags=41
2016/01/06 11:56:59 kid1| storeLateRelease: released 0 objects
squid2016/01/06 11:57:24 kid1| Starting new basicauthenticator helpers...
2016/01/06 11:57:24 kid1| helperOpenServers: Starting 1/20
'basic_ncsa_auth' processes
2016/01/06 11:58:57 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
local=10.163.17.250:13129 remote=x:48616 FD 16 flags=33: (92) Protocol
not available
2016/01/06 11:58:57 kid1| ERROR: NAT/TPROXY lookup failed to locate
original IPs on local=x:13129 remote=x:48616 FD 16 flags=33
2016/01/06 11:58:58 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
local=x:13129 remote=10.163.45.115:48617 FD 16 flags=33: (92) Protocol
not available




On Wed, Jan 6, 2016 at 11:43 AM, Amos Jeffries  wrote:

> On 6/01/2016 10:50 p.m., dbrb2 wrote:
> > Squid version and config options:
> >
> > Squid Cache: Version 3.5.12
> > Service Name: squid
> > configure options:  '--prefix=/usr' '--localstatedir=/var'
> > '--libexecdir=/lib/squid' '--datadir=/share/squid'
> > '--sysconfdir=/etc/squid' '--with-default-user=proxy'
> > '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
> > '--enable-icap-client' '--enable-ssl' '--enable-ssl-crtd'
> > '--with-default-user=squid' '--with-openssl'
>
> You have --with-default-user=X listed twice with two different account
> names. Pick one.
>
> Also --enable-ssl does not exist in 3.5. Remove.
>
> You are missing the --enable-linux-netfilter option that enables NAT
> interception on Linux.
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Host header forgery policy in service provider environment

[Amos Jeffries]

On 6/01/2016 10:10 p.m., Garri Djavadyan wrote:
>> On 2015-12-31 00:01, Garri Djavadyan wrote:
>>> Hello Squid members and developers!
>>>
>>> First of all, I wish you a Happy New Year 2016!
>>>
>>> The current Host header forgery policy effectively prevents a cache
>>> poisoning. But also, I noticed, it deletes verified earlier cached
>>> object. Is it possible to implement more careful algorithm as an
>>> option? For example, if Squid will not delete earlier successfully
>>> verified and valid cached object and serve forged request from the
>>> cache if would be more effective and in same time secure behavior.
>>
>>
>> This seems to be describing 
>> 
>>
>> So far we don't have a solution. Patches very welcome.
>>
>> Amos
> 
> Amos, can recheck the bug report? I found the root cause of the problem
> and presented possible prototype solution, which solves the problem in
> my environment. Thank you in advance!


Got the bug update notice. The double-check may take a while to track
down all the side effects. Thank you very much in advance anyhow. :-)

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] NAT/TPROXY lookup failed to locate original IPs

[dbrb2]

Squid version and config options:

Squid Cache: Version 3.5.12
Service Name: squid
configure options:  '--prefix=/usr' '--localstatedir=/var'
'--libexecdir=/lib/squid' '--datadir=/share/squid'
'--sysconfdir=/etc/squid' '--with-default-user=proxy'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--enable-icap-client' '--enable-ssl' '--enable-ssl-crtd'
'--with-default-user=squid' '--with-openssl'


Squid.conf:

auth_param basic program /lib/squid/basic_ncsa_auth /etc/squid/users
auth_param basic realm cctv

acl auth_users proxy_auth REQUIRED
http_access allow auth_users
http_port 13128
https_port 13129 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/usr/local/squid/ssl_cert/myCA.pem
ssl_bump none localhost
ssl_bump server-first all
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s
/usr/local/squid/var/lib/ssl_db -M 4MB
sslcrtd_children 5

On Wed, Jan 6, 2016 at 9:41 AM, Antony Stone [via Squid Web Proxy Cache] <
ml-node+s1019090n467546...@n4.nabble.com> wrote:

> On Wednesday 06 January 2016 at 10:36:20, dbrb2 wrote:
>
> > I am trying to build squid on Mint 17.3
> > kernel 3.19.0-32 geeric
> > Squid 3.5.12
>
> > when I try to proxy an SSL connection, the squid logs show:
> >
> > ERROR: NAT/TPROXY lookup failed to locate original IPs on local=
> > remote=yyy
> >
> > I'm not having much luck deciphering this...any ideas?
>
> Show us your squid.conf without comments or blank lines?
>
>
> Antony.
>
> --
> Behind the counter a boy with a shaven head stared vacantly into space,
> a dozen spikes of microsoft protruding from the socket behind his ear.
>
>  - William Gibson, Neuromancer (1984)
>
>Please reply to the
> list;
>  please *don't* CC
> me.
> ___
> squid-users mailing list
> [hidden email] 
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://squid-web-proxy-cache.1019090.n4.nabble.com/NAT-TPROXY-lookup-failed-to-locate-original-IPs-tp4675464p4675465.html
> To unsubscribe from NAT/TPROXY lookup failed to locate original IPs, click
> here
> 
> .
> NAML
> 
>




--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/NAT-TPROXY-lookup-failed-to-locate-original-IPs-tp4675464p4675466.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] NAT/TPROXY lookup failed to locate original IPs

[Amos Jeffries]

On 6/01/2016 10:50 p.m., dbrb2 wrote:
> Squid version and config options:
> 
> Squid Cache: Version 3.5.12
> Service Name: squid
> configure options:  '--prefix=/usr' '--localstatedir=/var'
> '--libexecdir=/lib/squid' '--datadir=/share/squid'
> '--sysconfdir=/etc/squid' '--with-default-user=proxy'
> '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
> '--enable-icap-client' '--enable-ssl' '--enable-ssl-crtd'
> '--with-default-user=squid' '--with-openssl'

You have --with-default-user=X listed twice with two different account
names. Pick one.

Also --enable-ssl does not exist in 3.5. Remove.

You are missing the --enable-linux-netfilter option that enables NAT
interception on Linux.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Host header forgery policy in service provider environment

[Garri Djavadyan]

>On 2015-12-31 00:01, Garri Djavadyan wrote:
>> Hello Squid members and developers!
>> 
>> First of all, I wish you a Happy New Year 2016!
>> 
>> The current Host header forgery policy effectively prevents a cache
>> poisoning. But also, I noticed, it deletes verified earlier cached
>> object. Is it possible to implement more careful algorithm as an
>> option? For example, if Squid will not delete earlier successfully
>> verified and valid cached object and serve forged request from the
>> cache if would be more effective and in same time secure behavior.
>
>
>This seems to be describing 
>
>
>So far we don't have a solution. Patches very welcome.
>
>Amos

Amos, can recheck the bug report? I found the root cause of the problem
and presented possible prototype solution, which solves the problem in
my environment. Thank you in advance!
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] confused over ipv6 failing on ipv4-only network

[Amos Jeffries]

On 6/01/2016 7:29 p.m., Jason Haar wrote:
> On 06/01/16 17:39, Amos Jeffries wrote:
>> On 6/01/2016 5:04 p.m., Jason Haar wrote:
>>> Hi there
>>>
>>> Weird - several times in the past couple of months I have found I cannot
>>> get to http://wiki.squid-cache.org/ - I get the error below from my
>>> squid-3.5.11 server which does not have a Global ipv6 address (it has a
>>> Local ipv6/fe80: on the Ethernet card - but nothing else). Google.com
>>> (which is fully ipv6 capable) works fine - so far only
>>> wiki.squid-cache.org has shown up this way to me (ie I don't see this
>>> error message.
>>>
>>> On the squid server, "dig a" shows valid ipv4 addresses and "dig "
>>> shows the ipv6 address - but why is squid even trying to connect over
>>> ipv6 If doesn't have an ipv6 address?
>>>
>>> Could this be a case of the "A" record failing to return fast enough,
>>> forcing squid to only try ipv6 - which then leads to the error message
>>> referring to the ipv6 address?
>> Squid waits for both A and  before continuing after DNS lookup. The
>> only way to get only IPv6 results is for your DNS server to produce no A
>> results at all. Timeout _could_ do that, but the default is 30 sec so
>> unlikely.
> 
> I think that must be the case, because when I saw the problem this
> morning, I immediately ssh'ed into the squid server and nslookup showed
> it was resolving the name to it's A record just fine (by then) - and
> telnet-ing to the IPv4 address was fine too. So it must have either
> timed out on the A lookups (but not the  records), or the DNS server
> didn't return A records at all? I don't think there's a way to query
> squid to see what it's current DNS cache is? That would definitively
> answer that question

The cache manager "ipcache" report contains a listing of the DNS cache:
 squidclient mgr:ipcache


> 
> 
>> The Squid wiki is dual-stacked with IPv4 addresses. Sice you have
>> v4-only network the thing to do is find out why the IPv4 are not
>> working for your Squid. 
> 
> Well yeah  - but I frankly don't see this on any other website (like
> google.com) - just wiki.squid-cache.org - so I think there's something
> going on between those DNS servers and my squid server sitting on a
> SPARK NZ network

Hmm. Wiki is in Italy IIRC so almost worst case RTT for us. Though I've
not seen such issues from my Orcon POP.

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] NAT/TPROXY lookup failed to locate original IPs

[Amos Jeffries]

On 7/01/2016 1:08 a.m., Ben Barker wrote:
> Thanks Amos - good points - thanks. Both now fixed - thought I still seem
> to be getting errors...sorry to be a bit inept here!
> 
> squid -v
> Squid Cache: Version 3.5.12
> Service Name: squid
> configure options:
>  '--prefix=/usr' '--localstatedir=/var' '--libexecdir=/lib/squid'
> '--datadir=/share/squid' '--sysconfdir=/etc/squid'
> '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
> '--enable-icap-client' '--enable-linux-netfilter' '--enable-ssl-crtd'
> '--with-default-user=squid' '--with-openssl'
> 
> cctv@bridgebox ~/squid-3.5.12 $ 2016/01/06 11:56:58 kid1| Current Directory
> is /home/cctv/squid-3.5.12
> 2016/01/06 11:56:58 kid1| Starting Squid Cache version 3.5.12 for
> i686-pc-linux-gnu...

> 2016/01/06 11:58:57 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
> local=10.163.17.250:13129 remote=x:48616 FD 16 flags=33: (92) Protocol
> not available

The first error means the kernel NAT tables do not have any record of
the connection that arrived on the Squid intercept port.

* Do not make test connections directly to the intercept port. Test it
*exactly* as if you are a client going straight to the Internet.

* Do not perform the NAT on any other machine.

Compare your NAT rules with these to ensure you have them all right
(notice how there are 4 rules):
 

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] kerberos authentication with a machine account doesn't work

[LYMN]

Hi,

We have been using kerberos authentication against Active Directory here
for a long time by using a SPN attached to a user account and exporting
the keytab.  The issue we have is that security policy mandates that
the password on the user account be changed which means we have to go
and regenerate keytabs every time this happens.  Not exactly difficult
but tedious nonetheless.

To avoid the password change I thought it may be an idea to use the
machine account and add a SPN (http/fqdn.is.here) to that.  I added:

kerberos method = secrets and keytab
dedicated keytab file = /etc/krb5.keytab

to the smb.conf so samba will manage the keytab for me then did:

net ads join
net ads keytab add http

klist -k shows me the principals that should be there and AD agrees they
exist.  I can get a TGT using:

kinit -k

without error (setting the UPN to host/fqdn.is.here@KERBEROS.REALM may
have helped this).  Doing a 

kinit -kS http/fqdn.is.here

works without error too.  So, I think kerberos is ok but with a squid
3.5.12 configured with negotiate_kerberos_auth I see the dreaded
message:

negotiate_kerberos_auth.cc(180): pid=4888 :2016/01/07 12:50:29| 
negotiate_kerberos_auth: ERROR: gss_accept_sec_context() failed: Unspecified 
GSS failure.  Minor code may provide more information. 

and only that, no minor code when I try to use the proxy with a browser
on a windows client.  Interestingly, doing a klist on the windows client
I can see a kerberos ticket for HTTP/fqdn.is.here that is for the proxy
I am testing.

Not sure what is missing here, I have a bee in my bonnet that this should
Just Work (tm) as the only real difference is that the SPN is attached
to a computer account not a user account - I would have thought as long
as the keytab is done correctly that this should not matter but clearly
something is not agreeing with me.

-- 
Brett Lymn
This email has been sent on behalf of one of the following companies within the 
BAE Systems Australia group of companies:

BAE Systems Australia Limited - Australian Company Number 008 423 005
BAE Systems Australia Defence Pty Limited - Australian Company Number 006 
870 846
BAE Systems Australia Logistics Pty Limited - Australian Company Number 086 
228 864

Our registered office is Evans Building, Taranaki Road, Edinburgh Parks,
Edinburgh, South Australia, 5111. If the identity of the sending company is
not clear from the content of this email please contact the sender.

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy or
disclose its content, but please reply to this email immediately and highlight
the error to the sender and then immediately delete the message.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

[Eliezer Croitoru]

On 07/01/2016 04:31, Jason Haar wrote:

On 06/01/16 00:04, Amos Jeffries wrote:

Yes. Squid always has been able to given enough RAM. Squid stores most
ACLs in memory as Splay trees, so entries are sorted by frequency of use
which is dynamically adapted over time. Regex are pre-parsed and
aggregated together for reduced matching instead of re-interpreted and
parsed per-request.

Great to hear. I've got some 600,000+ domain lists (ie dstdomain) and
60,000+ url lists (ie url_regex) acls, and there are a couple of
"gotchas" I've picked up during testing


commercial paid lists?


1. at startup squid reports "WARNING: there are more than 100 regular
expressions. Consider using less REs". Is that now legacy and ignorable?
(should that be removed?). Obviously I have over 60,000 REs



2. making any change to squid and restarting/reconfiguring it now means
I'm seeing a 12sec outage as squid reads those acls off SSD
drives/parses them/etc. With squidguard that outage is hidden because
squidguard uses indexed files instead of the raw files and that
parsing/etc can be done offline. That behavioral change is pretty
dramatic: making a minor, unrelated change to squid now involves a
10+sec outage (instead of <1sec). I'd say "outsourcing" this kind of
function to another process (such as url_rewriter or ICAP) still has
it's advantages ;-)


I have been working for a while on SquidBlocker which is a filtering 
engine\DB that has a built-in ICAP service.
My plan is to publish the stable version 1.0 at the end of the 
month(31/01/2016) after couple long month of testing in production.

It's not open source but parts of it are.

One of the main points with it was blazing fast online updates with 
almost 0 down time if at all, ie a restart doesn't really exists(The 
only reasons I have restarted the service was for an update).


Eliezer
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

[Jason Haar]

On 06/01/16 00:04, Amos Jeffries wrote:
> Yes. Squid always has been able to given enough RAM. Squid stores most
> ACLs in memory as Splay trees, so entries are sorted by frequency of use
> which is dynamically adapted over time. Regex are pre-parsed and
> aggregated together for reduced matching instead of re-interpreted and
> parsed per-request.
Great to hear. I've got some 600,000+ domain lists (ie dstdomain) and
60,000+ url lists (ie url_regex) acls, and there are a couple of
"gotchas" I've picked up during testing

1. at startup squid reports "WARNING: there are more than 100 regular
expressions. Consider using less REs". Is that now legacy and ignorable?
(should that be removed?). Obviously I have over 60,000 REs
2. making any change to squid and restarting/reconfiguring it now means
I'm seeing a 12sec outage as squid reads those acls off SSD
drives/parses them/etc. With squidguard that outage is hidden because
squidguard uses indexed files instead of the raw files and that
parsing/etc can be done offline. That behavioral change is pretty
dramatic: making a minor, unrelated change to squid now involves a
10+sec outage (instead of <1sec). I'd say "outsourcing" this kind of
function to another process (such as url_rewriter or ICAP) still has
it's advantages ;-)

-- 
Cheers

Jason Haar
Corporate Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] R: Problem with Squid 3.4.4 and NTLM authentication

[Job]

Hello Amos and thank you!

>> sinec i upgraded two Squid proxy servers to the Squid-3.4.4 versions, we 
>> have some huges bottleneck with ahtenticated ntlm (old style!) users.
>> If i disable authentication and enable per-ip surf, it works fine.

>From what earlier version?

I did upgrade from the 3.1.8 version (in that ntlm worked fine for us).


>3.4.4 is very outdated version of Squid. Current release is 3.5.12 or
>3.4.14.

OK, we will upgrade to latest 3.4.x!

But why n 3.1.8 NTLM (with the same squid.conf) worked fine?
Thank you again!

Francesco
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users