[squid-users] squid tproxy problem
Hi, I configured squid for tproxy feature in my network with bridge mode. I follow http://wiki.squid-cache.org/Features/Tproxy4 But I m not getting requests in access.log of squid. My configuration: cat /etc/squid/squid.conf # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 acl localhost src ::1/128 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl to_localhost dst ::1/128 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl SSL_ports port 443 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443# https acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl CONNECT method CONNECT acl mynetwork src '/etc/squid/mynetwork' acl cache_deny dst '/etc/squid/deny1' cache deny cache_deny # cache_mem 1024 MB # Recommended minimum Access Permission configuration: # # Only allow cachemgr access from localhost http_access allow manager localhost http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on localhost is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow mynetwork http_access allow localhost # And finally deny all other access to this proxy http_access deny all # Squid normally listens to port 3128 http_port 3128 http_port 3129 tproxy # We recommend you to use at least the following line. hierarchy_stoplist cgi-bin ? # Uncomment and adjust the following to add a disk cache directory. cache_dir aufs /cache/squid 25600 32 512 # Leave coredumps in the first cache dir coredump_dir /cache/squid httpd_suppress_version_string on # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp:144020%10080 refresh_pattern ^gopher:14400%1440 refresh_pattern -i (/cgi-bin/|\?) 00%0 refresh_pattern .020%4320 ip rule list 0:from all lookup local 32765:from all fwmark 0x1 lookup 100 32766:from all lookup main 32767:from all lookup default iptables -L -nvx -t mangle Chain PREROUTING (policy ACCEPT 959157 packets, 79545939 bytes) pkts bytes target prot opt in out source destination 10993 689414 DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket 16765 1000259 TPROXY tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 TPROXY redirect 0.0.0.0:3129 mark 0x1/0x1 Chain INPUT (policy ACCEPT 15122 packets, 1149717 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 959996 packets, 79295677 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 28272 packets, 10090599 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 988265 packets, 89386044 bytes) pkts bytes target prot opt in out source destination Chain DIVERT (1 references) pkts bytes target prot opt in out source destination 10993 689414 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1 10993 689414 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ebtables -t broute --list Bridge table: broute Bridge chain: BROUTING, entries: 2, policy: ACCEPT -p IPv4 -i eth0 --ip-proto tcp --ip-dport 80 -j redirect -p IPv4 -i eth1 --ip-proto tcp --ip-sport 80 -j redirect OS CENTOS 6 64 bit squid : 3.1.4 KERNEL : 2.6.32-71.29.1.el6.x86_64 Please guide me. Thanks, Benjamin
Re: [squid-users] Squid TProxy Problem
Dear Amos, Hi As the documentation suggests, I have used the following rules, but except the first one, others fail: ip rule add fwmark 1 lookup 100 ip -f inet route add local 0.0.0.0/0 dev lo table 100 ip -f inet route add local 0.0.0.0/0 dev eth0 table 10 Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Ali Majdzadeh ali.majdza...@gmail.com Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Ali Majdzadeh ali.majdza...@gmail.com: Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Amos Jeffries squ...@treenet.co.nz: On 08/06/11 22:53, Ali Majdzadeh wrote: Amos, Hi Thanks for your reply. The Squid box has only one NIC and it is connected to the internet via it's default gateway, I think I should have corrected our network diagram as follows: Internet- Gateway- Squid- Clients Does this configuration make any difference? That diagram is no different, but a 1-NIC squid box would be: Internet-Gateway-Clients. \-Squid or: Internet-Gateway--switch--Clients. \-Squid That makes a difference. If you bump cache.log up to ALL,5 during a test connection. You may see traffic arrive but then hang while connecting out. If you do see that behaviour in cache.log, the problems is at the gateway end. It MUST be able to detect and bypass the Squid outward traffic by MAC address or tcp_outgoing_tos instead of IP address. Amos Thanks again for your reply. I will try to reconfigure the whole solution from scratch to find out where I go wrong. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Amos Jeffriessqu...@treenet.co.nz: On 08/06/11 01:15, Ali Majdzadeh wrote: Amos, The configuration is as follows: Internet-Squid-Clients Would you please clarify what you mean by declaring routing packets to the squid box? That the packets actually do get passed/routed through the squid box and not via some other possible route. Does the above configuration conform to the so-called declaration? If those are physical wires or even just logical routing table entries, yes it does. If it is so, what should be done to solve the issue? Your packet counter incrementing is a good sign that the ruting layer is okay. Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Okay. That should do :). 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Given that you have a recent libcap. That means we must suspect the kernel handling once TPROXY marks the packets. The table 100 bit of the config has given a lot of people trouble. AFAIK normally you only have one such table entry and for TPROXY its internal to the kernel with the lo interface. BUT, some people have had to configure other interfaces to get it working. Try to add a table 100 (or whatever you called it) entry for each NIC the box has. If your kernel accepts them check access.log again. If your kernel denies multiple tables, erase the existing one and try creating one for each NIC. Repeating until you find one that works. OR, if that still fails. We have to get help from Balabit and/or Netfilter and figure WTF is happening. Amos Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding
Re: [squid-users] Squid TProxy Problem
Amos, Sorry for the typo; here are the rules: ip rule add fwmark 1 lookup 100 ip -f inet route add local 0.0.0.0/0 dev lo table 100 ip -f inet route add local 0.0.0.0/0 dev eth0 table 100 Warm Regards, Ali Majdzadeh Kohbanani 2011/6/11 Ali Majdzadeh ali.majdza...@gmail.com: Dear Amos, Hi As the documentation suggests, I have used the following rules, but except the first one, others fail: ip rule add fwmark 1 lookup 100 ip -f inet route add local 0.0.0.0/0 dev lo table 100 ip -f inet route add local 0.0.0.0/0 dev eth0 table 10 Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Ali Majdzadeh ali.majdza...@gmail.com Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Ali Majdzadeh ali.majdza...@gmail.com: Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Amos Jeffries squ...@treenet.co.nz: On 08/06/11 22:53, Ali Majdzadeh wrote: Amos, Hi Thanks for your reply. The Squid box has only one NIC and it is connected to the internet via it's default gateway, I think I should have corrected our network diagram as follows: Internet- Gateway- Squid- Clients Does this configuration make any difference? That diagram is no different, but a 1-NIC squid box would be: Internet-Gateway-Clients. \-Squid or: Internet-Gateway--switch--Clients. \-Squid That makes a difference. If you bump cache.log up to ALL,5 during a test connection. You may see traffic arrive but then hang while connecting out. If you do see that behaviour in cache.log, the problems is at the gateway end. It MUST be able to detect and bypass the Squid outward traffic by MAC address or tcp_outgoing_tos instead of IP address. Amos Thanks again for your reply. I will try to reconfigure the whole solution from scratch to find out where I go wrong. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Amos Jeffriessqu...@treenet.co.nz: On 08/06/11 01:15, Ali Majdzadeh wrote: Amos, The configuration is as follows: Internet-Squid-Clients Would you please clarify what you mean by declaring routing packets to the squid box? That the packets actually do get passed/routed through the squid box and not via some other possible route. Does the above configuration conform to the so-called declaration? If those are physical wires or even just logical routing table entries, yes it does. If it is so, what should be done to solve the issue? Your packet counter incrementing is a good sign that the ruting layer is okay. Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Okay. That should do :). 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Given that you have a recent libcap. That means we must suspect the kernel handling once TPROXY marks the packets. The table 100 bit of the config has given a lot of people trouble. AFAIK normally you only have one such table entry and for TPROXY its internal to the kernel with the lo interface. BUT, some people have had to configure other interfaces to get it working. Try to add a table 100 (or whatever you called it) entry for each NIC the box has. If your kernel accepts them check access.log again. If your kernel denies multiple tables, erase the existing one and try creating one for each NIC. Repeating until you find one that works. OR, if that still fails. We have to get help from Balabit and/or Netfilter and figure WTF is happening. Amos Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6
Re: [squid-users] Squid TProxy Problem
On 08/06/11 22:53, Ali Majdzadeh wrote: Amos, Hi Thanks for your reply. The Squid box has only one NIC and it is connected to the internet via it's default gateway, I think I should have corrected our network diagram as follows: Internet- Gateway- Squid- Clients Does this configuration make any difference? That diagram is no different, but a 1-NIC squid box would be: Internet-Gateway-Clients. \-Squid or: Internet-Gateway--switch--Clients. \-Squid That makes a difference. If you bump cache.log up to ALL,5 during a test connection. You may see traffic arrive but then hang while connecting out. If you do see that behaviour in cache.log, the problems is at the gateway end. It MUST be able to detect and bypass the Squid outward traffic by MAC address or tcp_outgoing_tos instead of IP address. Amos Thanks again for your reply. I will try to reconfigure the whole solution from scratch to find out where I go wrong. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Amos Jeffriessqu...@treenet.co.nz: On 08/06/11 01:15, Ali Majdzadeh wrote: Amos, The configuration is as follows: Internet-Squid-Clients Would you please clarify what you mean by declaring routing packets to the squid box? That the packets actually do get passed/routed through the squid box and not via some other possible route. Does the above configuration conform to the so-called declaration? If those are physical wires or even just logical routing table entries, yes it does. If it is so, what should be done to solve the issue? Your packet counter incrementing is a good sign that the ruting layer is okay. Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Okay. That should do :). 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Given that you have a recent libcap. That means we must suspect the kernel handling once TPROXY marks the packets. The table 100 bit of the config has given a lot of people trouble. AFAIK normally you only have one such table entry and for TPROXY its internal to the kernel with the lo interface. BUT, some people have had to configure other interfaces to get it working. Try to add a table 100 (or whatever you called it) entry for each NIC the box has. If your kernel accepts them check access.log again. If your kernel denies multiple tables, erase the existing one and try creating one for each NIC. Repeating until you find one that works. OR, if that still fails. We have to get help from Balabit and/or Netfilter and figure WTF is happening. Amos Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffriessqu...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to
Re: [squid-users] Squid TProxy Problem
Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Ali Majdzadeh ali.majdza...@gmail.com: Amos, Thanks for your reply. As you had depicted in the diagrams, I think you meant that the clients and the Squid box are both connected to the gateway through the switch, didn't you? If it is so, yes, they are connected, but the default gateway for the clients is set to the IP address of the Squid box. So, you mean we should insert a special firewall rule in our gateway in order to detect and bypass the Squid outward traffic by its MAC address, is that true? Does this method still preserves the clients' IP addresses? Sorry for my elementary questions and thanks in advance for your helpful notes. Warm Regards, Ali 2011/6/8 Amos Jeffries squ...@treenet.co.nz: On 08/06/11 22:53, Ali Majdzadeh wrote: Amos, Hi Thanks for your reply. The Squid box has only one NIC and it is connected to the internet via it's default gateway, I think I should have corrected our network diagram as follows: Internet- Gateway- Squid- Clients Does this configuration make any difference? That diagram is no different, but a 1-NIC squid box would be: Internet-Gateway-Clients. \-Squid or: Internet-Gateway--switch--Clients. \-Squid That makes a difference. If you bump cache.log up to ALL,5 during a test connection. You may see traffic arrive but then hang while connecting out. If you do see that behaviour in cache.log, the problems is at the gateway end. It MUST be able to detect and bypass the Squid outward traffic by MAC address or tcp_outgoing_tos instead of IP address. Amos Thanks again for your reply. I will try to reconfigure the whole solution from scratch to find out where I go wrong. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/8 Amos Jeffriessqu...@treenet.co.nz: On 08/06/11 01:15, Ali Majdzadeh wrote: Amos, The configuration is as follows: Internet- Squid- Clients Would you please clarify what you mean by declaring routing packets to the squid box? That the packets actually do get passed/routed through the squid box and not via some other possible route. Does the above configuration conform to the so-called declaration? If those are physical wires or even just logical routing table entries, yes it does. If it is so, what should be done to solve the issue? Your packet counter incrementing is a good sign that the ruting layer is okay. Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Okay. That should do :). 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Given that you have a recent libcap. That means we must suspect the kernel handling once TPROXY marks the packets. The table 100 bit of the config has given a lot of people trouble. AFAIK normally you only have one such table entry and for TPROXY its internal to the kernel with the lo interface. BUT, some people have had to configure other interfaces to get it working. Try to add a table 100 (or whatever you called it) entry for each NIC the box has. If your kernel accepts them check access.log again. If your kernel denies multiple tables, erase the existing one and try creating one for each NIC. Repeating until you find one that works. OR, if that still fails. We have to get help from Balabit and/or Netfilter and figure WTF is happening. Amos Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to
Re: [squid-users] Squid TProxy Problem
Amos, The configuration is as follows: Internet - Squid - Clients Would you please clarify what you mean by declaring routing packets to the squid box? Does the above configuration conform to the so-called declaration? If it is so, what should be done to solve the issue? Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffries squ...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
Re: [squid-users] Squid TProxy Problem
On 08/06/11 01:15, Ali Majdzadeh wrote: Amos, The configuration is as follows: Internet- Squid- Clients Would you please clarify what you mean by declaring routing packets to the squid box? That the packets actually do get passed/routed through the squid box and not via some other possible route. Does the above configuration conform to the so-called declaration? If those are physical wires or even just logical routing table entries, yes it does. If it is so, what should be done to solve the issue? Your packet counter incrementing is a good sign that the ruting layer is okay. Thanks again. By the way, we have compiled libcap from source and it is the latest version of the library. Okay. That should do :). 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Given that you have a recent libcap. That means we must suspect the kernel handling once TPROXY marks the packets. The table 100 bit of the config has given a lot of people trouble. AFAIK normally you only have one such table entry and for TPROXY its internal to the kernel with the lo interface. BUT, some people have had to configure other interfaces to get it working. Try to add a table 100 (or whatever you called it) entry for each NIC the box has. If your kernel accepts them check access.log again. If your kernel denies multiple tables, erase the existing one and try creating one for each NIC. Repeating until you find one that works. OR, if that still fails. We have to get help from Balabit and/or Netfilter and figure WTF is happening. Amos Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadehali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffriessqu...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2 -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
Re: [squid-users] Squid TProxy Problem
Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffries squ...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
Re: [squid-users] Squid TProxy Problem
Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffries squ...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
Re: [squid-users] Squid TProxy Problem
Amos, Sorry, the packet counter increments, I made a mistake, but still no logs either in access.log nor in cache.log. Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com: Amos, Hi The packet counter on -j TPROXY does not increment. So, why clients are able to surf the web? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Ali Majdzadeh ali.majdza...@gmail.com Amos, Hi Thanks for your reply. Ragarding the documentation, I have inserted the following routing rules: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 Now, access.log is populated with proper logs, but clients can not surf the web, I mean the proxy server is unable to forward http responses to clients' browsers. When the client enters for example www.google.com, the connection to the http server is established but the process halts at Waiting for www.google.com and after a while Squid reports the unablility to retreive the requested URL. By the way, we have disabled selinux. Any ideas? Warm Regards, Ali Majdzadeh Kohbanani 2011/6/6 Amos Jeffries squ...@treenet.co.nz: On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
[squid-users] Squid TProxy Problem
Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Warm Regards, Ali Majdzadeh Kohbanani
Re: [squid-users] Squid TProxy Problem
On 06/06/11 06:32, Ali Majdzadeh wrote: Hello All, I have setup the following configuration: Squid (3.1.12) (--enable-linux-netfilter passed as the one and only configure option) Kernel (2.6.38.3) iptables (1.4.11) I have added the following two directives in squid.conf: http_port 3128 http_port 3129 tproxy Also, I have configured iptables with the following rules: iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 Everything work as expected, I mean, the users can surf the web and the proxy server is transparent. The problem is that actually there is no caching. I mean, both cache.log and access.log files are empty. On That would be transparency to the point of not going through the proxy. access.log should have entries for each request. the other hand, if I manually set the proxy configuration in clients' browsers (the IP address of the squid server and port number 3128) everything is OK; the log files are incremented and objects are cached. Have anyone faced the same issue? Some. Its usually boiled down to missing out some details omitted. building against libcap2 or routing packets to the squid box for example. Are the packet counters on that -j TPROXY rule showing captures? Did you follow the rest of the feature config? ie the special sub-routing table? OS packet filtering toggles? selinux updated to allow tproxy? Is this box even routing or bridging port 80 traffic for the network? Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
