Re: [squid-users] tproxy configuration
On 25/02/2013 3:00 p.m., Roman Gelfand wrote: Amos, Do you have an idea as to what I am doing wrong here? Unfortunately no. All the things that were wrong have apparently been fixed without affecting the outcome. There is nothing more I can suggest besides a double-triple-check of the entire config along the whole pathway from WCCP router rules down through the Squid box iptables, kernel settings, Squid settings and back out through to the WCCP router again. Just in case something was missed. Amos
Re: [squid-users] tproxy configuration
Amos, Do you have an idea as to what I am doing wrong here? Thanks, On Fri, Feb 22, 2013 at 12:40 PM, Roman Gelfand rgelfa...@gmail.com wrote: Thanks for taking time to help me out. If I understood you correctly, I think I made the changes you mentioned including iptables -A FORWARD -i eth0 -j ACCEPT line. still no luck. Below, is the is the diagnostics. Chain PREROUTING (policy ACCEPT 13 packets, 8499 bytes) pkts bytes target prot opt in out source destination 337 93649 ACCEPT all -- anyany anywhere 192.168.8.21 226 48201 DIVERT tcp -- anyany anywhere anywhere socket 0 0 TPROXY tcp -- anyany anywhere anywhere tcp dpt:www TPROXY redirect 0.0.0.0:3228 mark 0x1/0x1 0 0 TPROXY tcp -- anyany anywhere anywhere tcp dpt:https TPROXY redirect 0.0.0.0:3229 mark 0x1/0x1 Chain INPUT (policy ACCEPT 576 packets, 150K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 644 packets, 135K bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 644 packets, 135K bytes) pkts bytes target prot opt in out source destination Chain DIVERT (1 references) pkts bytes target prot opt in out source destination 226 48201 MARK all -- anyany anywhere anywhere MARK set 0x1 226 48201 ACCEPT all -- anyany anywhere anywhere On Fri, Feb 22, 2013 at 2:14 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 5:07 p.m., Roman Gelfand wrote: On Thu, Feb 21, 2013 at 6:10 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 11:03 a.m., Roman Gelfand wrote: Please, find below the network topology, squid.conf and rc.local configuration files. It appears that the squid is not routing the http requests. I am not sure what I am doing wrong here Please note, the same squid.conf works on transparent proxy (non tproxy), for the exception of tproxy keyword and service changes. Thanks in advance, WAN || || wccp/gre tunnel || squid==Fortigate FW/RT Int ip 1 192.168.8.1 3.3|| Int ip 2 192.168.11.1 ip: 192.168.8.21 || Ext ip XX.XX.XXX.24 || || WLAN Router Int. ip 192.168.11.32 Ext. ip 192.168.7.1 || || || Client Workstation 192.168.7.110 #!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will exit 0 on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. # GRE Tunnel : echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre LOCALIP=192.168.8.21 FORTIDIRIP=192.168.8.1 FORTIIPID=XX.XX.XXX.254 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter What about rp_filter on eth0 where the traffic is actually exiting the Squid box? Could you elaborate on this.. What rp_filter does is prevent packets from local software using that interface from using IP addresses that do not belong to that box. The purpose of TPROXY being to spoof the _clients_ IP address on outgoing trafffic. Which does not leave the machine on lo, but through eth0 or some other interface. Amos
Re: [squid-users] tproxy configuration
Thanks for taking time to help me out. If I understood you correctly, I think I made the changes you mentioned including iptables -A FORWARD -i eth0 -j ACCEPT line. still no luck. Below, is the is the diagnostics. Chain PREROUTING (policy ACCEPT 13 packets, 8499 bytes) pkts bytes target prot opt in out source destination 337 93649 ACCEPT all -- anyany anywhere 192.168.8.21 226 48201 DIVERT tcp -- anyany anywhere anywhere socket 0 0 TPROXY tcp -- anyany anywhere anywhere tcp dpt:www TPROXY redirect 0.0.0.0:3228 mark 0x1/0x1 0 0 TPROXY tcp -- anyany anywhere anywhere tcp dpt:https TPROXY redirect 0.0.0.0:3229 mark 0x1/0x1 Chain INPUT (policy ACCEPT 576 packets, 150K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 644 packets, 135K bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 644 packets, 135K bytes) pkts bytes target prot opt in out source destination Chain DIVERT (1 references) pkts bytes target prot opt in out source destination 226 48201 MARK all -- anyany anywhere anywhere MARK set 0x1 226 48201 ACCEPT all -- anyany anywhere anywhere On Fri, Feb 22, 2013 at 2:14 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 5:07 p.m., Roman Gelfand wrote: On Thu, Feb 21, 2013 at 6:10 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 11:03 a.m., Roman Gelfand wrote: Please, find below the network topology, squid.conf and rc.local configuration files. It appears that the squid is not routing the http requests. I am not sure what I am doing wrong here Please note, the same squid.conf works on transparent proxy (non tproxy), for the exception of tproxy keyword and service changes. Thanks in advance, WAN || || wccp/gre tunnel || squid==Fortigate FW/RT Int ip 1 192.168.8.1 3.3|| Int ip 2 192.168.11.1 ip: 192.168.8.21 || Ext ip XX.XX.XXX.24 || || WLAN Router Int. ip 192.168.11.32 Ext. ip 192.168.7.1 || || || Client Workstation 192.168.7.110 #!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will exit 0 on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. # GRE Tunnel : echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre LOCALIP=192.168.8.21 FORTIDIRIP=192.168.8.1 FORTIIPID=XX.XX.XXX.254 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter What about rp_filter on eth0 where the traffic is actually exiting the Squid box? Could you elaborate on this.. What rp_filter does is prevent packets from local software using that interface from using IP addresses that do not belong to that box. The purpose of TPROXY being to spoof the _clients_ IP address on outgoing trafffic. Which does not leave the machine on lo, but through eth0 or some other interface. Amos
[squid-users] tproxy configuration
Please, find below the network topology, squid.conf and rc.local
configuration files. It appears that the squid is not routing the
http requests. I am not sure what I am doing wrong here
Please note, the same squid.conf works on transparent proxy (non
tproxy), for the exception of tproxy keyword and service changes.
Thanks in advance,
WAN
||
||
wccp/gre tunnel ||
squid==Fortigate FW/RT Int ip 1 192.168.8.1
3.3|| Int ip 2 192.168.11.1
ip: 192.168.8.21 || Ext ip XX.XX.XXX.24
||
||
WLAN Router Int. ip
192.168.11.32 Ext. ip 192.168.7.1
||
||
||
Client Workstation 192.168.7.110
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will exit 0 on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# GRE Tunnel :
echo Loading modules..
modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre
LOCALIP=192.168.8.21
FORTIDIRIP=192.168.8.1
FORTIIPID=XX.XX.XXX.254
echo changing routing and reverse path stuff..
echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter
echo 1 /proc/sys/net/ipv4/ip_forward
echo creating tunnel...
iptunnel add wccp0 mode gre remote $FORTIIPID local $LOCALIP dev eth0
ifconfig wccp0 127.0.1.1/32 up
echo creating routing table for tproxy...
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
echo creating iptables tproxy rules...
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
iptables -A FORWARD -i lo -j ACCEPT
iptables -A INPUT -s $FORTIDIRIP -p udp -m udp --dport 2048 -j ACCEPT
iptables -A INPUT -i wccp0 -j ACCEPT
iptables -A INPUT -p gre -j ACCEPT
iptables -t mangle -F
iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3228
iptables -t mangle -A PREROUTING -p tcp --dport 443 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3229
exit 0
squid.conf
---
#debug_options ALL,1 33,2
#debug_options ALL,1 33,2 28,9
hierarchy_stoplist cgi-bin
acl QUERY urlpath_regex cgi-bin
#cache_effective_user squid
shutdown_lifetime 1 second
visible_hostname server
httpd_suppress_version_string on
forwarded_for off
#1GB disk cache
cache_dir ufs /usr/local/var/cache/squid 1024 16 256
maximum_object_size 5 MB
cache_mem 1024 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size_in_memory 512 KB
request_header_access Referer deny all
reply_header_access Referer deny all
http_port 80 accel
acl site1 dstdomain site1.domain.com
acl site2 dstdomain site2.domain.com
acl site3 dstdomain site3.domain.com
acl site4 dstdomain site4.domain.com
acl site5 dstdomain site5.domain.com
acl site6 dstdomain site6.domain.com
acl site7 dstdomain site7.domain.com
https_port 443 cert=/etc/ssl/certs/domain_sites.crt
key=/etc/ssl/private/domain.key accel vport
# never_direct allow site1
always_direct allow site1
http_access allow site1
http_access deny site1
always_direct allow site2
http_access allow site2
http_access deny site2
always_direct allow site3
http_access allow site3
http_access deny site3
always_direct allow site4
http_access allow site4
http_access deny site4
always_direct allow site5
http_access allow site5
http_access deny site5
always_direct allow site6
http_access allow site6
http_access deny site6
always_direct allow site7
http_access allow site7
http_access deny site7
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src {WAN Network} # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly
plugged) machines
acl SSL_ports port 443
acl SSL_ports port 4435
acl SSL_ports port 8443
acl Safe_ports port 80 # http
acl Safe_ports port 8080 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl
Re: [squid-users] tproxy configuration
On 22/02/2013 11:03 a.m., Roman Gelfand wrote:
Please, find below the network topology, squid.conf and rc.local
configuration files. It appears that the squid is not routing the
http requests. I am not sure what I am doing wrong here
Please note, the same squid.conf works on transparent proxy (non
tproxy), for the exception of tproxy keyword and service changes.
Thanks in advance,
WAN
||
||
wccp/gre tunnel ||
squid==Fortigate FW/RT Int ip 1 192.168.8.1
3.3|| Int ip 2 192.168.11.1
ip: 192.168.8.21 || Ext ip XX.XX.XXX.24
||
||
WLAN Router Int. ip
192.168.11.32 Ext. ip 192.168.7.1
||
||
||
Client Workstation 192.168.7.110
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will exit 0 on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# GRE Tunnel :
echo Loading modules..
modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre
LOCALIP=192.168.8.21
FORTIDIRIP=192.168.8.1
FORTIIPID=XX.XX.XXX.254
echo changing routing and reverse path stuff..
echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter
What about rp_filter on eth0 where the traffic is actually exiting the
Squid box?
echo 1 /proc/sys/net/ipv4/ip_forward
echo creating tunnel...
iptunnel add wccp0 mode gre remote $FORTIIPID local $LOCALIP dev eth0
ifconfig wccp0 127.0.1.1/32 up
echo creating routing table for tproxy...
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
You may need this to be dev eth0 instead of dev lo. Experiment to find
out which.
echo creating iptables tproxy rules...
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
iptables -A FORWARD -i lo -j ACCEPT
What about forwarding of non-localhost traffic? such as the TPROXY
spoofed client IPs.
iptables -A INPUT -s $FORTIDIRIP -p udp -m udp --dport 2048 -j ACCEPT
iptables -A INPUT -i wccp0 -j ACCEPT
iptables -A INPUT -p gre -j ACCEPT
iptables -t mangle -F
iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3228
iptables -t mangle -A PREROUTING -p tcp --dport 443 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3229
exit 0
squid.conf
---
#debug_options ALL,1 33,2
#debug_options ALL,1 33,2 28,9
hierarchy_stoplist cgi-bin
acl QUERY urlpath_regex cgi-bin
#cache_effective_user squid
shutdown_lifetime 1 second
visible_hostname server
httpd_suppress_version_string on
forwarded_for off
#1GB disk cache
cache_dir ufs /usr/local/var/cache/squid 1024 16 256
maximum_object_size 5 MB
cache_mem 1024 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size_in_memory 512 KB
request_header_access Referer deny all
reply_header_access Referer deny all
http_port 80 accel
acl site1 dstdomain site1.domain.com
acl site2 dstdomain site2.domain.com
acl site3 dstdomain site3.domain.com
acl site4 dstdomain site4.domain.com
acl site5 dstdomain site5.domain.com
acl site6 dstdomain site6.domain.com
acl site7 dstdomain site7.domain.com
https_port 443 cert=/etc/ssl/certs/domain_sites.crt
key=/etc/ssl/private/domain.key accel vport
# never_direct allow site1
always_direct allow site1
http_access allow site1
http_access deny site1
always_direct allow site2
http_access allow site2
http_access deny site2
always_direct allow site3
http_access allow site3
http_access deny site3
always_direct allow site4
http_access allow site4
http_access deny site4
always_direct allow site5
http_access allow site5
http_access deny site5
always_direct allow site6
http_access allow site6
http_access deny site6
always_direct allow site7
http_access allow site7
http_access deny site7
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src {WAN Network} # RFC1918 possible internal network
acl localnet src
Re: [squid-users] tproxy configuration
On Thu, Feb 21, 2013 at 6:10 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 11:03 a.m., Roman Gelfand wrote: Please, find below the network topology, squid.conf and rc.local configuration files. It appears that the squid is not routing the http requests. I am not sure what I am doing wrong here Please note, the same squid.conf works on transparent proxy (non tproxy), for the exception of tproxy keyword and service changes. Thanks in advance, WAN || || wccp/gre tunnel || squid==Fortigate FW/RT Int ip 1 192.168.8.1 3.3|| Int ip 2 192.168.11.1 ip: 192.168.8.21 || Ext ip XX.XX.XXX.24 || || WLAN Router Int. ip 192.168.11.32 Ext. ip 192.168.7.1 || || || Client Workstation 192.168.7.110 #!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will exit 0 on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. # GRE Tunnel : echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre LOCALIP=192.168.8.21 FORTIDIRIP=192.168.8.1 FORTIIPID=XX.XX.XXX.254 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter What about rp_filter on eth0 where the traffic is actually exiting the Squid box? Could you elaborate on this.. echo 1 /proc/sys/net/ipv4/ip_forward echo creating tunnel... iptunnel add wccp0 mode gre remote $FORTIIPID local $LOCALIP dev eth0 ifconfig wccp0 127.0.1.1/32 up echo creating routing table for tproxy... ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 You may need this to be dev eth0 instead of dev lo. Experiment to find out which. echo creating iptables tproxy rules... iptables -A INPUT -i lo -j ACCEPT iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT iptables -A FORWARD -i lo -j ACCEPT What about forwarding of non-localhost traffic? such as the TPROXY spoofed client IPs. Could you elaborate on this, as well. iptables -A INPUT -s $FORTIDIRIP -p udp -m udp --dport 2048 -j ACCEPT iptables -A INPUT -i wccp0 -j ACCEPT iptables -A INPUT -p gre -j ACCEPT iptables -t mangle -F iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3228 iptables -t mangle -A PREROUTING -p tcp --dport 443 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3229 exit 0 squid.conf --- #debug_options ALL,1 33,2 #debug_options ALL,1 33,2 28,9 hierarchy_stoplist cgi-bin acl QUERY urlpath_regex cgi-bin #cache_effective_user squid shutdown_lifetime 1 second visible_hostname server httpd_suppress_version_string on forwarded_for off #1GB disk cache cache_dir ufs /usr/local/var/cache/squid 1024 16 256 maximum_object_size 5 MB cache_mem 1024 MB cache_swap_low 90 cache_swap_high 95 maximum_object_size_in_memory 512 KB request_header_access Referer deny all reply_header_access Referer deny all http_port 80 accel acl site1 dstdomain site1.domain.com acl site2 dstdomain site2.domain.com acl site3 dstdomain site3.domain.com acl site4 dstdomain site4.domain.com acl site5 dstdomain site5.domain.com acl site6 dstdomain site6.domain.com acl site7 dstdomain site7.domain.com https_port 443 cert=/etc/ssl/certs/domain_sites.crt key=/etc/ssl/private/domain.key accel vport # never_direct allow site1 always_direct allow site1 http_access allow site1 http_access deny site1 always_direct allow site2 http_access allow site2 http_access deny site2 always_direct allow site3 http_access allow site3 http_access deny site3 always_direct allow site4 http_access allow site4 http_access deny site4 always_direct allow site5 http_access allow site5 http_access deny site5 always_direct allow site6 http_access allow site6 http_access deny site6 always_direct allow site7 http_access allow site7 http_access deny site7 # # Recommended minimum configuration: # # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src
Re: [squid-users] tproxy configuration
On 22/02/2013 5:07 p.m., Roman Gelfand wrote: On Thu, Feb 21, 2013 at 6:10 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 22/02/2013 11:03 a.m., Roman Gelfand wrote: Please, find below the network topology, squid.conf and rc.local configuration files. It appears that the squid is not routing the http requests. I am not sure what I am doing wrong here Please note, the same squid.conf works on transparent proxy (non tproxy), for the exception of tproxy keyword and service changes. Thanks in advance, WAN || || wccp/gre tunnel || squid==Fortigate FW/RT Int ip 1 192.168.8.1 3.3|| Int ip 2 192.168.11.1 ip: 192.168.8.21 || Ext ip XX.XX.XXX.24 || || WLAN Router Int. ip 192.168.11.32 Ext. ip 192.168.7.1 || || || Client Workstation 192.168.7.110 #!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will exit 0 on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. # GRE Tunnel : echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre LOCALIP=192.168.8.21 FORTIDIRIP=192.168.8.1 FORTIIPID=XX.XX.XXX.254 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter What about rp_filter on eth0 where the traffic is actually exiting the Squid box? Could you elaborate on this.. What rp_filter does is prevent packets from local software using that interface from using IP addresses that do not belong to that box. The purpose of TPROXY being to spoof the _clients_ IP address on outgoing trafffic. Which does not leave the machine on lo, but through eth0 or some other interface. Amos
[squid-users] TPROXY Configuration
I have configured the tproxy as follows, but it appears packets are
not hitting squid. Please note, the wccp configuration on the router
is already working with squid http_port transparent configuration and,
obviously, different iptables configuration. Any help is appreciated.
Thanks in advance.
squid.conf
---
http_port 3228 tproxy
https_port 3229 tproxy ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/ssl/certs/domain.crt
key=/etc/ssl/private/domain.key
# FortiGate interface of wccp
wccp2_router 192.168.5.1
wccp2_service dynamic 90
wccp2_service_info 90 protocol=tcp flags=src_ip_hash priority=240 ports=80,443
wccp2_service dynamic 95
wccp2_service_info 95 protocol=tcp flags=dst_ip_hash,ports_source
priority=240 ports=80,443
# tunneling method GRE for forward traffic
wccp2_forwarding_method 1
# tunneling method GRE for return traffic
wccp2_return_method 1
# Assignemment method (default), only relevant if multiple caches used
wccp2_assignment_method 1
# wccp weight (default) ,only relevant if multiple caches used
wccp2_weight 1
# which interface to use for WCCP (0.0.0.0 determines the interface
from routing)
wccp2_address 0.0.0.0
rc.local
---
modprobe ip_gre
modprobe ip_tables
modprobe x_tables
ip tunnel add wccp0 mode gre remote 192.168.5.1 local 192.168.5.21 dev eth0
ip addr add 192.168.5.21/32 dev wccp0
ip link set wccp0 up
# Route to send the content back to the GRE tunnel
route add -net {wan interface ip} netmask 255.255.255.255 dev wccp0
# Disabling reverse path filtering and enable routing in the kernel
echo 0 /proc/sys/net/ipv4/conf/wccp0/rp_filter
echo 1 /proc/sys/net/ipv4/ip_forward
# Setup the redirection of traffic from the GRE tunnel to squid port 3128
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -i wccp0 -p tcp --dport 80 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port 3228
iptables -t mangle -A PREROUTING -i wccp0 -p tcp --dport 443 -j
TPROXY --tproxy-mark 0x1/0x1 --on-port 3229
exit 0
Re: [squid-users] tproxy configuration
On 7/17/2012 6:01 PM, Wayne Lee wrote: SNIP Many thanks Eliezer. I still have the same issue in that once the packets arrive on the squid box they are not actually diverted into the squid daemon and thus fail. I have managed to find a working solution and that is to not use wccp and just built a proper gre tunnel between the squid and cisco router, the DNAT/Redirect methods then work as expected. Thanks again Wayne if the packets are not diverted into squid there is something wrong with your setup. if you will post your squid config routes and iptables i will might be able to help you. for me squid works with wither tproxy\dnat\redirect + wccp or with basic routing rules. Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
Re: [squid-users] tproxy configuration
Sent from my iPad On 19 Jul 2012, at 19:58, Eliezer Croitoru elie...@ngtech.co.il wrote: On 7/17/2012 6:01 PM, Wayne Lee wrote: SNIP Many thanks Eliezer. I still have the same issue in that once the packets arrive on the squid box they are not actually diverted into the squid daemon and thus fail. I have managed to find a working solution and that is to not use wccp and just built a proper gre tunnel between the squid and cisco router, the DNAT/Redirect methods then work as expected. Thanks again Wayne if the packets are not diverted into squid there is something wrong with your setup. if you will post your squid config routes and iptables i will might be able to help you. for me squid works with wither tproxy\dnat\redirect + wccp or with basic routing rules. Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il Hello I followed your guide on this post although I swapped the wccp redirect statements around You had ip wccp 80 redirect out ip wccp 90 redirect in I changed it to ip wccp 90 redirect in ip wccp 80 redirect out No traffic was being redirected down the wccp until I changed it. Everything else was as you posted. Packets were not being diverted or tproxy'ed into squid which has been my issue all along. I'm happy to set it up that way again and provide whatever debug output required as I would prefer the wccp for failover purposes. Any/all help and guidance is appreciated. Regards Wayne
Re: [squid-users] tproxy configuration
On 7/20/2012 12:35 AM, Wayne Lee wrote: Sent from my iPad On 19 Jul 2012, at 19:58, Eliezer Croitoru elie...@ngtech.co.il wrote: On 7/17/2012 6:01 PM, Wayne Lee wrote: SNIP if the packets are not diverted into squid there is something wrong with your setup. if you will post your squid config routes and iptables i will might be able to help you. for me squid works with wither tproxy\dnat\redirect + wccp or with basic routing rules. Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il Hello I followed your guide on this post although I swapped the wccp redirect statements around You had ip wccp 80 redirect out ip wccp 90 redirect in I changed it to ip wccp 90 redirect in ip wccp 80 redirect out No traffic was being redirected down the wccp until I changed it. Everything else was as you posted. Packets were not being diverted or tproxy'ed into squid which has been my issue all along. I'm happy to set it up that way again and provide whatever debug output required as I would prefer the wccp for failover purposes. Any/all help and guidance is appreciated. Regards Wayne well the order dosnt matter because it's evaluated based on the IN and OUT status. so in any case you will put in before or after 90 the check wont be applied as IN on OUT. it's a one way check. anyway i'm happy it works good for you. i wrote a wiki page about how to set it up with a very nice diagram of the topology at: http://wiki.squid-cache.org/ConfigExamples/UbuntuTproxy4Wccp2 i was thinking about people that runs web cache with a linux router and not a Cisco device. They do not have this kind of a solution so i was thinking of writing some scripts and a small daemons pair. one for the linux router and the other for the cache servers. it will manage packet marking on iptables PREROUTING table with maybe some additional dynamic tables. and the other on the squid box to identify the it is still there and running. based on wccp methods it's pretty simply to implement. wccp is a binary protocol while i was thinking to implement it based on text + basic encryption option. i wrote already a nice pair of helpers that checks if a cache peer is runing and well. so it's only matter of signaling the current status from the cache to the router every specific predefined interval and making sure that the settings are intact. this guy wrote POTATO: https://github.com/wisq/potato with web interface and stuff for load balancing couple dsl line. the idea is kind the same and i think i can make it useful. Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
Re: [squid-users] tproxy configuration
what do you want to achieve? everything that can be achieved using WCCP can be achieved in other way with tproxy. Eliezer WCCP makes sure that only traffic on some specific ports(generally port 80 and 443) goes via Squid box. In case of tproxy all the traffic will flow via squid box if it has to work as a router and that may affect the performance of the squid box. I don't know if the tproxy feature can be achieved without making the squid box a router. -- Regards, Nipun Talukdar Bangalore
Re: [squid-users] tproxy configuration
Eliezer, Thanks for the links. The diagram in the first link is good but I don't know to read that language. Also, squid has a bug regarding its tproxy feature, it never spoofs the client IP. I made a small fix for that issue, but that was one year back and I lost the code with the fix. Regards, Nipun Talukdar Bangalore India i have found the way to make it all work together. this site shows perfectly how to set it up all together: http://bloggik.net/index.php/articles/networks/18-cisco/38-squid-tproxy-wccp the i got the source squid-users post at: http://www.squid-cache.org/mail-archive/squid-users/200906/0602.html http://www.mail-archive.com/squid-users@squid-cache.org/msg65056.html i will write it in the wiki all together back in Plain English that will explain all the things you need to take in account when implementing it and what can go wrong with it. ELiezer
Re: [squid-users] tproxy configuration
On 17/07/2012 10:44 p.m., nipun_mlist Assam wrote: what do you want to achieve? everything that can be achieved using WCCP can be achieved in other way with tproxy. Eliezer WCCP makes sure that only traffic on some specific ports(generally port 80 and 443) goes via Squid box. In case of tproxy all the traffic will flow via squid box if it has to work as a router and that may affect the performance of the squid box. The Squid box always has to do routing, even as a regluar proxy. How do you expect the packets to flow through it unless they are routed to their destinations? All the traffic is wrong. The Squid box does *not* have to route everything on the network. Nor even does it have to see anything beyond port 80 traffic. The rules you place on the Cisco decide what packets goes to the Squid box. WCCP is just a tunnel and special routing table. You create regular policy routing to pass packets through the WCCP GRE tunnel, you can do the same with a regular interface/outerface straight to pass only only port 80 or 443 packets to a Squid box router without WCCP. The only thing WCCP actually gains you is ability to split between multiple caches and easy failover when the cache(s) go down. I don't know if the tproxy feature can be achieved without making the squid box a router. No it can't. You just have to understand what a router *is* a bit better. When the packets arrive they are addressed to places which are not the Squid box. The kernel TCP security will only allow non-local packets to enter a box which is a router or bridge. If you choose bridge the packets have to be shifted into router mode for the box NAT systems to recieve. So either way you need routing just to receive the packets into Squid. TPROXY is a method of interception which preserves the client IP (or IPv6) as if the proxy was not there. Two abilities which NAT interception cannot provide. It still receives packets from the packet routing system of the kernel just like NAT. Amos
Re: [squid-users] tproxy configuration
On 7/17/2012 2:07 PM, nipun_mlist Assam wrote: Eliezer, Thanks for the links. The diagram in the first link is good but I don't know to read that language. Also, squid has a bug regarding its tproxy feature, it never spoofs the client IP. I made a small fix for that issue, but that was one year back and I lost the code with the fix. Regards, Nipun Talukdar Bangalore India SNIP there is no problem with squid and tproxy that wont spoof clients ip if i will add it later to squid wiki. diagram of the network: http://www1.ngtech.co.il/squid/wccp2.pngsetup correctly. squid config: ##start wccp2.sh #!/usr/bin/bash echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre gre LOCALIP=10.80.2.2 CISCODIRIP=10.80.2.1 #you must connect the gre tunnel to the cisco router IP identifier. CISCOIPID=192.168.10.127 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter echo 1 /proc/sys/net/ipv4/ip_forward echo creating tunnel... iptunnel add wccp0 mode gre remote $CISCOIPID local $LOCALIP dev eth1 ifconfig wccp0 127.0.1.1/32 up echo creating routing table for tproxy... ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 echo creating iptables tproxy rules... iptables -A INPUT -i lo -j ACCEPT iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT iptables -A FORWARD -i lo -j ACCEPT iptables -A INPUT -s $CISCODIRIP -p udp -m udp --dport 2048 -j ACCEPT iptables -A INPUT -i wccp0 -j ACCEPT iptables -A INPUT -p gre -j ACCEPT iptables -t mangle -F iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 ##end ##start add to squid.conf wccp2_router 10.80.2.1 wccp_version 2 wccp2_rebuild_wait on wccp2_forwarding_method gre wccp2_return_method gre wccp2_service standard 0 wccp2_service dynamic 80 wccp2_service dynamic 90 wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80 wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source priority=240 ports=80 ##end ##cisco config conf t ip access-list extended wccp permit ip 10.80.3.0 0.0.0.255 any ip access-list extended wccp_to_inside permit ip any 10.80.3.0 0.0.0.255 exit ip wccp 80 redirect-list wccp ip wccp 90 redirect-list wccp_to_inside !gw interface interface FastEthernet0/0.1 encapsulation dot1Q 1 native ip address 192.168.10.127 255.255.255.0 ip wccp 80 redirect out ip wccp 90 redirect in exit !proxy interface interface FastEthernet0/0.100 encapsulation dot1Q 100 ip address 10.80.2.1 255.255.255.0 ip wccp redirect exclude in exit !clients interface interface FastEthernet0/0.200 encapsulation dot1Q 200 ip address 10.80.3.1 255.255.255.0 exit !rotue to internet gw ip route 0.0.0.0 0.0.0.0 192.168.10.201 end ##cisco config end -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
Re: [squid-users] tproxy configuration
SNIP squid config: ##start wccp2.sh #!/usr/bin/bash echo Loading modules.. modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre gre LOCALIP=10.80.2.2 CISCODIRIP=10.80.2.1 #you must connect the gre tunnel to the cisco router IP identifier. CISCOIPID=192.168.10.127 echo changing routing and reverse path stuff.. echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter echo 1 /proc/sys/net/ipv4/ip_forward echo creating tunnel... iptunnel add wccp0 mode gre remote $CISCOIPID local $LOCALIP dev eth1 ifconfig wccp0 127.0.1.1/32 up echo creating routing table for tproxy... ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 echo creating iptables tproxy rules... iptables -A INPUT -i lo -j ACCEPT iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT iptables -A FORWARD -i lo -j ACCEPT iptables -A INPUT -s $CISCODIRIP -p udp -m udp --dport 2048 -j ACCEPT iptables -A INPUT -i wccp0 -j ACCEPT iptables -A INPUT -p gre -j ACCEPT iptables -t mangle -F iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 3129 ##end ##start add to squid.conf wccp2_router 10.80.2.1 wccp_version 2 wccp2_rebuild_wait on wccp2_forwarding_method gre wccp2_return_method gre wccp2_service standard 0 wccp2_service dynamic 80 wccp2_service dynamic 90 wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80 wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source priority=240 ports=80 ##end ##cisco config conf t ip access-list extended wccp permit ip 10.80.3.0 0.0.0.255 any ip access-list extended wccp_to_inside permit ip any 10.80.3.0 0.0.0.255 exit ip wccp 80 redirect-list wccp ip wccp 90 redirect-list wccp_to_inside !gw interface interface FastEthernet0/0.1 encapsulation dot1Q 1 native ip address 192.168.10.127 255.255.255.0 ip wccp 80 redirect out ip wccp 90 redirect in exit !proxy interface interface FastEthernet0/0.100 encapsulation dot1Q 100 ip address 10.80.2.1 255.255.255.0 ip wccp redirect exclude in exit !clients interface interface FastEthernet0/0.200 encapsulation dot1Q 200 ip address 10.80.3.1 255.255.255.0 exit !rotue to internet gw ip route 0.0.0.0 0.0.0.0 192.168.10.201 end ##cisco config end Many thanks Eliezer. I still have the same issue in that once the packets arrive on the squid box they are not actually diverted into the squid daemon and thus fail. I have managed to find a working solution and that is to not use wccp and just built a proper gre tunnel between the squid and cisco router, the DNAT/Redirect methods then work as expected. Thanks again Wayne
[squid-users] tproxy configuration
Hi, Is there anyway to use squid tproxy feature without configuring the squid box as a router ? Is it possible to use WCCP + tproxy combination to achieve the same? -- Regards, Nipun Talukdar Bangalore
Re: [squid-users] tproxy configuration
On 7/16/2012 1:47 PM, nipun_mlist Assam wrote: Hi, Is there anyway to use squid tproxy feature without configuring the squid box as a router ? Bridge. Is it possible to use WCCP + tproxy combination to achieve the same? there is an example at: http://code.google.com/p/lusca-cache/wiki/ExampleTproxy4Linux that lusca guy did. if you do ask me with all the features that cisco devices have the WCCP is nice but i prefer a more explicit way then auto setup. i have tried using this example for about 4 hours and to make WCCP somehow work but it seems like wither i did something wrong or it's not possible. what do you want to achieve? everything that can be achieved using WCCP can be achieved in other way with tproxy. Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
Re: [squid-users] tproxy configuration
On 7/16/2012 1:47 PM, nipun_mlist Assam wrote: Hi, Is there anyway to use squid tproxy feature without configuring the squid box as a router ? Is it possible to use WCCP + tproxy combination to achieve the same? well after digging i have found it unclear at all how the squid wiki examples explains WCCP and TPROXY. i have found the way to make it all work together. this site shows perfectly how to set it up all together: http://bloggik.net/index.php/articles/networks/18-cisco/38-squid-tproxy-wccp the i got the source squid-users post at: http://www.squid-cache.org/mail-archive/squid-users/200906/0602.html http://www.mail-archive.com/squid-users@squid-cache.org/msg65056.html i will write it in the wiki all together back in Plain English that will explain all the things you need to take in account when implementing it and what can go wrong with it. ELiezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
[squid-users] tproxy configuration
Hi, I want to deploy tproxy in my network .I m using rhel 5.5.Please provide me good document or configuration guide for getting good explanation.I m new to tproxy. And please suggest me for the same, means what are the caviates , kept in mind while using tproxy. Thanks, Benjo
Re: [squid-users] tproxy configuration
On 28/12/10 00:55, benjamin fernandis wrote: Hi, I want to deploy tproxy in my network .I m using rhel 5.5.Please provide me good document or configuration guide for getting good explanation.I m new to tproxy. Step #1: upgrade. And please suggest me for the same, means what are the caviates , kept in mind while using tproxy. TPROXYv2 which is likely the only one available with RHEL 5.x old software requires kernel patching and is no longer supported by any of the authors. Unless you are required to go to those great lengths to do it you may as well upgrade to a newer kernel-2.6.32+ and use a Squid-3.1+ with TPROXYv4 support. http://wiki.squid-cache.org/Features/Tproxy4 Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.10 Beta testers wanted for 3.2.0.4
