[squid-users] Squid 2.7 on Windows XP does not work
I have built Squid 2.7 STEABLE 9 with mingw on windows followed the instruction in wiki. After that, I ran command squid -z The CPU was occupied 100%. Command 'squid' and 'squid -k parse' does not work, too.
[squid-users] Antwort: Re: [squid-users] Question to cache_peer
Nyamul Hassan nya...@gmail.com schrieb am 03.07.2014 22:04:10: Von: Nyamul Hassan nya...@gmail.com An: andreas.resc...@mahle.com Kopie: Squid Users squid-users@squid-cache.org Datum: 03.07.2014 22:05 Betreff: Re: [squid-users] Question to cache_peer Hi Andreas, As per the wiki page http://wiki.squid-cache.org/Features/CacheHierarchy, did you try with the following two lines in your squid.conf: cache_peer parentcache.foo.com parent 3128 0 no-query default never_direct allow all That 2nd line forces the child to only talk to the parent. Regards HASSAN On Thu, Jul 3, 2014 at 8:49 PM, andreas.resc...@mahle.com wrote: Hi there, I've configured my squid as follow: cache_peer xxx.xxx.xxx.xxx parent 3128 0 no-query no-digest But a lot of traffic didn't pass the parent proxy. Why? How can i configure the child proxy that ALL traffic must pass the parent proxy? Mit freundlichen Grüßen / Kind regards Mr. Andreas Reschke andreas.resc...@mahle.com, http://www.mahle.com Hello Nyamul, I know this page and tried this line, but it doesn't work. The most traffic and all the https traffic goes direct and not through the parent. Mit freundlichen Grüßen / Kind regards Mr. Andreas Reschke andreas.resc...@mahle.com, http://www.mahle.com
[squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
Hassan definitely is correct. So, may be you just use a working config before trying alternatives: #ALL your ACL's first in squid.conf ! . cache_peer xx.xx.xx.xx parent 6139 0 no-query no-digest no-netdb-exchange never_direct allow all If this does not work, pls post your squid.conf again, as there were a few other annoyances. Any special messages in cache.log ? -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Question-to-cache-peer-tp416p441.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
babajaga augustus_me...@yahoo.de schrieb am 04.07.2014 09:49:24: Von: babajaga augustus_me...@yahoo.de An: squid-users@squid-cache.org Datum: 04.07.2014 09:51 Betreff: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer Hassan definitely is correct. So, may be you just use a working config before trying alternatives: #ALL your ACL's first in squid.conf ! . cache_peer xx.xx.xx.xx parent 6139 0 no-query no-digest no-netdb-exchange never_direct allow all If this does not work, pls post your squid.conf again, as there were a few other annoyances. Any special messages in cache.log ? -- View this message in context: http://squid-web-proxy-cache. 1019090.n4.nabble.com/Question-to-cache-peer-tp416p441.html Sent from the Squid - Users mailing list archive at Nabble.com. Hi there, I've taken this 2 lines at the end of the config no impact ! My squid.conf bgstproxyls01:~ # cat /etc/squid/squid.conf # # Recommended minimum configuration: # acl snmppublic snmp_community squid snmp_port 3401 snmp_incoming_address xxx.xxx.xxx.xxx snmp_outgoing_address xxx.xxx.xxx.xxx snmp_access allow all client_db off half_closed_clients off via off cache_mem 4096 MB ipcache_size 2028 fqdncache_size 2048 hosts_file /etc/hosts memory_pools off maximum_object_size 50 MB quick_abort_min 0 KB quick_abort_max 0 KB log_icp_queries off buffered_logs on #maximum_object_size 50 MB dns_nameservers xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx dns_nameservers xxx.xxx.xxx.xxx # acl manager proto cache_object # acl localhost src 127.0.0.1 # ::1 # acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 # ::1 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # neu acl SSL method CONNECT acl CONNECT method CONNECT # erlaubte Seiten ohne Internetberechtigung acl open-sites dstdomain /etc/squid/open-sites.txt # erlaubte Seiten ohne Internetberechtigung # verbotene Seiten acl denied-sites url_regex /etc/squid/denied-sites.txt acl selling-sites url_regex /etc/squid/selling-sites.txt acl social-sites url_regex /etc/squid/social-sites.txt # verbotene Seiten acl allowedurls dstdomain /etc/squid/bypass.txt external_acl_type LDAPLookup children-startup=10 children-idle=30 children-max=80 ttl=600 negative_ttl=30 %LOGIN /usr/sbin/ext_ldap_group_acl -d -b dc=behrgroup,dc=net -D CN=BGST-S-SQUID,OU=Service Accounts,OU=bgst,OU=de,DC=behrgroup,DC=net -W /etc/squid/ppp -f ((objectclass=user)(sAMAccountName=%v)(memberof:1.2.840.113556.1.4.1941:=CN=%a,OU=groups,OU=Proxy,OU=Global Groups,DC=behrgroup,dc=net)) -h xxx.xxx.xxx.xxx ## DEBUGGING #debug_options 28,9 # local manager http_access allow manager localhost http_access deny manager # nur safe SSL ab hier http_access deny !Safe_ports http_access deny CONNECT !SSL_ports deny_info http://bgstproxyls01/denied.html denied-sites # Squid normally listens to port 3128 http_port 3128 # We recommend you to use at least the following line. hierarchy_stoplist cgi-bin ? # Leave coredumps in the first cache dir coredump_dir /var/cache/squid # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 ### pure ntlm authentication auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --domain=BEHRGROUP.NET auth_param ntlm children 128 auth_param ntlm keep_alive off # Zeit-Steuerung für Indien acl indien proxy_auth external LDAPLookup GGPY-LO-Web-Time-Limited acl DAY time 05:30-13:30 # Zeit-Steuerung für Indien acl chkglwebhttp external LDAPLookup GGPY-LO-Web-Http acl sellingUser external LDAPLookup GGPY-LO-Web-Allowed-Selling acl socialUser external LDAPLookup GGPY-LO-Web-Allowed-Social acl allforbUser external LDAPLookup GGPY-LO-Web-Allowed-All acl ftpputUser external LDAPLookup GGPY-LO-Web-Ftp-Put acl loggingUser external LDAPLookup GGPY-LO-Web-Log-User acl auth proxy_auth REQUIRED #
Re: [squid-users] Squid 2.7 on Windows XP does not work
Squid -z usually consumes a lot of CPU cycles when it builds the storage, especially COSS. Can you share your cache_dir lines from squid.conf? Regards HASSAN On Fri, Jul 4, 2014 at 12:39 PM, Kzjnet kzj...@kzjnet.com wrote: I have built Squid 2.7 STEABLE 9 with mingw on windows followed the instruction in wiki. After that, I ran command squid -z The CPU was occupied 100%. Command 'squid' and 'squid -k parse' does not work, too.
[squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
OK, then we will have a look at the ACL-decisions (often a problem) and the peer selection within squid, using debug_options ALL,5 33,2 28,9 44,3 in squid.conf This will produce a detailed log about ACL processing, and peer selection, which is the most interesting. It will cause a lot of output to cache.log, so only to use it for a short period of time. In cache.log then simply search for peer_select and have a look around, why the parent cache is not chosen. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Question-to-cache-peer-tp416p444.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
babajaga augustus_me...@yahoo.de schrieb am 04.07.2014 11:06:24: Von: babajaga augustus_me...@yahoo.de An: squid-users@squid-cache.org Datum: 04.07.2014 11:08 Betreff: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer OK, then we will have a look at the ACL-decisions (often a problem) and the peer selection within squid, using debug_options ALL,5 33,2 28,9 44,3 in squid.conf This will produce a detailed log about ACL processing, and peer selection, which is the most interesting. It will cause a lot of output to cache.log, so only to use it for a short period of time. In cache.log then simply search for peer_select and have a look around, why the parent cache is not chosen. -- View this message in context: http://squid-web-proxy-cache. 1019090.n4.nabble.com/Question-to-cache-peer-tp416p444.html Sent from the Squid - Users mailing list archive at Nabble.com. Hi babajaga (What a name ;-) log file looks fine: 2014/07/04 11:11:28.540 kid1| peer_select.cc(151) peerSelect: peerSelect: http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg 2014/07/04 11:11:28.540 kid1| peer_select.cc(435) peerSelectFoo: peerSelectFoo: 'GET ecx.images-amazon.com' 2014/07/04 11:11:28.540 kid1| peer_select.cc(446) peerSelectFoo: peerSelectFoo: direct = DIRECT_UNKNOWN (never_direct to be checked) 2014/07/04 11:11:28.540 kid1| peer_select.cc(184) peerCheckNeverDirectDone: peerCheckNeverDirectDone: ALLOWED 2014/07/04 11:11:28.540 kid1| peer_select.cc(190) peerCheckNeverDirectDone: direct = DIRECT_NO (never_direct allow) 2014/07/04 11:11:28.540 kid1| peer_select.cc(435) peerSelectFoo: peerSelectFoo: 'GET ecx.images-amazon.com' 2014/07/04 11:11:28.541 kid1| peer_select.cc(125) peerSelectIcpPing: peerSelectIcpPing: http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg 2014/07/04 11:11:28.541 kid1| peer_select.cc(136) peerSelectIcpPing: peerSelectIcpPing: counted 0 neighbors 2014/07/04 11:11:28.541 kid1| peer_select.cc(675) peerGetSomeParent: peerGetSomeParent: GET ecx.images-amazon.com 2014/07/04 11:11:28.541 kid1| peer_select.cc(699) peerGetSomeParent: peerSelect: FIRSTUP_PARENT/194.99.121.200 2014/07/04 11:11:28.541 kid1| peer_select.cc(724) peerGetAllParents: peerGetAllParents: adding alive parent 194.99.121.200 2014/07/04 11:11:28.541 kid1| peer_select.cc(265) peerSelectDnsPaths: Find IP destination for: http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg' via 194.99.121.200 2014/07/04 11:11:28.541 kid1| peer_select.cc(265) peerSelectDnsPaths: Find IP destination for: http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg' via 194.99.121.200 2014/07/04 11:11:28.541 kid1| peer_select.cc(286) peerSelectDnsPaths: Found sources for 'http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg' 2014/07/04 11:11:28.541 kid1| peer_select.cc(287) peerSelectDnsPaths: always_direct = DENIED 2014/07/04 11:11:28.541 kid1| peer_select.cc(288) peerSelectDnsPaths: never_direct = ALLOWED 2014/07/04 11:11:28.541 kid1| peer_select.cc(298) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=194.99.121.200:3128 flags=1 2014/07/04 11:11:28.541 kid1| peer_select.cc(298) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=194.99.121.200:3128 flags=1 2014/07/04 11:11:28.541 kid1| peer_select.cc(301) peerSelectDnsPaths: timedout = 0 2014/07/04 11:11:28.541 kid1| peer_select.cc(94) ~ps_state: http://ecx.images-amazon.com/images/I/51F9heD9xiL._AC_AC_BR255_GR_SL50_.jpg 2014/07/04 11:11:28.563 kid1| peer_select.cc(151) peerSelect: peerSelect: 2014/07/04 11:11:28.598 kid1| peer_select.cc(446) peerSelectFoo: peerSelectFoo: direct = DIRECT_UNKNOWN (never_direct to be checked) 2014/07/04 11:11:28.598 kid1| peer_select.cc(184) peerCheckNeverDirectDone: peerCheckNeverDirectDone: ALLOWED 2014/07/04 11:11:28.598 kid1| peer_select.cc(190) peerCheckNeverDirectDone: direct = DIRECT_NO (never_direct allow) 2014/07/04 11:11:28.598 kid1| peer_select.cc(435) peerSelectFoo: peerSelectFoo: 'GET gdecz.hit.gemius.pl' 2014/07/04 11:11:28.598 kid1| peer_select.cc(125) peerSelectIcpPing: peerSelectIcpPing: http://gdecz.hit.gemius.pl/gdejs/inscreen_lib.js 2014/07/04 11:11:28.598 kid1| peer_select.cc(136) peerSelectIcpPing: peerSelectIcpPing: counted 0 neighbors 2014/07/04 11:11:28.598 kid1| peer_select.cc(675) peerGetSomeParent: peerGetSomeParent: GET gdecz.hit.gemius.pl 2014/07/04 11:11:28.598 kid1| peer_select.cc(699) peerGetSomeParent: peerSelect: FIRSTUP_PARENT/194.99.121.200 2014/07/04 11:11:28.598 kid1| peer_select.cc(724) peerGetAllParents: peerGetAllParents: adding alive parent 194.99.121.200 2014/07/04 11:11:28.598 kid1| peer_select.cc(265) peerSelectDnsPaths: Find IP destination for: http://gdecz.hit.gemius.pl/gdejs/inscreen_lib.js' via 194.99.121.200 2014/07/04 11:11:28.598 kid1| peer_select.cc(265) peerSelectDnsPaths: Find IP
[squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
So squid is exactly doing, what you are asking for: cache_peer = local=0.0.0.0 remote=194.99.121.200:3128 flags=1 But probably, this is not what you want, as it is the public IP on the web, the request is forwarded to. So you most likely should use an internal/local IP of your peer here, OR there is a problem with your routing. BTW: babajaga is a Russian witch. Sort of. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Question-to-cache-peer-tp416p446.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
babajaga augustus_me...@yahoo.de schrieb am 04.07.2014 11:54:04: Von: babajaga augustus_me...@yahoo.de An: squid-users@squid-cache.org Datum: 04.07.2014 11:55 Betreff: [squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer So squid is exactly doing, what you are asking for: cache_peer = local=0.0.0.0 remote=194.99.121.200:3128 flags=1 But probably, this is not what you want, as it is the public IP on the web, the request is forwarded to. So you most likely should use an internal/local IP of your peer here, OR there is a problem with your routing. BTW: babajaga is a Russian witch. Sort of. -- View this message in context: http://squid-web-proxy-cache. 1019090.n4.nabble.com/Question-to-cache-peer-tp416p446.html Sent from the Squid - Users mailing list archive at Nabble.com. Hi babajaga the answer to my problem is: always_direct deny all never_direct allow all So, now all traffic is forwarded to the parent. Thank you for your help !! Mit freundlichen Grüßen / Kind regards Mr. Andreas Reschke andreas.resc...@mahle.com, http://www.mahle.com
Re: [squid-users] Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
On 2014-07-05 00:12, andreas.resc...@mahle.com wrote: Hi babajaga the answer to my problem is: always_direct deny all never_direct allow all Also remove hierarchy_stoplist Amos
Re: [squid-users] Re: access denied
On 2014-07-04 16:56, winetbox wrote: ok, it's done. it works now on 1 eth. all i did: on squid: iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3129 on mikrotik: remove all redirect NAT, create a route to squid machine as internet gateway, create a mangle where src-nat from clients dst-port=80, are all routed to proxy gateway. i have another problem though, i do: # tail -f /var/log/squid3/access.log | grep TCP_HIT and if i: # tail -f /var/log/squid3/access.log i see everything is TCP_MISS, for example: 1404449047.279 2035 192.168.14.3 TCP_MISS/200 327 POST http://makasar.speedtest.telkom.net.id/speedtest/upload.php? - HIER_DIRECT/118.98.104.242 text/html 1404449049.441 4211 192.168.14.3 TCP_MISS/200 327 POST http://makasar.speedtest.telkom.net.id/speedtest/upload.php? - HIER_DIRECT/118.98.104.242 text/html 1404449052.162 2630 192.168.14.3 TCP_MISS/200 327 POST http://makasar.speedtest.telkom.net.id/speedtest/upload.php? - HIER_DIRECT/118.98.104.242 text/html 1404449052.966 3419 192.168.14.3 TCP_MISS/200 327 POST http://makasar.speedtest.telkom.net.id/speedtest/upload.php? - HIER_DIRECT/118.98.104.242 text/html something i missed? if if i don't wrongly recall, my last squid(squid 2.9) access.log, don't have HIER_DIRECT, it is just DIRECT. Means the same thing. HIER_ is a cosmetic bug, should be fixed in current releases. Amos
Re: [squid-users] Re: access denied
On 2014-07-04 15:19, winetbox wrote: This is because of the fix for CVE-2009-0801. NAT on a separate machine has never actually worked properly even in 2.7. The fix we have in current Squid involves verifying the TCP destination IP, which also enforces that NAT is performed on the Squid machine instead of remotely. You need to use policy routing or similar mechanisms on the router to get the packets to the Squid machine unchanged for interception to work. Amos on the contrary, my setup was working perfectly on those versions, because i'm not using the same machine for NAT routing. for routing, i leave everything on mikrotik, what squid do is only accept redirected request from mikrotik. TCP connections arriving at Squid had corrupted destination IP address due to NAT changes on the microtik. Old squid used to *guess* the destination based on Host: header in the HTTP request. This was proven to be a mistake (see CVE details) and current versions use the original dst IP (http://www.squid-cache.org/Doc/config/client_dst_passthru/). Amos
Re: [squid-users] Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
Hi Amos, Should the wiki article http://wiki.squid-cache.org/Features/CacheHierarchy be updated so that the never_direct allow all is preceded by always_direct deny all? Regards HASSAN On Fri, Jul 4, 2014 at 6:47 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 2014-07-05 00:12, andreas.resc...@mahle.com wrote: Hi babajaga the answer to my problem is: always_direct deny all never_direct allow all Also remove hierarchy_stoplist Amos
Re: [squid-users] Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: [squid-users] Re: Antwort: Re: [squid-users] Question to cache_peer
On 2014-07-05 00:56, Nyamul Hassan wrote: Hi Amos, Should the wiki article http://wiki.squid-cache.org/Features/CacheHierarchy be updated so that the never_direct allow all is preceded by always_direct deny all? The default for always_direct is to drop through and obey never_direct. There may be a bug in your particular version if you need to set always_direct at all. You do need to remove hierarchical_stoplist and correctly set nonheirarchical_direct though. The wiki page is not quite making that clear. They should result in errors with never_direct deny all rather than going direct though. So the behaviour you are seeing looks more like a bug in always_direct processing. Amos Regards HASSAN On Fri, Jul 4, 2014 at 6:47 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 2014-07-05 00:12, andreas.resc...@mahle.com wrote: Hi babajaga the answer to my problem is: always_direct deny all never_direct allow all Also remove hierarchy_stoplist Amos
[squid-users] Re: access denied
http_port 3129 intercept now work well now i'm trying to do the same for https, but doesn't work i put a new line on squid.conf https_port 3131 intercept # iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 3131 these doesn't work at all -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/access-denied-tp419p453.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: access denied
On 2014-07-05 01:51, winetbox wrote: http_port 3129 intercept now work well now i'm trying to do the same for https, but doesn't work i put a new line on squid.conf https_port 3131 intercept # iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 3131 these doesn't work at all Port 443 is more complicated as you have to decrypt the TLS traffic to reach the HTTP inside it. That means ssl-bump feature needs configuring if you are to handle the HTTP traffic inside the TLS encryption. NP: The latest releases will wrap intercepted port 443 traffic in a CONNECT provided you configure ssl_bump none for the relevant src or dst IP. If this is sufficient for your needs it would be best, as you avoid having to break the security encryption. Does require the latest 3.4 (or 3.HEAD) though. Amos
Re: [squid-users] TProxy Setup
Just some quick answers to your questions inline below. (I've not had time to consider this in detail sorry.) On 2014-07-04 03:03, Nyamul Hassan wrote: Thank you Amos Eliezer for your responses! Amos, we have enabled debug_options 11,2, but that did not show any HTTP request being received by Squid, not even after doing the changes that Eliezer suggested. But they did show up, when we reverted back to http_port 3127 intercept related configuration. More details below. That is the problem then. Something is blocking the traffic arriving at Squid listening port. selinux, rp_filter or ip_forward sysctl settings I usually find are the problem for this, although there have been a few cases where nobody could figure out why this was happening. Eliezer, we tried with the ip route add local default dev lo table 100, but still same problem. I think the wiki page http://wiki.squid-cache.org/Features/Tproxy4 needs to be updated as it clearly says dev eth0 and not dev lo. see the /!\ notes under in the wiki page under the section about setting up the route table. The interface(s) to attach the table to is the one receiving the packets. From your description I suspect you will have two interfaces - one for each of Rtr1 and Rtr2. For debugging try setting it for each interfaces receiving traffic and see if TPROXY starts working. Our setup would need a bit explanation. Please bear with me while I describe as below: For Traffic From Host: #Start# Host (eth0 A.B.170.10/26) -- -- (eth2 A.B.170.1/26) Rtr1 (eth2 A.B.170.1/26) -- -- (eth0 A.B.170.24/26) SquidBox (eth1 A.B.169.21/28) -- -- (eth2 A.B.169.17/28) Rtr2 (eth1 BGP peered uplink) -- -- Internet #End# For Traffic From Internet: #Start# Internet -- -- (eth1 BGP peered uplink) Rtr2 (eth2 A.B.169.17/28) -- -- (eth1 A.B.169.21/28) SquidBox (eth0 A.B.170.24/28) -- -- (eth0 A.B.170.10/26) Host #End# * In my understanding, this should not pass through Rtr1 as as SquidBox eth0 is in the same subnet as Host. All HTTP traffic should pass through SquidBox, and in fact through Squid itself. The TCP layer ports and packet serial numbers do not match what the client is aware of, so any traffic accidentally reaching it without going through Squid will be dropped. Put this off to later though. Right now the packets are not even getting into Squid. Both Rtr1 Rtr2 are Linux based router called Mikrotik, installed on x86 hardware. Rtr1 has the following rules: /ip firewall mangle add action=mark-routing chain=prerouting disabled=no dst-port=80 new-routing-mark=_to_squid_ passthrough=yes protocol=tcp src-address=A.B.170.10 /ip route add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=A.B.170.24 routing-mark=_to_squid_ scope=30 target-scope=10 Rtr2 has the following rules: /ip firewall mangle add action=mark-routing chain=prerouting disabled=no dst-address=A.B.170.10 new-routing-mark=_to_squid_ passthrough=yes protocol=tcp src-port=80 /ip route add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=A.B.169.21 routing-mark=_to_squid_ scope=30 target-scope=10 The policy routing rules are the same on Rtr1 when we use the REDIRECT rule in iptables -t nat for http_port 3127 intercept, and in that instance SquidBox works like a charm. All the HTTP request are now showing up in cache.log because of debug_options 11,2 as Amos suggested. Great. Thank you for these details. I am creating a Microtik wiki page based on them. However, whenever we remove the nat rules and introduce the mangle rules + ip rule + ip route in table 100 for http_port 3129 tproxy, Rtr1 shows that the packets are marked and forwarded to SquidBox. Even SquidBox properly logs the packets in /var/log/messages due to the mangle table rule, but Squid process on SquidBox does not seem to be receiving the packets. No HTTP request entry showing up in cache.log. IPTables -L for mangle show the following: [root@proxy01 ~]# iptables -vxnL --line-numbers -t mangle Chain PREROUTING (policy ACCEPT 235 packets, 29632 bytes) num pkts bytes target prot opt in out source destination 1 00 ACCEPT all -- * * 0.0.0.0/0 A.B.169.21 26174 821596 ACCEPT all -- * * 0.0.0.0/0 A.B.170.24 3100551367 ACCEPT all -- * * 0.0.0.0/0 A.B.174.0/24 4 00 ACCEPT all -- * * 0.0.0.0/0 M.N.0.66 5 49 3420 DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket 6 52 3840 LOGtcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 LOG flags 0 level 4 prefix `TProxy: ' 7 52 3840 TPROXY tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 TPROXY redirect 0.0.0.0:3129 mark 0x1/0x1 The IP rule route lists, and rt_tables rp_filter show: [root@proxy01 ~]# ip route list table squidtproxy local default dev lo
Re: [squid-users] TProxy Setup
That is the problem then. Something is blocking the traffic arriving at Squid
listening port. selinux, rp_filter or ip_forward sysctl settings I usually
find are the problem for this, although there have been a few cases where
nobody could figure out why this was happening.
We might be approaching that magical situation where we do not know
what is happening!
rp_filter is set to 0 for all as follows:
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter
/proc/sys/net/ipv4/conf/all/rp_filter
/proc/sys/net/ipv4/conf/default/rp_filter
/proc/sys/net/ipv4/conf/lo/rp_filter
/proc/sys/net/ipv4/conf/eth0/rp_filter
/proc/sys/net/ipv4/conf/eth1/rp_filter
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter -exec cat {} +
0
0
0
0
0
IP Rule Route list is as follows:
[root@proxy01 ~]# ip rule list
0: from all lookup local
32765: from all fwmark 0x1 lookup squidtproxy
32766: from all lookup main
32767: from all lookup default
[root@proxy01 ~]# ip route list table squidtproxy
local default dev eth0 scope host
see the /!\ notes under in the wiki page under the section about setting up
the route table.
The interface(s) to attach the table to is the one receiving the packets.
From your description I suspect you will have two interfaces - one for each
of Rtr1 and Rtr2.
For debugging try setting it for each interfaces receiving traffic and see if
TPROXY starts working.
While playing with the linux iptables / ip commands, I have come
across an interesting situation.
I modified the mangle rule to mark as 111, and updated the ip rule to show:
32765: from all fwmark 0x6f lookup squidtproxy
All other settings are unchanged.
No other changes were made. Under this situation, my test client was
getting web pages loaded! But, Squid was still not getting any
requests! Seemed like regular routing of traffic! I have checked both
routers, and confirmed that, traffic was passing through SquidBox, but
Squid process was not seeing it. :-/
Great. Thank you for these details. I am creating a Microtik wiki page based
on them.
If there is anything that I can help you with regarding the Mikrotik
(that's k for both characters) wiki page, I would be most obliged.
Regards
HASSAN
Re: [squid-users] TProxy Setup
Dear Amos,
We just found a small software:
https://github.com/kristrev/tproxy-example
As the author put it:
The example transparent proxy application accepts TCP connections on
the specified port (set to 9876 in tproxy_test.h) and attempts a TCP
connection to the original host. If it is successful, the application
starts forwarding data between the two connections (using splice()).
So, we compiled it and ran it, on port 9876. Then changed the
iptables mangle rules WITH ONLY the port 9876, all others remaining as
they were.
Everything is working perfectly! So, is it safe to assume that
iptables kernel is working perfectly? That there is a problem in
Squid?
Regards
HASSAN
On Sat, Jul 5, 2014 at 1:26 AM, Nyamul Hassan nya...@gmail.com wrote:
That is the problem then. Something is blocking the traffic arriving at
Squid listening port. selinux, rp_filter or ip_forward sysctl settings I
usually find are the problem for this, although there have been a few cases
where nobody could figure out why this was happening.
We might be approaching that magical situation where we do not know
what is happening!
rp_filter is set to 0 for all as follows:
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter
/proc/sys/net/ipv4/conf/all/rp_filter
/proc/sys/net/ipv4/conf/default/rp_filter
/proc/sys/net/ipv4/conf/lo/rp_filter
/proc/sys/net/ipv4/conf/eth0/rp_filter
/proc/sys/net/ipv4/conf/eth1/rp_filter
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter -exec cat {} +
0
0
0
0
0
IP Rule Route list is as follows:
[root@proxy01 ~]# ip rule list
0: from all lookup local
32765: from all fwmark 0x1 lookup squidtproxy
32766: from all lookup main
32767: from all lookup default
[root@proxy01 ~]# ip route list table squidtproxy
local default dev eth0 scope host
see the /!\ notes under in the wiki page under the section about setting up
the route table.
The interface(s) to attach the table to is the one receiving the packets.
From your description I suspect you will have two interfaces - one for each
of Rtr1 and Rtr2.
For debugging try setting it for each interfaces receiving traffic and see
if TPROXY starts working.
While playing with the linux iptables / ip commands, I have come
across an interesting situation.
I modified the mangle rule to mark as 111, and updated the ip rule to
show:
32765: from all fwmark 0x6f lookup squidtproxy
All other settings are unchanged.
No other changes were made. Under this situation, my test client was
getting web pages loaded! But, Squid was still not getting any
requests! Seemed like regular routing of traffic! I have checked both
routers, and confirmed that, traffic was passing through SquidBox, but
Squid process was not seeing it. :-/
Great. Thank you for these details. I am creating a Microtik wiki page based
on them.
If there is anything that I can help you with regarding the Mikrotik
(that's k for both characters) wiki page, I would be most obliged.
Regards
HASSAN
[squid-users] Re: access denied
oh well, i think i have to wait for a while to do so then. because i install my squid on ubuntu machine through package installer. i have to wait until they release a stable version of it so i can easily upgrade it. thanks for the assistance, we can tag this [SOLVED] -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/access-denied-tp419p458.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] TProxy Setup
Hey,
I am not sure if you understand you question which is:
I have a software that works on many many many many systems around the
world, Why is it not working for me? because of the setup or because of
the software?
I would not say that computers are saints or that software are perfect
but since I can use the proxy for so many systems and it works fine..
I raise the question: What is going on on your system setup?
If you will understand that something is wrong but not from squid side
you will be open to understand that something is wrongly configured.
I Tried to understand your network diagram but I cannot read it
well(sorry my bad).
If you can describe the setup in words I will try again to understand it.
I will try to build a setup with a mikrotik device to try and help you
and others that doesn't happen to make it work.
Eliezer
On 07/05/2014 12:02 AM, Nyamul Hassan wrote:
Dear Amos,
We just found a small software:
https://github.com/kristrev/tproxy-example
As the author put it:
The example transparent proxy application accepts TCP connections on
the specified port (set to 9876 in tproxy_test.h) and attempts a TCP
connection to the original host. If it is successful, the application
starts forwarding data between the two connections (using splice()).
So, we compiled it and ran it, on port 9876. Then changed the
iptables mangle rules WITH ONLY the port 9876, all others remaining as
they were.
Everything is working perfectly! So, is it safe to assume that
iptables kernel is working perfectly? That there is a problem in
Squid?
Regards
HASSAN
On Sat, Jul 5, 2014 at 1:26 AM, Nyamul Hassannya...@gmail.com wrote:
That is the problem then. Something is blocking the traffic arriving at Squid
listening port. selinux, rp_filter or ip_forward sysctl settings I usually find are
the problem for this, although there have been a few cases where nobody could figure
out why this was happening.
We might be approaching that magical situation where we do not know
what is happening!
rp_filter is set to 0 for all as follows:
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter
/proc/sys/net/ipv4/conf/all/rp_filter
/proc/sys/net/ipv4/conf/default/rp_filter
/proc/sys/net/ipv4/conf/lo/rp_filter
/proc/sys/net/ipv4/conf/eth0/rp_filter
/proc/sys/net/ipv4/conf/eth1/rp_filter
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter -exec cat {} +
0
0
0
0
0
IP Rule Route list is as follows:
[root@proxy01 ~]# ip rule list
0: from all lookup local
32765: from all fwmark 0x1 lookup squidtproxy
32766: from all lookup main
32767: from all lookup default
[root@proxy01 ~]# ip route list table squidtproxy
local default dev eth0 scope host
see the /!\ notes under in the wiki page under the section about setting up
the route table.
The interface(s) to attach the table to is the one receiving the packets.
From your description I suspect you will have two interfaces - one for each of Rtr1
and Rtr2.
For debugging try setting it for each interfaces receiving traffic and see if
TPROXY starts working.
While playing with the linux iptables / ip commands, I have come
across an interesting situation.
I modified the mangle rule to mark as 111, and updated the ip rule to show:
32765: from all fwmark 0x6f lookup squidtproxy
All other settings are unchanged.
No other changes were made. Under this situation, my test client was
getting web pages loaded! But, Squid was still not getting any
requests! Seemed like regular routing of traffic! I have checked both
routers, and confirmed that, traffic was passing through SquidBox, but
Squid process was not seeing it. :-/
Great. Thank you for these details. I am creating a Microtik wiki page based
on them.
If there is anything that I can help you with regarding the Mikrotik
(that's k for both characters) wiki page, I would be most obliged.
Regards
HASSAN
