[squid-users] Re: Squid not sending request to web
Okay so I would need to use the following Iptables script on the squid box correct? iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j REDIRECT --to-port 3127 Thanks so much for your help. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-not-sending-request-to-web-tp4665512p4665518.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: Squid not sending request to web
On 10/04/2014 11:49 p.m., fordjohn wrote: Okay so I would need to use the following Iptables script on the squid box correct? iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j REDIRECT --to-port 3127 Yes. With any appropriate routing config on the router necessary to get the packets to Squid box without altering their destination IP:port details. Amos
[squid-users] Re: Cache Windows Updates ONLY
I found the discussion on the web post. On Nabble, which I presume will not feed back to this list. I located the discussion forum from the web site, have subscribed, and hope the message will be useful. A web interface to this mailing list could be very useful to capture important information from those users who seldom have something to add. I use a similar configuration on my Squid to the one used by HilltopsGM. Microsoft have recently released an update 4Gb in size for Windows 8, with range request downloads. This will likely cause Squid to use excessive bandwidth. My cache was slaughtering bandwidth until I made some changes. it appears Microsoft now use psf files, which appear to cache OK. #Note: include psf files refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims #Having already defined the windowsupdate ACL, range_offset_limit -1 windowsupdate quick_abort_min -1 KB windowsupdate maximum_object_size 500 KB windowsupdate #And for a cache replacement policy oriented to #bandwidth conservation rather than latency cache_replacement_policy heap LFUDA -- My squid 3 configuration file now looks like: debug_options ALL,2 acl all src all http_access allow all cache_store_log /var/log/squid/store.log hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY hosts_file /etc/hosts refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern . 0 20% 4320 acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl purge method PURGE acl CONNECT method CONNECT cache_mem 256 MB http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge acl lan src 10.10.10.1/24 http_access allow localhost http_access allow lan visible_hostname myclient.hostname.com http_port 3128 cache_replacement_policy heap LFUDA maximum_object_size 20 KB cache_dir ufs /home/smb/squid/squid 7 16 256 coredump_dir /home/smb/squid/squid refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims acl windowsupdate dstdomain windowsupdate.microsoft.com acl windowsupdate dstdomain .update.microsoft.com acl windowsupdate dstdomain download.windowsupdate.com acl windowsupdate dstdomain redir.metaservices.microsoft.com acl windowsupdate dstdomain images.metaservices.microsoft.com acl windowsupdate dstdomain c.microsoft.com acl windowsupdate dstdomain www.download.windowsupdate.com acl windowsupdate dstdomain wustat.windows.com acl windowsupdate dstdomain crl.microsoft.com acl windowsupdate dstdomain swcdn.apple.com acl windowsupdate dstdomain data-cdn.mbupdates.com #header_access Pragma deny windowsupdate unrecognised in squid 3 #directives mentioned http://www1.us.squid-cache.org/mail-archive/squid-users/200506/0684.html - nick 16 Feb 09 range_offset_limit -1 windowsupdate quick_abort_min -1 KB windowsupdate maximum_object_size 500 KB windowsupdate ##9 April 2014 ##From http://wiki.squid-cache.org/Features/DelayPools ##limit squid to 1.2Mbit/second, reduce contention for updates delay_pools 1 delay_class 1 1 delay_access 1 allow all delay_parameters 1 120/120 -- On Tue, 20 Aug 2013 17:49:19 -0700 (PDT) HillTopsGM Wrote Does this make sense then: (START OF CODE FOR SQUID.CONF FILE) #== #Below is what I'd copy and past from the FAQ for windows updates: #== range_offset_limit -1 maximum_object_size 200 MB quick_abort_min -1 # Add one of these lines for each of the websites you want to cache. refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims refresh_pattern -i my.windowsupdate.website.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims # DONT MODIFY THESE LINES refresh_pattern \^ftp: 144020% 10080 refresh_pattern \^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 acl windowsupdate dstdomain windowsupdate.microsoft.com acl windowsupdate dstdomain .update.microsoft.com acl windowsupdate dstdomain download.windowsupdate.com acl windowsupdate dstdomain redir.metaservices.microsoft.com acl windowsupdate dstdomain images.metaservices.microsoft.com acl windowsupdate dstdomain c.microsoft.com acl windowsupdate dstdomain www.download.windowsupdate.com acl windowsupdate dstdomain wustat.windows.com acl windowsupdate dstdomain crl.microsoft.com acl windowsupdate dstdomain sls.microsoft.com
[squid-users] How to change redirection path to forward to www.earth.com/moon insted of moon.earth.com ?
Can someone please advise me about a problem that I have. I'v deployed Squid3 Reverse Proxy Server. I have one server in the internal network that is forwarded through Squid. My external network is called earth.com and the internal server is called moon. I'v managed to set Squid to forward packets that come for moon.earth.com to internal server moon which is on IP 192.168.1.10 My question is: How can I setup Squid to forward packets when it receives http://www.earth.com/moon instead of the current setup - http://moon.earth.com My current config lines are these: --- acl issues_users dstdomain moon.earth.com http_access allow moon_users cache_peer 192.168.1.10 parent 8080 0 no-query originserver name=moon cache_peer_domain moon moon.earth.com cache_peer_access moon allow moon_users cache_peer_access moon deny all -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/How-to-change-redirection-path-to-forward-to-www-earth-com-moon-insted-of-moon-earth-com-tp4665521.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Cache Windows Updates ONLY
I notice Microsoft update for windows 8 is adding query strings to URLs as a token. This makes it hard for open source caches to work effectively with Microsoft Windows 8 updates. Maybe this is a method to force users to use Microsoft's proprietary windows update caching software. The recent KB2919355 update is a whopping 4Gb download. Bigger than an ISO for windows 8. I don't understand why Microsoft make the updates so large, and why they make them difficult to cache. It is almost as if they wish to maximise the bandwidth windows update consumes. The form is: http://bg.v4.a.dl.ws.microsoft.com/dl/content/d/updt/2013/09/f4d26fdb-d520-48da-add6-6a3c0832d14a_6fda48f8c83be2a15f49b83b10fc3dc8c1d15774.appx?P1=TOKENP2=TOKENP3=TOKENP4=TOKEN The server won't deliver the file unless the tokens are in place. Whenever a file is fetched, it appears to be the same irrespective of the tokens. I will carry out more research based on checksums of multiple files to make sure. These same files are typically fetched using range requests. The file example above is over 1Gb. Well worth caching. I'm looking for a way to configure squid3 so that if the domain is ws.microsoft.com and if the URL includes .appx?P1= then the URL is fetched with query string from the source and stored without query string. Any future request should match and deliver the stored file irrespective of any query string. Is there a convenient way to configure Squid to do this? Thanks.
Re: [squid-users] Re: Cache Windows Updates ONLY
Hey Nick, In a case you do know the tokens meaning and if it is working properly you can try to use StoreID in 3.4.X http://wiki.squid-cache.org/Features/StoreID It is designed to allow you this specific issue you are sure it is. About the 4GB or 1GB updates it's pretty simple. Microsoft release an update which contains everything about the about even that the update for your machine is only part of the file. This is what the last time I verified the issue. Also there is another side that OS become more and more complex and an update can be really big which almost replacing half of the OS components. What ever goes for you from the options is fine and still I have not seen microsoft cache solution. How is it called? Eliezer On 04/10/2014 08:50 PM, Nick Hill wrote: Is there a convenient way to configure Squid to do this? Thanks.
[squid-users] Re: Cache Windows Updates ONLY
Should I change the cache allow mywindowsupdates always_direct allow all ... to cache allow mywindowsupdates cache deny all To ONLY cache the windows updates, cache allow mywindowsupdates cache deny all would be correct. # #always_direct allow all #This is NOT related to caching. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Re-Cache-Windows-Updates-ONLY-tp4665520p4665524.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Cache Windows Updates ONLY
The server won't deliver the file unless the tokens are in place. Whenever a file is fetched, it appears to be the same irrespective of the tokens. I will carry out more research based on checksums of multiple files to make sure. I very doubt to be the same ... . Because this would not make sense. youtube does something similar for their videos, and there the tokens contain add info like resolution of the movie, as it is distributed in different ones. Depending upon actual connection speed, for instance. So, the only reason to have random tokens in your case would be to confuse the caches, which I doubt. OR it might signal some info regarding the size of the range requests. Then it would be safe to ignore the tokens, as you are considering, as the complete file will be cached within squid, and the different ranges serviced from there. (Note: This is something, youtube did some time ago. ) So you might test with different connections speeds, too. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Re-Cache-Windows-Updates-ONLY-tp4665520p4665525.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: Cache Windows Updates ONLY
On 11/04/2014 12:28 a.m., Nick Hill wrote: I found the discussion on the web post. On Nabble, which I presume will not feed back to this list. I located the discussion forum from the web site, have subscribed, and hope the message will be useful. A web interface to this mailing list could be very useful to capture important information from those users who seldom have something to add. I use a similar configuration on my Squid to the one used by HilltopsGM. Microsoft have recently released an update 4Gb in size for Windows 8, with range request downloads. This will likely cause Squid to use excessive bandwidth. My cache was slaughtering bandwidth until I made some changes. it appears Microsoft now use psf files, which appear to cache OK. #Note: include psf files refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip|psf) 4320 80% 43200 reload-into-ims #Having already defined the windowsupdate ACL, range_offset_limit -1 windowsupdate quick_abort_min -1 KB windowsupdate maximum_object_size 500 KB windowsupdate Thank you for the details. I have updated the wiki patterns and notes: http://wiki.squid-cache.org/SquidFaq/WindowsUpdate Amos
[squid-users] Re: How to change redirection path to forward to www.earth.com/moon insted of moon.earth.com ?
Search for the comments to url_rewrite_program in squid.conf. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/How-to-change-redirection-path-to-forward-to-www-earth-com-moon-insted-of-moon-earth-com-tp4665521p4665527.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] How to change redirection path to forward to www.earth.com/moon insted of moon.earth.com ?
On 11/04/2014 3:56 a.m., Strafe wrote: Can someone please advise me about a problem that I have. I'v deployed Squid3 Reverse Proxy Server. I have one server in the internal network that is forwarded through Squid. My external network is called earth.com and the internal server is called moon. I'v managed to set Squid to forward packets that come for moon.earth.com to internal server moon which is on IP 192.168.1.10 My question is: How can I setup Squid to forward packets when it receives http://www.earth.com/moon instead of the current setup - http://moon.earth.com Firstly, Squid does not forward packets. Squid forwards HTTP messages. What you ask can be done in several ways ... 1) Stick with the current setup mapping only the domain names. It is both the Right Way to map HTTP requests between domains and avoids many complications and problems. For your existing config: * drop the cache_peer_domain directive * Make the internal server aware that it is servicing the public domain www.earth.com and the entire HTTP ecosystem will operate as designed by RFC 2616 specifications. ** Virtual hosting is your friend ** 2) If you can, setup a URL redirection which forwards clients to the new URL. This is the Right Way to make a URL point at a different URL in HTTP. a) Squid-3.2 and newer can setup a redirection like so: acl foo dstdomain moon.earth.com deny_info 302:http://moon.earth.com/moon%R foo http_access deny foo b) Squid-3.1 and older you need to setup a URL redirector to emit 30x status code and the new URL: acl foo dstdomain moon.earth.com url_rewrite_program /somescript url_rewrite_access allow foo Doing (2) in either form requires that both www.earth.com and moon.earth.com are visible and available to the public clients. 3) if you really have no choice at all you. And I mean that very seriously - this is an absolute last resort. You can setup a URL-rewriter program like (2a) above, but omitting the 30x status code. This changes the URL delivered to the peer server but leaves the client thinking its accessing moon.earth.com URLs. When undertaking (3) you had best audit your site code, contents, and user abilities to ensure that: * server MUST NOT generate any absolute-URLs using the domain which the webserver is asked for. * page content MUST NOT use any relative-URL beginning with '/' characters. Unless they are prefixed /moon by the origin server. * Content-Disposition headers meet the criteria above * Content-Location headers meet the criteria above * Location headers meet the criteria above ... anything containing URLs must meet the absolute-URL and relative-URL criteria. These conditions apply to URLs *anywhere* in the HTTP protocol and content objects emitted by the web server application. Generated and static content alike is affected, as are compressed archives with URL file meta tags and DB entries of user input URLs. *anything* with URLs. If any one of these conditions on URL-rewriting are missed during your audit, or future site changes cause one to be untrue. Then the clients receiving traffic with those broken URL details will encounter issues of various types trying to access or use them. HTH Amos
Re: [squid-users] Re: Cache Windows Updates ONLY
On 11 April 2014 05:15, babajaga augustus_me...@yahoo.de wrote: I very doubt to be the same ... . Because this would not make sense. youtube does something similar for their videos, and there the tokens contain add info like resolution of the movie, as it is distributed in different ones. Depending upon actual connection speed, for instance. I performed a SGA1 checksum on the downloaded file. The checksum was 6fda48f8c83be2a15f49b83b10fc3dc8c1d15774 The file was downloaded using wget, with the tokens. This matches the part of the file name between the underscore and period. The only thing we need for Squid to match, is the part of the URL between the underscore and period. If the checksum matches, we know the content we are serving up is correct.
Re: [squid-users] Squid Question about method GET
On 10/04/2014 2:27 a.m., MIGUEL ANGEL AGUAYO ORTUÑO wrote: I had this config befor acl my_url dstdomain jakjak.dit.upm.es redirector_access allow my_url redirect_children 1 redirect_rewrites_host_header off redirect_program /etc/squid/dashcheck.pl but this configuration only aims for the destiny domain and I'm trying to use this configuration to match the file types I want acl my_url urlpath_regex \.(mpd|m4s)$ redirector_access allow my_url redirect_children 1 redirect_rewrites_host_header off redirect_program /etc/squid/dashcheck.pl but the thing is that when I enter http://jakjak.dit.upm.es/mpd/sintel.mpd It doesnt enter to the redirector why?? With the information provided your dashcheck.pl helper SHOULD be receiving the URLs you indicated. The URL provided seems not to exist for public access, so I am unable to perform any kind of checks verifying the accuracy of your statements about the URL. The below is a *guess* about where the likely problem *might* be... Perhapse the statement made earlier about the URL syntax was incorrect - which would make your regex pattern also incorrect. The URI path segment matched by urlpath_regex contains both resource and query sub-segments. If there is any query-string on the URLs your pattern MUST account for it somehow. Also, with an explicit redirect_access directive present only requests which meet the ACL criteria are sent to the helper. The implicit default in your above config is redirector_access deny all. I suggest removing the redirect_access and ACL while you perfect what the helper is doing. Last I heard you were trying to figure out where to locate pieces of the helpepr input. Has that been resolved with the info provided earlier? Amos
