Re: [squid-users] Re: Unable to get HULU ads to cache (not partial content)

2014-07-10 Thread Eliezer Croitoru

More relevant to us:
http://bugs.squid-cache.org/show_bug.cgi?id=3830

I am not sure what the bug causes exactly, whether it will use a 4MB 
default maximum size on the cache_dir.


Eliezer

On 07/09/2014 11:16 PM, Antony Stone wrote:

I see you have maximum_object_size in your quid.conf after cache_dir.

Try specifying maximum_object_size before cache_dir and you shouldn't need the
max_size option to cache_dir.

See alsohttps://bugzilla.redhat.com/show_bug.cgi?id=951224


Regards,


Antony.




[squid-users] Re: Unable to get HULU ads to cache (not partial content)

2014-07-09 Thread Ian Nofziger
Fought with this one for days, and figured it out hours after sending
this email. I apologize for the noise. I'm replying to my own message
in case it helps someone else.

For reasons I don't quite understand, maximum_object_size 2048 MB
wasn't sufficient to let it cache large objects into cache_dir.
However as soon as I changed my cache_dir line to this:

cache_dir aufs /var/cache/squid 50176 14 256 max-size=2147483648

suddenly it started to cache these larger files.

On Wed, Jul 9, 2014 at 1:16 PM, Ian Nofziger ian.nofzi...@gmail.com wrote:
 Hi,

 I've been trying to set up squid for a variety of reasons, but one is
 to cache commercials from HULU. HULU takes up a huge portion of my
 bandwidth. It'd be pointless to try to cache the TV shows since we
 typically only watch them once (I think they're encrypted anyway), but
 I'd like to cache the unencrypted commercials since those get played
 over and over again.

 The commercials come in as partial content (206 code), so I wrote a
 little script that grabs the list of videos with a 206 code each night
 and does a wget -O /dev/null [full address for video] to cache the
 commercials for the next day. This seems to be working just fine. The
 problem is, it still will not cache the videos (even though it is no
 longer a partial content request). Can someone help sort me out?

 (P.S. I'm very much a beginner, so my config file is a result of just
 copying a lot of stuff off the internet... I'll gladly take
 suggestions even if its not directly related to the HULU ads.)

 Thanks in advance for your help!

 store.log:

 1404923465.907 RELEASE -1  B493FE4264AFC396D382596A22280AA5
 200 1404923474 1403637739 1436459474 video/mp4 12018601/12018601 GET
 http://ads-v-darwin.hulu.com/published/2014/6/24/7/826d2864_2b05_4dd7_9ced_c9992a1dd04d_12d382_AdsTranscode_16x9_1403420_24623827_H264_3200.mp4

 access.log:

 1404923465.907 4383 192.168.1.80 TCP_MISS/200 12019050 GET
 http://ads-v-darwin.hulu.com/published/2014/6/24/7/826d2864_2b05_4dd7_9ced_c9992a1dd04d_12d382_AdsTranscode_16x9_1403420_24623827_H264_3200.mp4
 - HIER_DIRECT/8.27.246.254 video/mp4


 # ACCESS CONTROLS
 # 
 -

 ## local network
 #acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
 #acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
 acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
 acl localnet src fc00::/7   # RFC 4193 local private network range
 acl localnet src fe80::/10  # RFC 4291 link-local (directly
 plugged) machines

 ## windows update
 acl windowsupdate dstdomain windowsupdate.microsoft.com
 acl windowsupdate dstdomain .update.microsoft.com
 acl windowsupdate dstdomain download.windowsupdate.com
 acl windowsupdate dstdomain redir.metaservices.microsoft.com
 acl windowsupdate dstdomain images.metaservices.microsoft.com
 acl windowsupdate dstdomain c.microsoft.com
 acl windowsupdate dstdomain www.download.windowsupdate.com
 acl windowsupdate dstdomain wustat.windows.com
 acl windowsupdate dstdomain crl.microsoft.com
 acl windowsupdate dstdomain sls.microsoft.com
 acl windowsupdate dstdomain productactivation.one.microsoft.com
 acl windowsupdate dstdomain ntservicepack.microsoft.com

 ## hulu videos
 acl huluvids dstdomain ads-v-darwin.hulu.com
 acl huluvids dstdomain assets.hulu.com
 acl huluvids dstdomain assets.huluim.com
 acl huluvids dstdomain p.hulu.com
 acl huluvids dstdomain m.hulu.com
 acl huluvids dstdomain ib.huluim.com
 acl huluvids dstdomain s.hulu.com

 ## do not cache sites
 acl directsites dstdomain /etc/squid3/go_direct.txt

 ## malware sites
 acl malwareblockedsites dstdomain /etc/squid3/malware_blocked_sites

 ## fix for netflix on tv
 acl VIERAWMDRM url_regex ^http://us.vieraconnect.tv/wmdrm$

 ## basic ports
 acl SSL_ports port 443
 acl SSL_ports port 563
 acl SSL_ports port 81
 acl SSL_ports port 2087
 acl SSL_ports port 1
 acl Safe_ports port 563
 acl Safe_ports port 631
 acl Safe_ports port 901
 acl Safe_ports port 81
 acl Safe_ports port 3128
 acl Safe_ports port 1025-65535
 acl Safe_ports port 210
 acl Safe_ports port 80 # http
 acl Safe_ports port 21 # ftp
 acl Safe_ports port 443 # https
 acl Safe_ports port 70 # gopher
 acl Safe_ports port 210 # wais
 acl Safe_ports port 1025-65535 # unregistered ports
 acl Safe_ports port 280 # http-mgmt
 acl Safe_ports port 488 # gss-http
 acl Safe_ports port 591 # filemaker
 acl Safe_ports port 777 # multiling http
 acl purge method PURGE
 acl CONNECT method CONNECT

 ## windows update
 acl wuCONNECT dstdomain www.update.microsoft.com
 acl wuCONNECT dstdomain sls.microsoft.com

 http_access allow CONNECT wuCONNECT localnet
 http_access allow windowsupdate localnet
 http_access allow huluvids
 http_access deny malwareblockedsites
 http_access allow localnet
 cache allow huluvids
 cache deny VIERAWMDRM
 cache deny directsites

 http_access deny !Safe_ports
 http_access deny CONNECT 

Re: [squid-users] Re: Unable to get HULU ads to cache (not partial content)

2014-07-09 Thread Antony Stone
On Wednesday 09 July 2014 at 21:54:15, Ian Nofziger wrote:

 Fought with this one for days, and figured it out hours after sending
 this email. I apologize for the noise. I'm replying to my own message
 in case it helps someone else.
 
 For reasons I don't quite understand, maximum_object_size 2048 MB
 wasn't sufficient to let it cache large objects into cache_dir.
 However as soon as I changed my cache_dir line to this:
 
 cache_dir aufs /var/cache/squid 50176 14 256 max-size=2147483648
 
 suddenly it started to cache these larger files.

I see you have maximum_object_size in your quid.conf after cache_dir.

Try specifying maximum_object_size before cache_dir and you shouldn't need the 
max_size option to cache_dir.

See also https://bugzilla.redhat.com/show_bug.cgi?id=951224


Regards,


Antony.

-- 
Tinned food was developed for the British Navy in 1813.

The tin opener was not invented until 1858.

   Please reply to the list;
 please *don't* CC me.


Re: [squid-users] Re: Unable to get HULU ads to cache (not partial content)

2014-07-09 Thread Ian Nofziger
On Wed, Jul 9, 2014 at 4:16 PM, Antony Stone
antony.st...@squid.open.source.it wrote:
 On Wednesday 09 July 2014 at 21:54:15, Ian Nofziger wrote:

 Fought with this one for days, and figured it out hours after sending
 this email. I apologize for the noise. I'm replying to my own message
 in case it helps someone else.

 For reasons I don't quite understand, maximum_object_size 2048 MB
 wasn't sufficient to let it cache large objects into cache_dir.
 However as soon as I changed my cache_dir line to this:

 cache_dir aufs /var/cache/squid 50176 14 256 max-size=2147483648

 suddenly it started to cache these larger files.

 I see you have maximum_object_size in your quid.conf after cache_dir.

 Try specifying maximum_object_size before cache_dir and you shouldn't need the
 max_size option to cache_dir.

 See also https://bugzilla.redhat.com/show_bug.cgi?id=951224


 Regards,


 Antony.

 --
 Tinned food was developed for the British Navy in 1813.

 The tin opener was not invented until 1858.

Please reply to the list;
  please *don't* CC me.

Thanks Antony! I'll give that a try.