[squid-users] Re: Youtube Changes
youtube back to static id -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659706.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: Youtube Changes
I am following this thread closely since inception and wanted to test
this on my squid 2.7.9.
However I am getting a problem with the second refresh_pattern regex.
2013/04/24 15:37:36| parse_refreshpattern: Unknown option
'(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?)':
ignore-must-revali..
On 24 April 2013 04:35, syaifuddin ucok_karn...@yahoo.com wrote:
ragheb68 wrote
Hi syaifuddin,
Trying to implement the changes you proposed on squid 2.7stable 9. Got
confused there is nothing in your email concerning refresh patterns and
other old youtube settings in squid.conf should all of them be removed and
replaced with new ones or just add the new acls along with the old ones. I
have currently installed File::Readbackwards Perl module on fedora 13 and
added your lines to my already old squid.conf with old youtube among other
settings and replaced the old storeurl.pl with yours but still I can see
TCP_MISS on all youtube requests in access.log file. Any ideas what could
be the cause?
strip_query_terms off
acl yutub dstdomain .youtube.com .youtu.be
logformat squid1 %{Referer}h %ru
access_log /var/log/squid/yt.log squid1 yutub
acl redirec urlpath_regex -i redirect_counter=1cms_redirect=yes
ir=1rr=12
acl videocache url_regex -i \.(youtu|google).*\.com\/videoplayback
storeurl_access deny redirec
storeurl_access allow videocache
storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 1
storeurl_rewrite_concurrency 10
cache deny redirec
refresh_pattern ^.*youtube\.com\.SQUIDINTERNAL.*$ 43200 99% 43200
override-expire ignore-reload ignore-no-cache ignore-private negative-ttl=0
refresh_pattern
(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200 99%
43200 override-expire ignore-reload ignore-must-revalidate ignore-no-cache
ignore-private negative-ttl=0
--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659620.html
Sent from the Squid - Users mailing list archive at Nabble.com.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223
I can't hear you -- I'm using the scrambler.
[squid-users] Re: Youtube Changes
Odhiambo Washington-4 wrote I am following this thread closely since inception and wanted to test this on my squid 2.7.9. However I am getting a problem with the second refresh_pattern regex. 2013/04/24 15:37:36| parse_refreshpattern: Unknown option '(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?)': ignore-must-revali.. im so sorry. i forgot to remove refresh_pattern option. here correct refresh_pattern refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200 99% 43200 override-expire ignore-reload ignore-no-cache ignore-private negative-ttl=0 -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659626.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: Youtube Changes
This doesn't seem to work for me - so far. Here is what I have added
to squid.conf:
strip_query_terms off
acl yutub dstdomain .youtube.com .youtu.be
logformat squid1 %{Referer}h %ru
access_log /usr/local/squid/logs/yt.log squid1 yutub
acl redirec urlpath_regex -i redirect_counter=1cms_redirect=yes ir=1rr=12
acl videocache url_regex -i \.(youtu|google).*\.com\/videoplayback
storeurl_access deny redirec
storeurl_access allow videocache
storeurl_rewrite_program /usr/local/etc/squid/scripts/storeurl.pl
storeurl_rewrite_children 1
storeurl_rewrite_concurrency 10
cache deny redirec
refresh_pattern ^.*youtube\.com\.SQUIDINTERNAL.*$ 43200 99% 43200
override-expire ignore-reload ignore-no-cache ignore-private
negative-ttl=0
refresh_pattern
(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200
99% 43200 override-expire ignore-reload ignore-no-cache ignore-private
negative-ttl=0
After making the changes and giving `-k reconfigure` to squid, the
process dies and I get the following on cache.log:
2013/04/24 17:12:26| WARNING: store_rewriter #1 (FD 7) exited
2013/04/24 17:12:26| Too few store_rewriter processes are running
FATAL: The store_rewriter helpers are crashing too rapidly, need help!
Squid Cache (Version 2.7.STABLE9): Terminated abnormally.
CPU Usage: 0.036 seconds = 0.012 user + 0.024 sys
Maximum Resident Size: 6916 KB
Page faults with physical i/o: 0
My /usr/local/squid/logs/yt.log is empty
On 24 April 2013 16:22, syaifuddin ucok_karn...@yahoo.com wrote:
Odhiambo Washington-4 wrote
I am following this thread closely since inception and wanted to test
this on my squid 2.7.9.
However I am getting a problem with the second refresh_pattern regex.
2013/04/24 15:37:36| parse_refreshpattern: Unknown option
'(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?)':
ignore-must-revali..
im so sorry. i forgot to remove refresh_pattern option.
here correct refresh_pattern
refresh_pattern
(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200 99%
43200 override-expire ignore-reload ignore-no-cache ignore-private
negative-ttl=0
--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659626.html
Sent from the Squid - Users mailing list archive at Nabble.com.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223
I can't hear you -- I'm using the scrambler.
[squid-users] Re: Youtube Changes
Odhiambo Washington-4 wrote This doesn't seem to work for me - so far. Here is what I have added to squid.conf: 2013/04/24 17:12:26| WARNING: store_rewriter #1 (FD 7) exited 2013/04/24 17:12:26| Too few store_rewriter processes are running FATAL: The store_rewriter helpers are crashing too rapidly, need help! Squid Cache (Version 2.7.STABLE9): Terminated abnormally. CPU Usage: 0.036 seconds = 0.012 user + 0.024 sys Maximum Resident Size: 6916 KB Page faults with physical i/o: 0 My /usr/local/squid/logs/yt.log is empty try chmod 555 /usr/local/etc/squid/scripts/storeurl.pl and try again -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659629.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Re: Youtube Changes
I made mine mode 755 (chmod +x /usr/local/etc/squid/scripts/storeurl.pl) Any idea why perl -C storeurl.pl gets stuck forever?? On 24 April 2013 17:56, syaifuddin ucok_karn...@yahoo.com wrote: Odhiambo Washington-4 wrote This doesn't seem to work for me - so far. Here is what I have added to squid.conf: 2013/04/24 17:12:26| WARNING: store_rewriter #1 (FD 7) exited 2013/04/24 17:12:26| Too few store_rewriter processes are running FATAL: The store_rewriter helpers are crashing too rapidly, need help! Squid Cache (Version 2.7.STABLE9): Terminated abnormally. CPU Usage: 0.036 seconds = 0.012 user + 0.024 sys Maximum Resident Size: 6916 KB Page faults with physical i/o: 0 My /usr/local/squid/logs/yt.log is empty try chmod 555 /usr/local/etc/squid/scripts/storeurl.pl and try again -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659629.html Sent from the Squid - Users mailing list archive at Nabble.com. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 I can't hear you -- I'm using the scrambler.
Re: [squid-users] Re: Youtube Changes
On 23/04/2013 6:15 a.m., babajaga wrote: YES: Very important: READ IT ! :-) How about updating the Squid wiki pages with this recent information? reading four pages of dense regex patterns with the odd word or two of documentation in the middle is not an great way to spend the evening. http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube is a collaborative effort between the people interested fixing YT cacheability problems. Please also correct those squid.conf settings which are documented. The regex patterns added to squid.conf do not need .* or .*$ on the ends. # acl yutub url_regex -i .*youtube\.com\/.*$ # acl yutub url_regex -i .*youtu\.be\/.*$ acl yutub url_regex -i youtu(be\.com|\.be)\/ or maybe even: acl yutub dstdomain .youtube.com .youtu.be # acl redirec urlpath_regex -i .*redirect_counter=1cms_redirect=yes # acl redirec urlpath_regex -i .*ir=1rr=12 acl redirec urlpath_regex -i redirect_counter=1cms_redirect=yes ir=1rr=12 # acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*redirect_counter=1.*$ # acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*cms_redirect=yes.*$ acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*(redirect_counter=1|cms_redirect=yes) Amos
[squid-users] Re: Youtube Changes
Amos Jeffries-2 wrote How about updating the Squid wiki pages with this recent information? reading four pages of dense regex patterns with the odd word or two of documentation in the middle is not an great way to spend the evening. http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube is a collaborative effort between the people interested fixing YT cacheability problems. Please also correct those squid.conf settings which are documented. The regex patterns added to squid.conf do not need .* or .*$ on the ends. # acl yutub url_regex -i .*youtube\.com\/.*$ # acl yutub url_regex -i .*youtu\.be\/.*$ acl yutub url_regex -i youtu(be\.com|\.be)\/ or maybe even: acl yutub dstdomain .youtube.com .youtu.be # acl redirec urlpath_regex -i .*redirect_counter=1cms_redirect=yes # acl redirec urlpath_regex -i .*ir=1rr=12 acl redirec urlpath_regex -i redirect_counter=1cms_redirect=yes ir=1rr=12 # acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*redirect_counter=1.*$ # acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*cms_redirect=yes.*$ acl reddeny url_regex -i c\.youtube\.com\/videoplayback.*(redirect_counter=1|cms_redirect=yes) Amos thank's amos, i will correc ACL, and i'm sorry if my english so bad. because i still learn english also squid -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659615.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Youtube Changes
Hi syaifuddin, Trying to implement the changes you proposed on squid 2.7stable 9. Got confused there is nothing in your email concerning refresh patterns and other old youtube settings in squid.conf should all of them be removed and replaced with new ones or just add the new acls along with the old ones. I have currently installed File::Readbackwards Perl module on fedora 13 and added your lines to my already old squid.conf with old youtube among other settings and replaced the old storeurl.pl with yours but still I can see TCP_MISS on all youtube requests in access.log file. Any ideas what could be the cause? -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659616.html Sent from the Squid - Users mailing list archive at Nabble.com.
Fwd: [squid-users] Re: Youtube Changes
Hello again, I liked the idea of what you did and especially you are using Perl - ReadBackwards Module . Squid is writing a new log and only logging \.youtube\.com requests or adding 2 links on the same line and then you create a subroutine perl to get the returned 2 parameters .. (V/docid/video_id) cpn ...etc . It only writes the video_id when the id does not starts with o-[a-zA-Z0-9-_] which is being written into cache as is , but i solved it by rewriting the whole videoplayback url without ([?]range=[^]*) , ([?]ratebypass=[a-z]*) and also changed the [?]vq=auto to medium or large just to get rid of the auto rate due to the slow connection. did you check it with ranges if the id starts with o- ?. What else youtube would do in the future .. changing the video ID or keep everything hidden?. I liked the idea of what you did and congratulations :-). Thank you so much Ghassan
Re: Fwd: [squid-users] Re: Youtube Changes
Hi Ghassan, Have you implemented the youtube perl script changes to squid2.7 of 3.0 head. Can you please post the new perl script you have amended and what about the changes that should be done to old settings in the squid.conf file that reflected the old youtube caching method should all of these be removed from squid.conf and replaced by syaifuddin settings what about refresh_patterns changes if any? Your help will be much appreciated. Ragheb -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659618.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: Fwd: [squid-users] Re: Youtube Changes
Hello syaiffudin i noticed that in one of your threads you mentioned to compile squid with --enable-referer-log I have this disabled in my 2.7 squid compilation. Could that be the problem that I am still getting misses with my youtube videos. I have also used amos corrections in the ACLs still no youtube videos are being cached as hits only misses. Thanks -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659619.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Youtube Changes
ragheb68 wrote
Hi syaifuddin,
Trying to implement the changes you proposed on squid 2.7stable 9. Got
confused there is nothing in your email concerning refresh patterns and
other old youtube settings in squid.conf should all of them be removed and
replaced with new ones or just add the new acls along with the old ones. I
have currently installed File::Readbackwards Perl module on fedora 13 and
added your lines to my already old squid.conf with old youtube among other
settings and replaced the old storeurl.pl with yours but still I can see
TCP_MISS on all youtube requests in access.log file. Any ideas what could
be the cause?
strip_query_terms off
acl yutub dstdomain .youtube.com .youtu.be
logformat squid1 %{Referer}h %ru
access_log /var/log/squid/yt.log squid1 yutub
acl redirec urlpath_regex -i redirect_counter=1cms_redirect=yes
ir=1rr=12
acl videocache url_regex -i \.(youtu|google).*\.com\/videoplayback
storeurl_access deny redirec
storeurl_access allow videocache
storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 1
storeurl_rewrite_concurrency 10
cache deny redirec
refresh_pattern ^.*youtube\.com\.SQUIDINTERNAL.*$ 43200 99% 43200
override-expire ignore-reload ignore-no-cache ignore-private negative-ttl=0
refresh_pattern
(get_video\?|videoplayback\?|videodownload\?|\.flv\?|\.fid\?) 43200 99%
43200 override-expire ignore-reload ignore-must-revalidate ignore-no-cache
ignore-private negative-ttl=0
--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659620.html
Sent from the Squid - Users mailing list archive at Nabble.com.
Re: Fwd: [squid-users] Re: Youtube Changes
ragheb68 wrote
Hello syaiffudin i noticed that in one of your threads you mentioned to
compile squid with --enable-referer-log I have this disabled in my 2.7
squid compilation. Could that be the problem that I am still getting
misses with my youtube videos. I have also used amos corrections in the
ACLs still no youtube videos are being cached as hits only misses.
Thanks
build option --enable-referer-log not necessary anymore since we have this
config
acl yutub dstdomain .youtube.com .youtu.be
logformat squid1 %{Referer}h %ru
access_log /var/log/squid/yt.log squid1 yutub
and don't forget to change or add this too
*strip_query_terms off
*
hope this can help you
--
View this message in context:
http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659621.html
Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Youtube Changes
try this https://code.google.com/p/tempat-sampah/source/browse/storeurl.pl and read it -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659600.html Sent from the Squid - Users mailing list archive at Nabble.com.
[squid-users] Re: Youtube Changes
YES: Very important: READ IT ! :-) -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-Changes-tp4659599p4659604.html Sent from the Squid - Users mailing list archive at Nabble.com.
