Re: [squid-users] Web/URL categorisation list

2014-07-25 Thread Marcus Kool
Hi Alan, On http://www.squid-cache.org/Misc/redirectors.html you can find a list of URL redirectors. ufdbGuard is a free URL redirector that supports free databases and a commercial database from www.urlfilterdb.com Marcus On 07/25/2014 08:33 AM, Alan Dawson wrote: Hi, Apologies if this is

Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged

2014-07-25 Thread Marcus Kool
riginal Message- From: Marcus Kool [mailto:marcus.k...@urlfilterdb.com] Sent: 25 July 2014 00:37 To: RYAN Justin Subject: Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged Juz, The systems seems to have a ver

Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged

2014-07-24 Thread Marcus Kool
Juz, It helps if you describe the system in more detail. What is the configuration of Squid (squid.conf without the comments) and how are the data store file systems spread over the disks ? For immediate results, you can reduce the disk cache or even temporarily disable the disk cache. Marcus

Re: [squid-users] memory_cache_shared no support for atomic operations

2014-06-10 Thread Marcus Kool
On 06/09/2014 07:10 PM, Eliezer Croitoru wrote: On 06/10/2014 12:43 AM, Cassiano Martin wrote: Yes its 32 bit custom built OS As far as I can remember the shared memory needed 64bit OS and HW. I am not 100% sure yet. Eliezer I am not sure but vaguely recall that it works on 32bit when the

Re: [squid-users] configuring Eliezer RPMs for CentOS 6 for SMP

2014-05-17 Thread Marcus Kool
On 05/16/2014 06:47 PM, Fernando Lozano wrote: Hi, I don't quite agree with you. Let me expose my views so each member of the list can weight pros and cons: >Not answering this thread, but would like to ask some related points >for anyone who may be listening in: > >1. RPMs. > >For practical

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-07 Thread Marcus Kool
On 05/07/2014 10:55 AM, Pawel Mojski wrote: W dniu 2014-05-07 15:40, Marcus Kool pisze: [...] certificate chain: Certificate chain 0 s:/CN=*.gateway.messenger.live.com i:/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=MSIT Machine Auth CA 2 1 s:/DC=com/DC=microsoft/DC=corp/DC=redmond

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-07 Thread Marcus Kool
On 05/07/2014 06:44 AM, Pawel Mojski wrote: W dniu 2014-05-07 04:52, Jay Jimenez pisze: Hi Marcus and Amos, [...] I'm wondering if there's someone who successfully allowed Skype to fake CONNECT to squid (I'm referring to interception not explicit proxying). I cannot fully implement https i

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-02 Thread Marcus Kool
On 05/02/2014 08:21 AM, Jay Jimenez wrote: Hi Amos, Thank you for the response. Any advice of how would I know exactly what SSL/TLS version skype is using and how do I enable those versions to my squid box? It has been a while since I investigated Skype but my findings at that time were tha

Re: [squid-users] squid 3.4.4 and squidGuard 1.4 redirection 404 error

2014-04-24 Thread Marcus Kool
Or you switch to ufdbGuard which has active support, a configuration option to specify which version of Squid is used, regular updates and is 3x faster. ufdbGuard is free Open Source Software and can be downloaded from sourceforge.net and www.urlfilterdb.com Marcus On 04/24/2014 08:32 AM, Thom

Re: [squid-users] need help with ubuntu upgrade procedure

2014-03-25 Thread Marcus Kool
One way of doing this is to find the ubuntu spec file for the ubuntu package for Squid and use the spec file to build a new squid 3.4.x package. and then install the new package. This way all files locations will remain the same and you can also use the package manager to do an easy downgrade

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-23 Thread Marcus Kool
something about them. What are my options? I do want to lean more about these but I am not sure what to look where to look and how to look. I am looking for more directions about the subject since it's important and not only to me. Thanks, Eliezer On 22/01/14 17:06, Marcus Kool wrote: For th

Re: [squid-users] Re: Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-22 Thread Marcus Kool
On 01/22/2014 03:06 PM, babajaga wrote: IOs have a variable size and for writing an object to a file with the aufs store, the OS write meta data to the file system log, updates the inode table and writes the data to a new file. So for aufs for one logical 'write object to disk' there are 3 IOs

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-22 Thread Marcus Kool
? Thanks In Advance, Eliezer On 20/01/14 03:21, Marcus Kool wrote: The raw transfer speed of a disk is only interesting when an application does very large sequential I/Os and squid does not do that. Squid writes a lot to disk and reads relatively little and since the average object size is oft

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-19 Thread Marcus Kool
On 01/19/2014 04:42 AM, Eliezer Croitoru wrote: While working here and there I have seen that ZFS is a very robust FS. I will not compare it to any others because there is no need for that. OK so zfs, ext3, ext4 and others are FS which sits on SPINNING disks or flash drives. The SATA and SAS

Re: [squid-users] squid 3.4.1 UPGRADE WARNING: URL rewriter reponded with garbage '

2013-12-17 Thread Marcus Kool
On 12/16/2013 12:48 PM, jeffrey j donovan wrote: On Dec 15, 2013, at 10:43 PM, Eliezer Croitoru wrote: Please refer to: http://bugs.squid-cache.org/show_bug.cgi?id=3978 Which is a *bug* in squidguard due to change\upgrade of squid helpers interface. Eliezer squidguard communicates with

[squid-users] ufdbGuard 1.31-9 is released to support Squid 3.4.1

2013-12-16 Thread Marcus Kool
ufdbGuard, the URL redirector for Squid, has a new patch release to support the new URL rewriter protocol of Squid 3.4.1. This patch release introduces the keyword "squid-version" to support all versions of Squid. ufdbGuard 1.31-9 can be downloaded from http://sourceforge.net and http://www.u

Re: [squid-users] Re: SquidGuard not filtering

2013-12-04 Thread Marcus Kool
The quick and easy solution is to use ufdbGuard. ufdbGuard works like squidGuard but does not have issues like bad performance nor has it lack of support and maintenance. So if you want a free alternative for squidGuard, look at ufdbGuard. ufdbGuard can be downloaded from www.sourceforge.net or ww

Re: [squid-users] Squid and Squidguard using high disk IO

2013-11-09 Thread Marcus Kool
On Sat, Nov 09, 2013 at 11:16:12PM +0100, Loïc BLOT wrote: > Hello Kaya, > first, don't forget to look at sysctl kern.maxfiles values. > Also improve daemon FD values in login.conf for squid. Don't forget each > connection is a FD (1 connection for the client, 1 for the transaction > to remote sit

Re: [squid-users] Http works HTTPS gives cert errors. No errors in logs.

2013-10-18 Thread Marcus Kool
I think "Blocking HTTPS-based sites" needs to be added to the FAQ: Blocking HTTP is easy because the HTTP protocol has well-defined response codes to do this. HTTPS actually is SSL-wrapped HTTP and SSL does not allow any kind of interference, redirection or manipulation and cannot be blocked lik

Re: [squid-users] Squidguard, redirect and https

2013-10-17 Thread Marcus Kool
The problem is not Squid nor HTTPS. The problem is that the HTTP protocol has a standard that allows redirection and the HTTPS protocol does not. The HTTPS protocol was designed to be secure and does not allow any type of interference. So, all filtering technologies have the same issue: how to b

Re: [squid-users] content filtering question

2013-10-15 Thread Marcus Kool
On 10/15/2013 10:42 AM, Marko Cupać wrote: On Tue, 15 Oct 2013 10:26:59 -0300 Marcus Kool wrote: I suggest to look at ufdbGuard. It is a URL redirector for Squid and Thank you for the tip, I would like to try it. Is it possible to install it on FreeBSD? yes it is used on various

Re: [squid-users] content filtering question

2013-10-15 Thread Marcus Kool
On 10/15/2013 09:51 AM, Marko Cupać wrote: I am advancing into replacement of NTLM/dansguardian with kerberos/squid and icap and mapped AD groups with help of LDAP authorization, but there are a few things for which I haven't find solution so far: 1. More informative error messages for users

Re: [squid-users] Removing overlapping subdomains from blacklists

2013-08-21 Thread Marcus Kool
On Wed, Aug 21, 2013 at 05:27:55PM +0100, Andrew Wood wrote: > Hi > > Can someone please help me work out an algorithm to remove overlapping > subdomains from a blackclist such as shallalist to prevent errors such as: > > ERROR: 'interracialcandy.tumblr.com' is a subdomain of '.tumblr.com' > 2

Re: [squid-users] Squid Memory and Page Faults

2013-07-24 Thread Marcus Kool
On Wed, Jul 24, 2013 at 03:01:20PM -0700, Golden Shadow wrote: > Thanks Amos, Eliezer and Markus for your replies! > > @Eliezer: The server has 2 X 2.7 GHz CPUs, each with 12 cores. Squid version > is 3.3.7 compiled from source and I'm running only one squid worker. > > @Marcus: What is maximum

Re: [squid-users] Squid Memory and Page Faults

2013-07-24 Thread Marcus Kool
On Thu, Jul 25, 2013 at 06:58:56AM +1200, Amos Jeffries wrote: > On 25/07/2013 1:05 a.m., Golden Shadow wrote: > >Hi there! > > > >My squid is installed on a server with 192 GB of RAM. I have the following > >directives in squid.conf: > > > >cache_mem 143360 MB > >maximum_object_size_in_memory 300

[squid-users] ufdbGuard, the URL filter for Squid, has a new release

2013-06-27 Thread Marcus Kool
ufdbGuard v1.31 has been released on June 27, 2013. ufdbGuard is a free URL filter for Squid and can be used with your own, a free URL database or a commercial URL database. ufdbGuard has many other features which you can read about on http://www.urlfilterdb.com The main new features of ufdbGu

Re: [squid-users] parent request order

2013-06-24 Thread Marcus Kool
On 06/24/2013 12:44 PM, T Ls wrote: Am 24.06.2013 13:08, schrieb Marcus Kool: On 06/24/2013 06:01 AM, T Ls wrote: Am 19.6.2013 16:13, schrieb Marcus Kool: On Wed, Jun 19, 2013 at 09:27:54AM -0300, Marcus Kool wrote: On 06/19/2013 09:02 AM, T Ls wrote: >... What means "not a

Re: [squid-users] parent request order

2013-06-24 Thread Marcus Kool
On 06/24/2013 06:01 AM, T Ls wrote: Am 19.6.2013 16:13, schrieb Marcus Kool: On Wed, Jun 19, 2013 at 09:27:54AM -0300, Marcus Kool wrote: On 06/19/2013 09:02 AM, T Ls wrote: >... >What I want to do is: >a) switch to alternative parents automatically >b) in case of P_1-failure: di

Re: [squid-users] Squid performance profiling

2013-06-20 Thread Marcus Kool
On 06/20/2013 06:51 AM, Amos Jeffries wrote: If anyone is interested with very detailed benchmarks, then I can provide them. Yes please :-) PS. could you CC the squid-dev mailing list as well with the details. The more developer eyes we can get on this data the better. Although please do

Re: [squid-users] parent request order

2013-06-19 Thread Marcus Kool
On Wed, Jun 19, 2013 at 09:27:54AM -0300, Marcus Kool wrote: > > > On 06/19/2013 09:02 AM, T Ls wrote: > >Hi, > > > >we are using squid (S_{1,2,3}) to connect our sites to the internet via > >parent proxies (P_{1,2}), there is no direct line to the inte

Re: [squid-users] parent request order

2013-06-19 Thread Marcus Kool
On 06/19/2013 09:02 AM, T Ls wrote: Hi, we are using squid (S_{1,2,3}) to connect our sites to the internet via parent proxies (P_{1,2}), there is no direct line to the internet. The constellation looks something like this: (P_1) (P_2) |/ \ /-\ | | W

Re: [squid-users] Squid Hardware requirements.

2013-06-15 Thread Marcus Kool
On 06/15/2013 02:01 AM, csn233 wrote: The 14 MB per GB is documented in the Squid wiki and based on the >>>observation that the avergae object size is 13 KB. >>>If you only have 20-30% of the formula you may have a larger average >>>object size or only use 20-30% of the confgured disk cache. >

Re: [squid-users] Squid Hardware requirements.

2013-06-14 Thread Marcus Kool
On 06/14/2013 01:03 PM, csn233 wrote: On Fri, Jun 14, 2013 at 11:34 PM, Marcus Kool wrote: - more expensive (disks + battery-backed I/O controller) Expensive disks/battery-backed are over-kill. More/adequate spindles should do the job just as well. Why do you need a battery-backed

Re: [squid-users] Squid Hardware requirements.

2013-06-14 Thread Marcus Kool
On Fri, Jun 14, 2013 at 09:53:20PM +0800, csn233 wrote: > With YMMV in mind, I get different mileage: > > On Fri, Jun 14, 2013 at 7:41 PM, Marcus Kool > wrote: > > and if your network pipe has sufficient capacity, also fetching > > an object again from the internet is can

Re: [squid-users] Squid Hardware requirements.

2013-06-14 Thread Marcus Kool
On 06/14/2013 07:15 AM, Stephan Viljoen wrote: Hi There, I need to build a proxy server for an ISP handling about 4000 ip addresses over a 125Mbps of Internet bandwidth and were wondering what the specs for such a server would be? It's going to be a transparent squid server configured with Tpr

Re: [squid-users] Squid Optimization

2013-06-12 Thread Marcus Kool
, Jun 11, 2013 at 6:36 PM, Marcus Kool wrote: On 06/11/2013 06:43 PM, Matthew Ceroni wrote: We are running squid as our primary proxy here at my office. What I am noticing is that connectivity is fine but every now and then the browser sits with "Sending request". If I hope on the

Re: [squid-users] Squid Optimization

2013-06-11 Thread Marcus Kool
On 06/11/2013 06:43 PM, Matthew Ceroni wrote: We are running squid as our primary proxy here at my office. What I am noticing is that connectivity is fine but every now and then the browser sits with "Sending request". If I hope on the proxy and view the access log I don't see it logging my re

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-11 Thread Marcus Kool
On 06/11/2013 10:33 AM, Eliezer Croitoru wrote: ufdbGuard does not pause answering queries from Squid during a reload since that would pause Squid and is considered an interruption of service. ufdbGuard releases the current URL database, loads a new configuration and loads a new URL database i

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-11 Thread Marcus Kool
On 06/11/2013 11:57 AM, Jose-Marcio Martins wrote: On 06/11/2013 02:36 PM, Marcus Kool wrote: So my question is... a more direct and precise question, just about ufdbGuard. While ufdbGuard reloads its URL database, does it pauses answering queries ? If yes, how long does it takes

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-11 Thread Marcus Kool
On 06/11/2013 09:09 AM, Jose-Marcio Martins wrote: On 06/11/2013 12:50 PM, Marcus Kool wrote: There is a big misunderstanding: in the old days when the only URL filter was squidguard, Squid had the be reloaded in order for squidguard to reloads its database. And when Squid reloads

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-11 Thread Marcus Kool
On 06/11/2013 04:13 AM, Jose-Marcio Martins wrote: Thanks for the info, On 06/11/2013 05:26 AM, Amos Jeffries wrote: On 11/06/2013 9:03 a.m., Jose-Marcio Martins wrote: When Squid reloads it pauses *everything* it is doing while the reload is happening. * 100% of resources get dedicated

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-10 Thread Marcus Kool
ons out there... Or maybe I didn't understood what you're talking about. On 06/10/2013 05:43 PM, Squidblacklist wrote: On Mon, 10 Jun 2013 12:16:40 -0300 Marcus Kool wrote: [discussion about proposal 1 deleted] About solution 2: Consider the following scenario: Suppose the pare

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-10 Thread Marcus Kool
Actually, I proposed two solutions. While the bash script is messy I will admit, the optimal solution of having a parent and child proxy is rather elegant, fault tolerant, and works without issue. The child proxy simply ignores and bypasses the parent proxy while the reload procedure is underwa

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-09 Thread Marcus Kool
On 06/09/2013 12:59 PM, Alex Rousskov wrote: On 06/09/2013 03:29 AM, Eliezer Croitoru wrote: >Would you prefer a filtering based on a reload or a persistent DB like >mongoDB or tokyo tyrant? I would prefer to improve Squid so that reconfiguration has no disrupting effects on traffic, elimina

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-09 Thread Marcus Kool
On 06/09/2013 10:06 AM, Eliezer Croitoru wrote: On 6/9/2013 3:52 PM, Marcus Kool wrote: I do not understand the performance figure. Can you give more details ? Best regards, Marcus Yes indeed. The performance of an ICAP service is in another level from helper since it has concurrency

Re: [squid-users] what are the Pros and cons filtering urls using squid.conf?

2013-06-09 Thread Marcus Kool
On 06/09/2013 06:29 AM, Eliezer Croitoru wrote: I have seen http://www.squidblacklist.org/ which is a very nice idea but I am wondering if squid.conf and other squid products are the good choice for any place. For a mission critical proxy server you will need to prevent any "reload" of the

[squid-users] squidguard not redirecting

2013-06-05 Thread Marcus Kool
hi, did you see this message ? On Fri, May 24, 2013 at 01:49:10PM -0300, Carlos Defoe wrote: > Well, it seems that the problem has returned. > > Now i think it might be caused when logrotate runs, and therefore > "squid -k rotate". It's just a guess, since rotating is the only > action that affe

Re: [squid-users] Rewriting URL after the ACL check

2013-05-27 Thread Marcus Kool
You have a loop. The HTTP 302 code says to the browser: re-fetch the content and use the new URL. So, when you rewrite www.example.com to 302:www.example.com/?foo=bar it works well, goes back to the browser and the browser does a GET www.example.com/?foo=bar which gets processed by Squid and also

Re: [squid-users] squidguard not redirecting

2013-05-18 Thread Marcus Kool
On 05/17/2013 11:40 PM, csn233 wrote: You can use ufdbGuard free. So it's the filter DB component that's not free. Thanks for clarifying. No. ufdbGuard is free software, the same as squidguard. ufdbGuard works with free databases or your own URL blacklist, just like squidguard. ufdbGuar

Re: [squid-users] squidguard not redirecting

2013-05-17 Thread Marcus Kool
On Sat, May 18, 2013 at 01:27:36AM +0800, csn233 wrote: > On Sat, May 18, 2013 at 1:09 AM, csn233 wrote: > >>> I didn't bother spending any more time on it. > >> > >> ufdbGuard is a more powerful substitute of squidGuard. > >> Has regular updates and even free support. > >> > >> Marcus > > Sure.

Re: [squid-users] squidguard not redirecting

2013-05-17 Thread Marcus Kool
On Sat, May 18, 2013 at 12:28:20AM +0800, csn233 wrote: > > Has anyone experienced this? > > SG has numerous problems which caused it not to do what it's supposed > to, including that "emergency" mode thing. Here are some things to > consider: > > 1) a BIG blacklist is overhyped - when I had a go

Re: [squid-users] Re: slow browsing in centos 6.3 with squid 3 !!

2013-02-25 Thread Marcus Kool
On 02/25/2013 01:51 PM, Ahmad wrote: hi Marcus , u must be correct , i have 32 G total memory , ive confiured in cache_mem =2 M which is 20 G !! ive noticed that squid hanged and got down after some hours of operating !! i looked at all logs of squid and squidguard , i didnt find any err

Re: [squid-users] Re: slow browsing in centos 6.3 with squid 3 !!

2013-02-25 Thread Marcus Kool
Ahmad, Also look at http://wiki.squid-cache.org/SquidFaq/SquidMemory Hint: a 1500 GB disk cache requires a very large in-memory index. Also reserve memory for squidguard and the OS. Marcus On 02/25/2013 05:18 AM, Ahmad wrote: Amos Jeffries-2 wrote On 25/02/2013 12:30 a.m., Ahmad wrote: he

Re: [squid-users] Re: slow browsing in centos 6.3 with squid 3 !!

2013-02-24 Thread Marcus Kool
Ahmad, If you think the problem is squidguard, you have to get sure about this. I suggest to disable squidguard and see if the performance get better to confirm that the bottleneck is indeed caused by squidguard. IF it is confirmed that squidguard is the bottleneck you can either try to optimise

[squid-users] The URL filter ufdbGuard v1.30 has been released

2013-01-23 Thread Marcus Kool
chat - SafeSearch enforcement for many search engines - time-based user policies - supports the free anti-phishing URL table from www.phishtank.com - supports monitoring - provides basic statistics - provides a mini-HTTP server to display a suitable text/image for blocked objects Marcus Kool

Re: [squid-users] Filter by time and white-black lists

2013-01-22 Thread Marcus Kool
On 01/22/2013 02:10 AM, Amos Jeffries wrote: On 22/01/2013 2:22 a.m., Marcus Kool wrote: Although Squid ACLs can do almost anything they are not the simplest thing to configure. A URL redirector like ufdbGuard offloads work from Squid and can do the same thing using a more intuitive

Re: [squid-users] Filter by time and white-black lists

2013-01-21 Thread Marcus Kool
Although Squid ACLs can do almost anything they are not the simplest thing to configure. A URL redirector like ufdbGuard offloads work from Squid and can do the same thing using a more intuitive configuration: source admin { ip 10.77.88.47 } source managers { ip 10.77.88.42-10.77.88.46 } s

Re: [squid-users] Squid3 extremely slow for some website cnn.com

2012-12-11 Thread Marcus Kool
I have problems with optimizely.com for a long time. When I use a proxy in the USA, retrieving websites is OK but when I stop using the proxy in the USA it fails to load. I live in Brazil and you have a problem being in Egypt so my guess is that optimizely.com hangs when you are in a country for

Re: [squid-users] Allowing skype through on an ssl bumped proxy

2012-11-27 Thread Marcus Kool
I have seen this issue on 3.1.x and cannot find anything in the Changelog that indicates that this issue is resolved in 3.3. What I observed in 3.1 is that sslbump assumes that all CONNECTs are used for SSL-wrapped HTTP traffic and lets all applications that use port 443 for other protocols hang

Re: [squid-users] Trouble filtering/denying HTTPS traffic

2012-10-17 Thread Marcus Kool
On 10/17/2012 02:18 AM, Amos Jeffries wrote: On 17/10/2012 4:08 p.m., Cameron Charles wrote: Hi all, I am currently trying to setup basic "url/domain level" filtering on HTTPS traffic using an external acl, i can see clearly in the access log that the information i require is there and the ex

Re: [squid-users] From Squid 2.7 STABLE 6 to....

2012-10-11 Thread Marcus Kool
In case of replacement of failing hardware, I suggest to install the same as you had: 2.7stable6 will work for you. After the hardware replacement I recommend to plan and test an upgrade. Although the Squid development team recommends 3.2.x I prefer to go for 3.1.x because at this time there are

Re: [squid-users] Valgrind results on 3.2.1

2012-09-27 Thread Marcus Kool
On 09/27/2012 03:39 PM, t...@raynersw.com wrote: Some new stuff has been logged by valgrind overnight- it's at the bottom of this e-mail. Looks like mostly dupes but I see some that might be new. However I still don't see any accounting for the amount of leakage I'm seeing. Right now, squid

Re: [squid-users] Which RAID for this ?

2012-09-17 Thread Marcus Kool
There is a FAQ about RAID: http://wiki.squid-cache.org/SquidFaq/RAID On 09/17/2012 01:01 PM, Will I am wrote: It's for about 500 users connections. 2012/9/17 Baird, Josh: I would probably do one RAID1 for the OS, and then one RAID10 for everything else... but, it really depends on how much

Re: [squid-users] Migration from Squid 2 to 3: ACLs no longer accepting duplicates

2012-08-30 Thread Marcus Kool
On 08/29/2012 06:13 PM, Derek Simon wrote: Nichols, It appears that I need to preprocess the lists before handing them to Squid, removing all subdomains were the parent domain is present. I don't see anything special about your configuration that would somehow remedy this issue. Perhaps you

[squid-users] ufdbGuard v1.29 has been released

2012-08-29 Thread Marcus Kool
Version 1.29 of ufdbGuard, the free URL filter for Squid, has been released. This release fixes 10 issues and gained support for the URL list of PhishTank, support for new URL subcategories and has other new features. Visit http://www.urlfilterdb.com/en/release-1.29.html for all details.

Re: [squid-users] File corruption with reverse proxy

2012-08-28 Thread Marcus Kool
On 08/28/2012 10:21 AM, Andy Osborne wrote: Hi All, I'm trying to debug a problem at the moment where a small proportion of people visiting our sites (Squid 3.1.20 as a reverse proxy with Apache origin servers) are seeing file corruption. So far all the confirmed cases are using Windows 7 wit

Re: AW: [squid-users] [Squidguard] squidGuard stops blocking randomly after a while

2012-08-27 Thread Marcus Kool
If squidGuard has a showstopper, I suggest to look at ufdbGuard. ufdbGuard has regular releases, a support desk, and additional features to block VPNs, block unsafe HTTPS traffic, enforce SafeSearch and much higher performance. Marcus On 08/27/2012 08:12 AM, Stefan Bauer wrote: -Ursprüngli

Re: [squid-users] Squid and huge list

2012-08-25 Thread Marcus Kool
If Squid on your system with the large domainlist consumes on average more than 75% CPU. On 08/24/2012 11:12 PM, Rafael Gomes wrote: Marcus, The Squid consumes 75% CPU each request or just when starts? On Fri, Aug 24, 2012 at 7:13 PM, Marcus Kool wrote: Rafael, although Squid can handle

Re: [squid-users] Squid and huge list

2012-08-24 Thread Marcus Kool
Rafael, although Squid can handle large lists of domains and URLs, squidguard and ufdbguard are faster. So if your Squid process consumes 75% CPU or more, I recommend to use ufdbguard (3x faster than squidguard). Marcus On 08/24/2012 06:03 PM, Rafael Gomes wrote: hi guys, I know it may be a

Re: [squid-users] shared memory seems to allow size of 32K **1KB** segments (32MB)...

2012-08-07 Thread Marcus Kool
Amos referred to a mail thread where the issues of the 32K limit were discussed. This thread does not suggest that 32K is a hard shared memory system limit but merely that 32K is a Squid-specific design limitation. Oracle and other RDBMS use very large shared memory segments of multiple gigabyte

Re: [squid-users] Low memory usage

2012-07-20 Thread Marcus Kool
And if you want Squid to respond fast to cache cleanups, change >> cache_swap_low 85 >> cache_swap_high 95 into cache_swap_low 94 cache_swap_high 95 Marcus On 07/20/2012 10:53 AM, Marcus Kool wrote: If you have 14 GB physical memory, it is not recommended to set cache_mem 14 GB.

Re: [squid-users] Low memory usage

2012-07-20 Thread Marcus Kool
If you have 14 GB physical memory, it is not recommended to set cache_mem 14 GB. See the FAQ for more details: http://wiki.squid-cache.org/SquidFaq/SquidMemory Marcus On 07/20/2012 10:17 AM, Guido Marino Lorenzutti wrote: Hi people! Im running 3.1.20 with ntlm and it works much better! I was u

Re: [squid-users] slow reconfigure on squid3

2012-07-04 Thread Marcus Kool
s than the format of ufdbGuard reducing further the need for memory resources. Marcus cheers Jim UK On 2 July 2012 14:44, Marcus Kool wrote: Squid reconfigure can indeed take a long time. Especially when Squid uses lots of memory and starts helpers. Starting helpers takes a large amount

Re: [squid-users] slow reconfigure on squid3

2012-07-02 Thread Marcus Kool
Squid reconfigure can indeed take a long time. Especially when Squid uses lots of memory and starts helpers. Starting helpers takes a large amount of kernel resources when Squid is large, e.g. 2+ GB since it forks itself and replaces its copy by a new process. The fork can take a long time. If y

Re: [squid-users] acl forbidden_domains dstdom_regex "file.txt" with huge file fails

2012-06-20 Thread Marcus Kool
On 06/20/2012 06:43 AM, Matus UHLAR - fantomas wrote: On 19.06.12 18:52, Stefan Bauer wrote: with a 30 MB file. Squid is instantly terminating if this acl-stanza is set active. Where can and do we have to tune squid settings to achive this? terminating with what reason? I would not wonder i

Re: [squid-users] Redirector idea using request header

2012-05-05 Thread Marcus Kool
On 05/05/2012 06:31 AM, gslavic wrote: Hi, there, Is there a way to read "Request Header" BEFORE it is writen in access.log ? Example: There are three versions of internet site. One for Simbian phones, one for Android phones, one for regular internet browsers. In request header there is a field

Re: [squid-users] Re: how the big guys are doing it - Caching dynamic content with squid

2012-04-26 Thread Marcus Kool
On 04/26/2012 01:07 PM, x-man wrote: Hi Marcus, thanks for reply. I just came to know from you about the ICAP solution. As far as I get it, it will adapt the content so the SQUID can cache it itself, by making the dynamic stuff in appropriate way so that squid can consume it as non dynamic? I

Re: [squid-users] how the big guys are doing it - Caching dynamic content with squid

2012-04-25 Thread Marcus Kool
I don't know how others (a.k.a. the "big guys") are doing it, but I have a suggestion: YouTube and others sometimes change the way they redirect a browser to different stream servers and use different URLs for the same video, so the solution should be configurable with a configuration file saying

Re: [squid-users] slow internet browsing.

2012-04-24 Thread Marcus Kool
A 10 MB flat file of domains contains roughly 50 domains. You also use time-based ACLs and ACLs based on IP address range. This config can be implemented using ufdbGuard, a free URL rewriter for Squid. ufdbGuard is better maintained than squidGuard and has more features. Marcus On 04/24/2

Re: [squid-users] problem with loaded server

2012-04-09 Thread Marcus Kool
and can by noticed clearly in the top program, my squid is 2.5stable9, and the os is fedora thanks with my best regards --- On Sat, 4/7/12, Marcus Kool wrote: From: Marcus Kool Subject: Re: [squid-users] problem with loaded server To: "Mustafa Raji" Cc: squid-users@squid-cach

Re: [squid-users] problem with loaded server

2012-04-07 Thread Marcus Kool
On 04/07/2012 04:37 PM, Mustafa Raji wrote: hi i have server which i think it's overloaded i checked the TIME_WAIT TCP connection status it's normal and below the limits of the os, and file descriptor is set for high value and no warning in cache.log about the file descriptor, so i think it's

Re: [squid-users] Metrics to calculate 'best' values for cache_mem and cache_dir?

2012-03-27 Thread Marcus Kool
On 03/27/2012 10:42 AM, Peter Gaughran wrote: With the recent (enlightening) posts from Christian, Marcus et al, I was wondering what the metrics actually are for calculating cache_mem and cache_dir? I've read (for Linux at least) aufs is superior to diskd, so I've made the appropriate decisio

Re: [squid-users] Problems with squid in a campus setup

2012-03-27 Thread Marcus Kool
But read the FAQ about memory usage and a large disk cache: http://wiki.squid-cache.org/SquidFaq/SquidMemory Squid uses an additional 512*14 MB = 7.1 GB for the index of the disk cache. I suggest to downsize to 1 GB in-memory index which implies to use only 73 GB disk cache. Ah okay, here's

Re: [squid-users] Problems with squid in a campus setup

2012-03-26 Thread Marcus Kool
Youtube may be hogging your pipe but it is better to know than to guess. The access.log shows content sizes so with a simple awk script it should be easy to find out. I have also seen many sites where advertisements and trackers consume 15% bandwidth. This may vary. So blocking ads and trackers i

Re: [squid-users] Squid - Content Filter

2012-03-23 Thread Marcus Kool
ufdbGuard is an active fork of squidGuard. ufdbGuard also does HTTPS probing for safer HTTPS traffic and has safesearch enforcement. On 03/23/2012 05:04 AM, JC Putter wrote: Hi, We are using squid with Kerberos authentication (AD). What is a good content filter to use these days? Dansguardi

Re: [squid-users] blacklist

2012-03-04 Thread Marcus Kool
ufdbGuard is free software. it works with free and commercial URL databases. Esteban Torres Rodríguez wrote: El día 4 de marzo de 2012 16:03, David Touzeau escribió: Have you tried ufdbguard (www.urlfilterdb.com) ? More fastest than squidguard I need something free. Le 04/03/2012 13:37,

Re: [squid-users] squid "stops working" several times a day

2012-02-29 Thread Marcus Kool
the reads have many errors, about 25% and your other email shows the error code EAGAIN. All calls to connect() fail and all calls to recfrom() fail. Its seems to me that the system has a resource problem. Running out of file descriptor or other system resources. I suggest to look at /var/log/me

Re: [squid-users] URL rewrite on Squid 3.1.6 as ReverseProxy for Exchange OWA

2012-02-20 Thread Marcus Kool
For HTTP is receives the full URL but for HTTPS it only receives the domainname. The URL rewriter feature was designed to rewrite HTTP-based URLs and cannot rewrite HTTPS-URLs. Marcus Fried Wil wrote: Hi Guys, Thx @Eliezer for reply. I know redirection page, thx :), but i want to use squidg

Re: [squid-users] Caching in Afghanistan

2012-02-19 Thread Marcus Kool
And use a blocker for ads and trackers. You will be surprised to see how much bandwidth is wasted with them. Pieter De Wit wrote: On 18/02/2012 23:56, jbrodi...@gci.net wrote: Hello there everyone, I'm currently deployed to Afghanistan and have recently set up a VSAT connection with approximate

Re: [squid-users] Squid ICAP Problem

2012-02-19 Thread Marcus Kool
gVeR SoNiC wrote: Hello everbody. I'm using Squid3 (squid/3.1.14) on Ubuntu 11.10. I'm using it with my own-made ICAP server which is RFC 3507 compliant. The problem is: When chunked transfer encoding occurs (encapsulated http message has a body), sometimes it fails with my send function giv

Re: [squid-users] Email , VOIP , RDP with SQUID

2012-02-17 Thread Marcus Kool
Muhammad, have you looked at ufdbGuard? It is a free URL filter with time-based ACLs, ACLs for groups, user-defined sets of URLs, support for free and commercial URL databases. The only major item on your wishlist that ufdbGuard does not do, is the delay pool, but that is something you can do wit

Re: [squid-users] Squid block list

2012-02-15 Thread Marcus Kool
ufdbGuard is a free URL filter for Squid which has the time-related ACL feature to block sites only during business hours. The Reference Manual of ufdbGuard explains the technical details. If you have a small set of sites that you want to block, you can make your own URL table and use ufdbGuard f

Re: [squid-users] ICAP Processing Times

2012-01-27 Thread Marcus Kool
The ICAP protocol does not stream in the sense that it forwards piece by piece. The ICAP protocol only supports a preview which for Squid has a maximum of 64 KB. So a large file with preview mode enabled, can send a configurable size (between 1 byte and 64 KB) of the first part of the content to

[squid-users] ufdbGuard v1.28 is out - check out the URL filter for Squid

2012-01-19 Thread Marcus Kool
ufdbGuard version 1.28 has been released on January 19, 2012. ufdbGuard is a URL filter for Squid with the following features: - filter web access based on rules for users, times, website category - works with free and commercial URL databases - can enforce SafeSearch for all major search engines

Re: [squid-users] blacklist to block adults sites

2011-08-29 Thread Marcus Kool
Alexus, Many tried and failed. It is not possible to filter accurately based on a list of words. You need a professional filter solution. ufdbGuard is a free URL filter for Squid. It works with both free URL databases and a commercial database. ufdbGuard produces feedback about the quality of

Re: [squid-users] squid redirect question

2011-07-28 Thread Marcus Kool
Look at ufdbGuard, a free replacement for squidGuard, which includes a mini HTTP server, ufdbhttpd, that is only used for the redirects of ufdbGuard. ufdbhttpd has no config file, no problems with installation. Marcus Bruce Bauman wrote: I am running a web browser, squid, and squidguard all on

Re: [squid-users] I see this error in cache.log file no free membufs

2011-07-20 Thread Marcus Kool
The message indicates that the numbers of membufs should be because there are insufficent membufs to use for caching objects. The reason for having 'insufficient membufs' is explained below. Given the fact that the average object size is 13 KB, the given configuration effectively puts a very lar

Re: [squid-users] Streaming video content (e.g. BBC news to flash based player) - always stops?

2011-07-18 Thread Marcus Kool
b.scorecardresearch.com/b There are many sites that use tracking URLs/gifs and they can slow down end user experience. Marcus Karl Pielorz wrote: --On 08 July 2011 12:47 -0300 Marcus Kool wrote: Well, I still would like to know the URL because I like to observe which set of URLs this eventually

Re: [squid-users] Streaming video content (e.g. BBC news to flash based player) - always stops?

2011-07-08 Thread Marcus Kool
e has encountered and found a solution to this issue. BMatz -Original Message- From: Marcus Kool [mailto:marcus.k...@urlfilterdb.com] Sent: Friday, July 08, 2011 6:28 AM To: Karl Pielorz Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Streaming video content (e.g. BBC news to

Re: [squid-users] Streaming video content (e.g. BBC news to flash based player) - always stops?

2011-07-08 Thread Marcus Kool
I live in Brazil and sometimes watch BBC videos using Squid without issues. Can you give a link to an example URL which causes problems ? Marcus Karl Pielorz wrote: Hi, We've tried running a number of Squid versions (from 2.7.9 through to 3.2.0.9) but they all seem to suffer when streaming

  1   2   3   >