Re: [squid-users] Squid high memory usage

2016-06-15 Thread Marcus Kool
On 06/15/2016 04:30 AM, FredB wrote: Maybe I'm wrong, but the server is also using many memories for TCP cat /proc/net/sockstat sockets: used 13523 TCP: inuse 8612 orphan 49 tw 31196 alloc 8728 mem 18237 UDP: inuse 14 mem 6 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 netstat

Re: [squid-users] Redirect after sslbump teminate

2016-06-12 Thread Marcus Kool
On 06/12/2016 12:34 PM, Eng Hooda wrote: Hello Squid Users, I have searched for this but I could not find an answer. After I peek for media streaming sites using sslbump , I terminate the connection on match , which produces secure connection failed on the client browser . Is there a way to

Re: [squid-users] Peek'n Splice (ssl_bump) and authentication Somewhat OT: Content Filter with https

2016-06-10 Thread Marcus Kool
On 06/09/2016 11:26 PM, Sergio Belkin wrote: 2016-06-08 20:30 GMT-03:00 Marcus Kool <marcus.k...@urlfilterdb.com <mailto:marcus.k...@urlfilterdb.com>>: On 06/08/2016 07:53 PM, Sergio Belkin wrote: Thanks Eliezer, good summary. I've changed the subject to ref

Re: [squid-users] Peek'n Splice (ssl_bump) and authentication Somewhat OT: Content Filter with https

2016-06-08 Thread Marcus Kool
On 06/08/2016 07:53 PM, Sergio Belkin wrote: Thanks Eliezer, good summary. I've changed the subject to reflect better the issue. As far I undestand from documention one can bump https only by interception. No. ssl-bump works very well with regular proxy mode, i.e. the browsers configure

Re: [squid-users] Somewhat OT: Content Filter with https

2016-06-08 Thread Marcus Kool
On 06/08/2016 05:54 PM, Sergio Belkin wrote: - Not need of interception. is that possible? It depends. If you support smartphones, you most likely need interception since not all apps can be configured to use a proxy. With only desktops, interception is not required but

Re: [squid-users] Somewhat OT: Content Filter with https

2016-06-08 Thread Marcus Kool
On 06/08/2016 05:05 PM, Sergio Belkin wrote: Hi, I've been using a few years ago squid+dansguardian. But nowadays, DG is not maintained anymore. I know that exists squidGuard, ufdbGuard, and e2guardian. Features should be: - Blocking https url's Blocking HTTPS URLs is easy. However,

Re: [squid-users] Squid high memory usage

2016-06-06 Thread Marcus Kool
On 06/06/2016 07:27 AM, FredB wrote: Thanks for your answer What is cache_mem ? See also http://wiki.squid-cache.org/SquidFaq/SquidMemory Actually 25 Gb I tried different values, but I guess no matter, the problem is that the squid limit is only 50% of ram After that the swap is

Re: [squid-users] Squid high memory usage

2016-06-06 Thread Marcus Kool
On 06/06/2016 04:27 AM, FredB wrote: Hello all, I'm trying to use a server with 64 Go of ram, but I'm faced with a problem, squid can't works with more than 50% of memory What is cache_mem ? See also http://wiki.squid-cache.org/SquidFaq/SquidMemory After that the swap is totally full and

Re: [squid-users] runing squid on second processor

2016-05-01 Thread Marcus Kool
On 04/29/2016 07:17 PM, joe wrote: hi i have 2 cpu 4 core each i need to leave alone first processor and use the second one for squid and its helper is that will do ??? taskset 0x00f0 squid -YC -f /etc/squid/squid.conf or other way around ?? so i can keep the kernel and other program

Re: [squid-users] Survey on assertions: When the impossible happens

2016-02-29 Thread Marcus Kool
This is not really #4. It is an enhancement for any of the three options. IIRC, Squid even supported gdb stack tracing natively on some platforms (but a script would arguably be better, except for busy proxies that cannot be blocked for 2-4 seconds it takes to run that script). This already

Re: [squid-users] Survey on assertions: When the impossible happens

2016-02-29 Thread Marcus Kool
* Choices. Overall, there are three options for handling an impossible situation: 1. Quit Squid process. This is what Squid does today in most cases. When the impossible happens, you get a crash. Very predictable. No malformed/corrupted/misleading HTTP messages (some are truncated).

Re: [squid-users] URL Rewrite for https via Squidguard

2016-01-09 Thread Marcus Kool
hope and a possible way forward. regards Darren B. Sent from Mailbird <http://www.getmailbird.com/?utm_source=Mailbird_medium=email_campaign=sent-from-mailbird> On 9/01/2016 5:46:36 PM, Marcus Kool <marcus.k...@urlfilterdb.com> wrote: On 01/09/2016 05:07 AM, Darren wrote

Re: [squid-users] URL Rewrite for https via Squidguard

2016-01-09 Thread Marcus Kool
On 01/09/2016 05:07 AM, Darren wrote: Hi I am trying to hack squidguard to allow me to redirect users attempts to connect to blocked https enabled sites. Some sites are allowed and the bulk are not. Currently I can see the Connect details being handed to SG for processing and if I change

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2016-01-08 Thread Marcus Kool
On 01/07/2016 06:48 PM, Jason Haar wrote: On 08/01/16 01:56, Marcus Kool wrote: Can you explain what the huge number of regexes is used for ? malware urls. I'm scraping them from publicly available sources like phishtank, malwaredomains.com. Ironically, they don't need to be regexes

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2016-01-07 Thread Marcus Kool
On 01/07/2016 12:31 AM, Jason Haar wrote: On 06/01/16 00:04, Amos Jeffries wrote: Yes. Squid always has been able to given enough RAM. Squid stores most ACLs in memory as Splay trees, so entries are sorted by frequency of use which is dynamically adapted over time. Regex are pre-parsed and

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2015-12-28 Thread Marcus Kool
On 12/28/2015 08:46 PM, George Hollingshead wrote: I've had squid3.0 running with squidGuard on my old ubuntu 10.04 system with no problems for a few months now. I just recently was enlightened by Yuri how to compile using a local copy of openssl so i could upgrade to latest squid. This

Re: [squid-users] Host header forgery affects pure splice environment too?

2015-12-28 Thread Marcus Kool
On 12/28/2015 01:33 AM, Jason Haar wrote: On 28/12/15 14:34, Amos Jeffries wrote: [...] I think we know what the problem is: TOR is making TLS connections (I don't know if they're HTTPS) on port 443 and uses SNI names that aren't real? peeking on tor-proxy-2.cypherpunks.to shows a

Re: [squid-users] Using subordinate CA for SSL Bump

2015-12-14 Thread Marcus Kool
On 12/14/2015 09:16 PM, Amos Jeffries wrote: With all that looking hopeful, and the certs identified as the secondary chain being attached (everything except the firstprimary/signing cert). I'm not actually finding anywhere sending the actual signing certificate itself during the bumping

Re: [squid-users] Peek and splice without replacing the certificates

2015-12-14 Thread Marcus Kool
On 12/14/2015 06:43 AM, Парфенович Н.А. wrote: Hello! Show you how to use Squid in transparent mode for tracking HTTPS without replacing the certificates? My squid.conf: http://pastebin.ru/AWU8LXvK. If such a configuration file to use version 3.5.8 squid compiled using Libressl, everything

Re: [squid-users] Squid memory leak on ubuntu 14.04

2015-11-27 Thread Marcus Kool
I do not have the detail of Ubuntu 14.04 but most likely 12.04 and 14.04 have a different version of malloc (see "man malloc") which allocates gigabytes of virtual memory. Most likely you see in top that the resident memory is what you expect that Squid uses (comparable as on 12.04) and the

Re: [squid-users] Block google pictures

2015-11-27 Thread Marcus Kool
You can force Google safesearch, even with HTTPS. Google only needs that you put a CNAME entry in your DNS server for www.google.com. See https://support.google.com/websearch/answer/186669?hl=en Option 3 for more information. Marcus On 11/26/2015 12:27 PM, Funke, Martin wrote: Im using

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-12 Thread Marcus Kool
with HTTP is simple because the HTTP protocol has a built-in mechanism for redirection that proxies can use. Marcus I can also provide squid logs, but tell me what because I've got a lot... Regards, EG Le 05/11/2015 14:01, Marcus Kool a écrit : On 11/04/2015 08:55 PM, Edouard Gaulué wrote

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-12 Thread Marcus Kool
I cannot make much of the logs and expect that information is missing. But using just logic, it seems that Squid has a problem with the redirect to a CONNECT. I suggest to set debug all,9 and to look closely at what happens with the redirection. Marcus On 11/12/2015 10:02 AM, Edouard Gaulué

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-05 Thread Marcus Kool
ice stuff leading squid to an unpredictable situation. Is there a way to play on order things happen in squid? Regards, EG Le 04/11/2015 14:10, Marcus Kool a écrit : You need to know what squidGuard actually sends to Squid. squidGuard does not have a debug option for this, so you have to set d

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-04 Thread Marcus Kool
' Marcus On 11/04/2015 10:55 AM, Edouard Gaulué wrote: Le 04/11/2015 11:00, Amos Jeffries a écrit : On 4/11/2015 12:48 p.m., Marcus Kool wrote: I suspect that the problem is that you redirect a HTTPS-based URL to an HTTP URL and Squid does not like that. Marcus To give it a try in that direction I

Re: [squid-users] ICAP and HTTPS

2015-10-06 Thread Marcus Kool
On 10/06/2015 06:05 PM, Rafael Akchurin wrote: Hello Paul, Eliezer, Alex, We (diladele ICAP) have an open bug /feature requests for this: https://github.com/ra-at-diladele-com/qlproxy_external/issues/731 https://github.com/ra-at-diladele-com/qlproxy_external/issues/726 As

Re: [squid-users] Ssl-Bump and revoked server certificates

2015-10-06 Thread Marcus Kool
On 10/06/2015 07:18 PM, Jason Haar wrote: On 06/10/15 23:21, Walter H. wrote: Hello, can you please provide an example of how to use this in squid.conf #create external acl checker that returns "ERR" or "OK" based on cert data sent to it external_acl_type checkIfHTTPS children-max=20

Re: [squid-users] On what methods does url filtering needs to apply?

2015-09-28 Thread Marcus Kool
"content filtering" may filter only content while a generic filter may filter anything including malware that uses PUT, OPTION and/or HEAD to upload credit card data. So it depends on what you want to filter. If it is downloadable content only, you can stick with filtering GET POST CONNECT.

Re: [squid-users] after changed from 3.4.13 to 3.5.8 sslbump doesn't work for the site https://banking.postbank.de/

2015-09-26 Thread Marcus Kool
On 09/26/2015 03:03 PM, Dieter Bloms wrote: Hallo Marcus, On Thu, Sep 17, Marcus Kool wrote: I just tried accessing https://banking.postbank.de/ using Squid 3.5.8 and Chrome. I also got the ERR_CONNECTION_CLOSED error. thank you for testing, so I think the fault is not my config. May

Re: [squid-users] stoping after rotate

2015-09-09 Thread Marcus Kool
nior <jorge...@gmail.com <mailto:jorge...@gmail.com>>: ok, I'll do it 2015-09-08 21:30 GMT-03:00 Marcus Kool <marcus.k...@urlfilterdb.com <mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 09:23 PM, Jorgeley Junior wrote:

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
mentioned that the swap is 32 GB. What is the size of the physical memory ? Did you already increase the swap ? Marcus 2015-09-05 15:08 GMT-03:00 Marcus Kool <marcus.k...@urlfilterdb.com <mailto:marcus.k...@urlfilterdb.com>>: On Linux, an important sysctl parameter that de

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
:00 Marcus Kool <marcus.k...@urlfilterdb.com <mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 08:11 AM, Jorgeley Junior wrote: Thank you all, this is the output: vm.overcommit_memory = 0 vm.swappiness = 60 I have a Redhat 6.6

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
20:25 GMT-03:00 Marcus Kool <marcus.k...@urlfilterdb.com <mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 10:39 AM, Jorgeley Junior wrote: I have 8GB physical memory and my swap is 32GB. I didn't increase the swap yet, should I? You must start

Re: [squid-users] stoping after rotate

2015-09-05 Thread Marcus Kool
On Linux, an important sysctl parameter that determines how Linux behaves with respect to VM allocation is vm.overcommit_memory (should be 0). And vm.swappiness is important to tune servers (should be 10-15). Which version of Linux do you have and what is the output of sysctl -a | grep -e

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
On 09/01/2015 05:14 AM, FredB wrote: More precisely I reduced the ttl of the first line refresh_pattern -i \.(htm|html|xml|css)(\?.*)?$ 10080 100% 10080 #All File 30 days max refresh_pattern -i \.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt)(\?.*)?$ 43200 100% 43200

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
And, of course, universal rule for store_id_access. I think that this works well for trackers gifs but not for other gifs with parameters. Store ID is powerful instrument for deduplication cache story. Which permits not to use terabytes disks. 02.09.15 0:00, Marcus Kool пишет: On 09/01

Re: [squid-users] HTTPS URL Rewrite

2015-09-01 Thread Marcus Kool
When a browser requests https://www.example.com/index.html, Squid with ssl-bump sends two requests to the URL rewriter: 1. CONNECT www.example.com:443 2. GET https://www.example.com/index.html The URL rewriter must _not_ block the first and send an alternative URL for the second. Caveat:

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
victims, like the few gifs that actually have a different image depending on the parameter. 02.09.15 0:16, Marcus Kool пишет: > > On 09/01/2015 03:08 PM, Yuri Voinov wrote: >> > Better to write store-id rule which cut off parameters and store gif. > > Something li

Re: [squid-users] Squid 3.5.7, cache_swap_high, bug or not bug ?

2015-08-29 Thread Marcus Kool
On 08/28/2015 08:53 PM, FredT wrote: Hi Amos, We have applied the patch with the client on the squid in prod a coule of hours ago... We can see now a real aggressive objects cleaning I can confirm a 200 obj/sec is a minimal number with huge traffic, you could fix the value a bit higher

Re: [squid-users] ssl_bump updates coming in 3.5.8

2015-08-21 Thread Marcus Kool
I do not want to spoil things, but did you already read my latest addition to bug 4303 ? Marcus On 08/21/2015 04:28 AM, Amos Jeffries wrote: Hi all, Christos has managed (we think) to resolve a fairly major design issue that has been plaguing the 3.5 series peek-and-splice feature so far.

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-25 Thread Marcus Kool
. If memory is plentyful, just make sure that the OS has a large file system cache. So reduce mem_cahce of Squid a little and tune the OS with vm.swappiness=10 in /etc/sysctl.conf Best regards Marcus Have a nice weekend! Regards, Jens Gesendet: Freitag, 24. Juli 2015 um 19:01 Uhr Von: Marcus Kool

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-24 Thread Marcus Kool
: Freitag, 24. Juli 2015 um 14:33 Uhr Von: Marcus Kool marcus.k...@urlfilterdb.com An: Jens Offenbach wolle5...@gmx.de, squid-users@lists.squid-cache.org Betreff: Re: [squid-users] Squid3: 100 % CPU load during object caching On 07/24/2015 03:25 AM, Jens Offenbach wrote: I have made a quick test

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-23 Thread Marcus Kool
I am not sure if it is relevant, maybe it is: I am developing an ICAP daemon and after the ICAP server sends a 100 continue Squid sends the object to the ICAP server in small chunks of varying sizes: 4095, 5813, 1448, 4344, 1448, 1448, 2896, etc. Note that the interval of receiving the chunks is

Re: [squid-users] Squid 3.5.5 ssl_bump and ufdbGuard

2015-07-20 Thread Marcus Kool
First an introduction in blocking HTTPS: HTTPS is a protocol that is designed to be non-interceptable, and if it is intercepted, the browser will notify the user about this interception. This is very different from HTTP which can easily be intercepted and the interceptor can redirect a browser

Re: [squid-users] AUFS vs. DISKS

2015-07-15 Thread Marcus Kool
On 07/15/2015 11:59 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Amos, I think, auds queue must be buffered more better and smoother. On some OS (I've tested) peak loads leads performance degradation. Periodically. That is why I'm not using aufs. This makes

Re: [squid-users] AUFS vs. DISKS

2015-07-15 Thread Marcus Kool
On 07/15/2015 11:39 AM, Amos Jeffries wrote: On 16/07/2015 1:51 a.m., Stakres wrote: Hi Fred, tests from my side: DISKD with TCP_HIT objects: 564KB/s with wget, the same url you have tested. AUFS with TCP_HITS objects: 47.8M/s, same wget, same squid, same url, same all. Wget with AUFS:

Re: [squid-users] Squid and ufdbGuard, display blocked URL on client browser address bar

2015-07-10 Thread Marcus Kool
On 07/10/2015 12:54 AM, Amos Jeffries wrote: On 10/07/2015 9:51 a.m., David Touzeau wrote: Hi ikna This can be done, but you need to forget the ufdbgclient and create yourself a new one that is able to connect to the ufdbguard server in order to get ufdbguard results. In this case, you have

Re: [squid-users] acl for redirect

2015-07-01 Thread Marcus Kool
, not an office, so changing from a proxy to a DNS server is not an option, since we would also be required to change all several thousand of our customers DNS settings. On 6/30/2015 17:30 PM, Marcus Kool wrote: I suggest to read this: https://support.google.com/websearch/answer/186669 and look at option

Re: [squid-users] acl for redirect

2015-06-30 Thread Marcus Kool
I suggest to read this: https://support.google.com/websearch/answer/186669 and look at option 3 of section 'Keep SafeSearch turned on for your network' Marcus On 06/30/2015 05:48 PM, Mike wrote: Scratch that (my previous email to this list), google disabled their insecure sites when used as

Re: [squid-users] Memory usage question

2015-06-19 Thread Marcus Kool
What is the physical memory size ?? You might want to read the faq on memory: http://wiki.squid-cache.org/SquidFaq/SquidMemory Marcus On 06/19/2015 07:19 AM, Alex Samad wrote: Hi I recently push my squid VM memory up to 65G i pushed up squid usage (i thought) to 40G squid.conf cache_mem

Re: [squid-users] squidGuard configuration test - echo test

2015-06-08 Thread Marcus Kool
Helmut, you can download ufdbGuard here: https://www.urlfilterdb.com/downloads/software_doc.html and here: http://sourceforge.net/projects/ufdbguard/ ufdbGuard is just like Squid free Open Source Software. The trial license on www.urlfilterdb.com is about the URL database. Best regards,

Re: [squid-users] squidGuard configuration test - echo test

2015-06-07 Thread Marcus Kool
The URL director interface was changed with Squid 3.4, see also http://wiki.squid-cache.org/Features/Redirectors The latest version of squidguard is 1.5 beta from 2010 and squidGuard does not support the new interface of Squid. ufdbGuard is also a URL redirector and since it has regular

Re: [squid-users] Best solution for content filtering using squid?

2015-05-07 Thread Marcus Kool
Hi, What is it that makes you want to go to a better solution ? did you look at ufdbGuard? Marcus PS: Beware! I am biased since I wrote ufdbGuard. On 05/07/2015 03:52 PM, Bob Cochran wrote: Hi, What is the best solution with squid for content filtering using lists of domains that should

Re: [squid-users] HTTPS Filtering and SSL-Bump

2015-04-23 Thread Marcus Kool
On 04/23/2015 05:52 PM, Jonathan Chretien wrote: Hi all. I'm trying to implement the filtering of https content for a particular url. The only thing that I'm trying to do it's to unlock corporate video on the Youtube website. I do not want to unlock everything on Youtube but only our

Re: [squid-users] Integrate Squid Kerberos auth and Squidguard ldapusersearch into AD

2015-04-15 Thread Marcus Kool
On 04/15/2015 11:38 AM, tchristin wrote: Hi all, I'm having trouble with Squid Kerberos auth and the Squidguard ldapusersearch that I use to apply ACLs by Active Directory groups membership. The problem is : - Squid and Squidguard see my user as : 'user@domain.local' so the '%s' variable of

Re: [squid-users] Refresh ACL list only

2015-03-17 Thread Marcus Kool
On 03/17/2015 04:32 PM, Brendan Kearney wrote: On Tue, 2015-03-17 at 16:13 -0300, Marcus Kool wrote: it has a configuration option to respond with 'allow all' during a reconfiguration. a Fail-Open policy can be a security gap, and should be considered carefully before implementing

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-17 Thread Marcus Kool
On 02/17/2015 11:30 AM, Yuri Voinov wrote: Also, gents. ufdbGuard is cool, but: - Where is good documentation? I found only one connon PDF. No performance recommendations, no administrator's guide - this good piece of software not so trivial as squidGuard, i.e., I don't know, how to

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-17 Thread Marcus Kool
On 02/16/2015 11:43 PM, Amos Jeffries wrote: PS. Marcus, perhapse you should go on search around to find distro maintainers who are publishing SG and convince them to replace the defaults with ufdbguard. I have to do that periodically to clear up old Squid versions being forced on users. It

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-12 Thread Marcus Kool
Yuri, I suggest to consider using ufdbGuard instead of squidGuard. Besides being faster is has a different structure: the redirector that squid starts is a small lightweight process that forwards requests to ufdbguardd, a multithreaded daemon which has the URL database in memory. The database

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-12 Thread Marcus Kool
redirector to commercial one is not an option. ufdbGuard is not a commercial redirector, but is free and works with any free database or your own database/blacklist. It has an additional option to use a commercial database. 13.02.15 2:06, Marcus Kool пишет: Yuri, I suggest to consider using

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2015-02-04 Thread Marcus Kool
On 02/04/2015 04:24 AM, Omid Kosari wrote: The only reason for extend is more capacity . Currently there is no problem with current setup except capacity . I can replace each SSD with new 500GB which doubles the capacity and it is not enough . and old SSDs will be unusable . So i prefer a long

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2015-02-03 Thread Marcus Kool
Hi Omid, The I/O requirements can be estimated well if you tell more about the environment. If you know the number of requests/second that Squid prcoesses you can add a percentage to increase performance and calculate the desired I/Os per second (IOPS). When you have the desired IOPS, you can

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/24/2015 11:24 PM, Amos Jeffries wrote: On 25/01/2015 9:39 a.m., Marcus Kool wrote: On 01/24/2015 10:15 AM, Amos Jeffries wrote: On 22/01/2015 10:11 a.m., Marcus Kool wrote: I am using Squid 3.4.9 and have an issue with tcp_outgoing_address. The Squid server is connceted

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/25/2015 01:12 PM, Amos Jeffries wrote: On 25/01/2015 11:43 p.m., Marcus Kool wrote: On 01/24/2015 11:24 PM, Amos Jeffries wrote: On 25/01/2015 9:39 a.m., Marcus Kool wrote: On 01/24/2015 10:15 AM, Amos Jeffries wrote: On 22/01/2015 10:11 a.m., Marcus Kool wrote: I am using Squid

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/25/2015 02:33 PM, Amos Jeffries wrote: On 26/01/2015 4:59 a.m., Marcus Kool wrote: The debug trace starts with: Xaction.cc(133) openConnection: *Adaptation::Icap::OptXact* opens connection to 10.10.0.6:1344 and then comm.cc(549) comm_openex: comm_openex: Attempt open socket

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common,

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
+ tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
team but there is currently no sponsor to implement a new protocol to filter non-HTTP data in Squid. Marcus On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool marcus.k...@urlfilterdb.com mailto:marcus.k...@urlfilterdb.com wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote: -BEGIN

Re: [squid-users] WARNING: there are more than 100 regular expressions

2014-11-27 Thread Marcus Kool
blocking facebook and twitter can be done with ACLs based on dstdomain. they are much faster than REs. Marcus On 11/27/2014 10:01 AM, navari.lore...@gmail.com wrote: ok i don't intend to use REs for blacklisting but only for blocking some sites like facebook twitter... In the other file i have

Re: [squid-users] High CPU-Usage with squid 3.4.9 (and/or 3.4.4)

2014-11-10 Thread Marcus Kool
during our last tests (with 3.4.x) we also tried the worker option. it does not matter if workers are enabled or not. with more workers the cpu rise seems to be somewhat slower. so it is not connected to (smp)workers. it is the external auth helper - although the squid process and not the

Re: [squid-users] Best way to deny access to URLs in Squid 3.3.x?

2014-10-14 Thread Marcus Kool
With every set of requirements, there is an other best way. To selectively block websites and also block SSH tunnels, VPNs, proxies and remote software (some of which are detected on the fly) you can also use ufdbGuard. Your mileage varies with which URL database you use. Marcus On

Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged

2014-07-25 Thread Marcus Kool
: Marcus Kool [mailto:marcus.k...@urlfilterdb.com] Sent: 25 July 2014 00:37 To: RYAN Justin Subject: Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged Juz, The systems seems to have a very small config. 32 MB

Re: [squid-users] Web/URL categorisation list

2014-07-25 Thread Marcus Kool
Hi Alan, On http://www.squid-cache.org/Misc/redirectors.html you can find a list of URL redirectors. ufdbGuard is a free URL redirector that supports free databases and a commercial database from www.urlfilterdb.com Marcus On 07/25/2014 08:33 AM, Alan Dawson wrote: Hi, Apologies if this is

Re: [squid-users] FW: Problem with server IO resource, need to reduce logging level by excluding specific sites from being logged

2014-07-24 Thread Marcus Kool
Juz, It helps if you describe the system in more detail. What is the configuration of Squid (squid.conf without the comments) and how are the data store file systems spread over the disks ? For immediate results, you can reduce the disk cache or even temporarily disable the disk cache. Marcus

Re: [squid-users] memory_cache_shared no support for atomic operations

2014-06-10 Thread Marcus Kool
On 06/09/2014 07:10 PM, Eliezer Croitoru wrote: On 06/10/2014 12:43 AM, Cassiano Martin wrote: Yes its 32 bit custom built OS As far as I can remember the shared memory needed 64bit OS and HW. I am not 100% sure yet. Eliezer I am not sure but vaguely recall that it works on 32bit when the

Re: [squid-users] configuring Eliezer RPMs for CentOS 6 for SMP

2014-05-17 Thread Marcus Kool
On 05/16/2014 06:47 PM, Fernando Lozano wrote: Hi, I don't quite agree with you. Let me expose my views so each member of the list can weight pros and cons: Not answering this thread, but would like to ask some related points for anyone who may be listening in: 1. RPMs. For practically

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-07 Thread Marcus Kool
On 05/07/2014 06:44 AM, Pawel Mojski wrote: W dniu 2014-05-07 04:52, Jay Jimenez pisze: Hi Marcus and Amos, [...] I'm wondering if there's someone who successfully allowed Skype to fake CONNECT to squid (I'm referring to interception not explicit proxying). I cannot fully implement https

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-07 Thread Marcus Kool
On 05/07/2014 10:55 AM, Pawel Mojski wrote: W dniu 2014-05-07 15:40, Marcus Kool pisze: [...] certificate chain: Certificate chain 0 s:/CN=*.gateway.messenger.live.com i:/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=MSIT Machine Auth CA 2 1 s:/DC=com/DC=microsoft/DC=corp/DC=redmond

Re: [squid-users] Skype SSL is incompatible with OpenSSL

2014-05-02 Thread Marcus Kool
On 05/02/2014 08:21 AM, Jay Jimenez wrote: Hi Amos, Thank you for the response. Any advice of how would I know exactly what SSL/TLS version skype is using and how do I enable those versions to my squid box? It has been a while since I investigated Skype but my findings at that time were

Re: [squid-users] squid 3.4.4 and squidGuard 1.4 redirection 404 error

2014-04-24 Thread Marcus Kool
Or you switch to ufdbGuard which has active support, a configuration option to specify which version of Squid is used, regular updates and is 3x faster. ufdbGuard is free Open Source Software and can be downloaded from sourceforge.net and www.urlfilterdb.com Marcus On 04/24/2014 08:32 AM,

Re: [squid-users] need help with ubuntu upgrade procedure

2014-03-25 Thread Marcus Kool
One way of doing this is to find the ubuntu spec file for the ubuntu package for Squid and use the spec file to build a new squid 3.4.x package. and then install the new package. This way all files locations will remain the same and you can also use the package manager to do an easy downgrade

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-23 Thread Marcus Kool
something about them. What are my options? I do want to lean more about these but I am not sure what to look where to look and how to look. I am looking for more directions about the subject since it's important and not only to me. Thanks, Eliezer On 22/01/14 17:06, Marcus Kool wrote: For the NAS

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-22 Thread Marcus Kool
, Marcus Kool wrote: The raw transfer speed of a disk is only interesting when an application does very large sequential I/Os and squid does not do that. Squid writes a lot to disk and reads relatively little and since the average object size is often around 13 KB, this is also the average I/O size

Re: [squid-users] Re: Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-22 Thread Marcus Kool
On 01/22/2014 03:06 PM, babajaga wrote: IOs have a variable size and for writing an object to a file with the aufs store, the OS write meta data to the file system log, updates the inode table and writes the data to a new file. So for aufs for one logical 'write object to disk' there are 3

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2014-01-19 Thread Marcus Kool
On 01/19/2014 04:42 AM, Eliezer Croitoru wrote: While working here and there I have seen that ZFS is a very robust FS. I will not compare it to any others because there is no need for that. OK so zfs, ext3, ext4 and others are FS which sits on SPINNING disks or flash drives. The SATA and SAS

Re: [squid-users] squid 3.4.1 UPGRADE WARNING: URL rewriter reponded with garbage '

2013-12-17 Thread Marcus Kool
On 12/16/2013 12:48 PM, jeffrey j donovan wrote: On Dec 15, 2013, at 10:43 PM, Eliezer Croitoru elie...@ngtech.co.il wrote: Please refer to: http://bugs.squid-cache.org/show_bug.cgi?id=3978 Which is a *bug* in squidguard due to change\upgrade of squid helpers interface. Eliezer

[squid-users] ufdbGuard 1.31-9 is released to support Squid 3.4.1

2013-12-16 Thread Marcus Kool
ufdbGuard, the URL redirector for Squid, has a new patch release to support the new URL rewriter protocol of Squid 3.4.1. This patch release introduces the keyword squid-version to support all versions of Squid. ufdbGuard 1.31-9 can be downloaded from http://sourceforge.net and

Re: [squid-users] Re: SquidGuard not filtering

2013-12-04 Thread Marcus Kool
The quick and easy solution is to use ufdbGuard. ufdbGuard works like squidGuard but does not have issues like bad performance nor has it lack of support and maintenance. So if you want a free alternative for squidGuard, look at ufdbGuard. ufdbGuard can be downloaded from www.sourceforge.net or

Re: [squid-users] Squid and Squidguard using high disk IO

2013-11-09 Thread Marcus Kool
On Sat, Nov 09, 2013 at 11:16:12PM +0100, Loïc BLOT wrote: Hello Kaya, first, don't forget to look at sysctl kern.maxfiles values. Also improve daemon FD values in login.conf for squid. Don't forget each connection is a FD (1 connection for the client, 1 for the transaction to remote site,

Re: [squid-users] Http works HTTPS gives cert errors. No errors in logs.

2013-10-18 Thread Marcus Kool
I think Blocking HTTPS-based sites needs to be added to the FAQ: Blocking HTTP is easy because the HTTP protocol has well-defined response codes to do this. HTTPS actually is SSL-wrapped HTTP and SSL does not allow any kind of interference, redirection or manipulation and cannot be blocked like

Re: [squid-users] Squidguard, redirect and https

2013-10-17 Thread Marcus Kool
The problem is not Squid nor HTTPS. The problem is that the HTTP protocol has a standard that allows redirection and the HTTPS protocol does not. The HTTPS protocol was designed to be secure and does not allow any type of interference. So, all filtering technologies have the same issue: how to

Re: [squid-users] content filtering question

2013-10-15 Thread Marcus Kool
On 10/15/2013 09:51 AM, Marko Cupać wrote: I am advancing into replacement of NTLM/dansguardian with kerberos/squid and icap and mapped AD groups with help of LDAP authorization, but there are a few things for which I haven't find solution so far: 1. More informative error messages for users

Re: [squid-users] content filtering question

2013-10-15 Thread Marcus Kool
On 10/15/2013 10:42 AM, Marko Cupać wrote: On Tue, 15 Oct 2013 10:26:59 -0300 Marcus Kool marcus.k...@urlfilterdb.com wrote: I suggest to look at ufdbGuard. It is a URL redirector for Squid and Thank you for the tip, I would like to try it. Is it possible to install it on FreeBSD? yes

Re: [squid-users] Removing overlapping subdomains from blacklists

2013-08-21 Thread Marcus Kool
On Wed, Aug 21, 2013 at 05:27:55PM +0100, Andrew Wood wrote: Hi Can someone please help me work out an algorithm to remove overlapping subdomains from a blackclist such as shallalist to prevent errors such as: ERROR: 'interracialcandy.tumblr.com' is a subdomain of '.tumblr.com'

Re: [squid-users] Squid Memory and Page Faults

2013-07-24 Thread Marcus Kool
On Thu, Jul 25, 2013 at 06:58:56AM +1200, Amos Jeffries wrote: On 25/07/2013 1:05 a.m., Golden Shadow wrote: Hi there! My squid is installed on a server with 192 GB of RAM. I have the following directives in squid.conf: cache_mem 143360 MB maximum_object_size_in_memory 300 KB

Re: [squid-users] Squid Memory and Page Faults

2013-07-24 Thread Marcus Kool
On Wed, Jul 24, 2013 at 03:01:20PM -0700, Golden Shadow wrote: Thanks Amos, Eliezer and Markus for your replies! @Eliezer: The server has 2 X 2.7 GHz CPUs, each with 12 cores. Squid version is 3.3.7 compiled from source and I'm running only one squid worker. @Marcus: What is maximum

[squid-users] ufdbGuard, the URL filter for Squid, has a new release

2013-06-27 Thread Marcus Kool
ufdbGuard v1.31 has been released on June 27, 2013. ufdbGuard is a free URL filter for Squid and can be used with your own, a free URL database or a commercial URL database. ufdbGuard has many other features which you can read about on http://www.urlfilterdb.com The main new features of

Re: [squid-users] parent request order

2013-06-24 Thread Marcus Kool
On 06/24/2013 06:01 AM, T Ls wrote: Am 19.6.2013 16:13, schrieb Marcus Kool: On Wed, Jun 19, 2013 at 09:27:54AM -0300, Marcus Kool wrote: On 06/19/2013 09:02 AM, T Ls wrote: ... What I want to do is: a) switch to alternative parents automatically b) in case of P_1-failure: distribute N_1

Re: [squid-users] parent request order

2013-06-24 Thread Marcus Kool
On 06/24/2013 12:44 PM, T Ls wrote: Am 24.06.2013 13:08, schrieb Marcus Kool: On 06/24/2013 06:01 AM, T Ls wrote: Am 19.6.2013 16:13, schrieb Marcus Kool: On Wed, Jun 19, 2013 at 09:27:54AM -0300, Marcus Kool wrote: On 06/19/2013 09:02 AM, T Ls wrote: ... What means not available

Re: [squid-users] Squid performance profiling

2013-06-20 Thread Marcus Kool
On 06/20/2013 06:51 AM, Amos Jeffries wrote: If anyone is interested with very detailed benchmarks, then I can provide them. Yes please :-) PS. could you CC the squid-dev mailing list as well with the details. The more developer eyes we can get on this data the better. Although please

<    1   2   3   4   >