Re: [clamav-users] [External] Re: Scan very slow

2019-04-14 Thread Paul Kosinski via clamav-users
Regexes can be slow or even extremely slow to apply, depending on the implementation. Backtracking is the worst, perhaps taking exponential time, but often is cut off by artificial limits. Does ClamAV perchance precompute Deterministic Finite Automata for the regexes? These run fast, but take

Re: [clamav-users] [External] Re: Scan very slow

2019-04-11 Thread Paul Kosinski via clamav-users
Does clamd use multi-threading for the various "engines" within a single scan, or only to handle multiple requests from different sources? On Tue, 9 Apr 2019 21:29:43 + "Micah Snyder \(micasnyd\) via clamav-users" wrote: > Maarten, > > Your test results are pretty great. I really like

Re: [clamav-users] Database updated over unencrypted connection?

2019-03-20 Thread Paul Kosinski via clamav-users
t gotten to this one yet. > This debate, while interesting is essentially pointless. We’re going > to do it. > > Sent from my  iPhone > > > On Mar 17, 2019, at 21:25, Paul Kosinski via clamav-users > > wrote: > > > > Looking at the PiperMail

Re: [clamav-users] Database updated over unencrypted connection?

2019-03-17 Thread Paul Kosinski via clamav-users
Looking at the PiperMail thread about how ClamAV verifies CVD signatures, I see two things that concern me. First, it says it uses "an implementation of RSA inspired by http://www.erikyyy.de/yyyRSA/;. How well has this implementation been vetted? I'm not a crypto expert (by any means), but people

Re: [clamav-users] Problem with freshclam updating daily-25380.cdiff

2019-03-06 Thread Paul Kosinski via clamav-users
I spoke too soon! Although 0.100.2 didn't hang, it did have to download 25380 several times -- while claiming success each time! On Wed, 6 Mar 2019 15:54:04 -0500 Paul Kosinski via clamav-users wrote: > For once (?) we're not having any problem with this update. Maybe it's > because

Re: [clamav-users] Problem with freshclam updating daily-25380.cdiff

2019-03-06 Thread Paul Kosinski via clamav-users
For once (?) we're not having any problem with this update. Maybe it's because we're still running 0.100.2? On Wed, 6 Mar 2019 14:05:30 + "Micah Snyder \(micasnyd\) via clamav-users" wrote: > I also am seeing the same thing. > Killing freshclam an starting it again reproduces the process

Re: [clamav-users] ClamAV 0.101.0 / HAVP

2019-02-04 Thread Paul Kosinski
Micah, This is great news! I will be trying it out soon. HAVP's latest HAVP changelog shows various people already making contributions, so I wonder how that would play with GitHub. Paul Kosinski P.S. HAVP already seems to have a presence on SourceForge, but with no code -- just old comments

Re: [clamav-users] Input Stream Scanning for very large files

2019-01-25 Thread Paul Kosinski
"If you think there's a problem, why not deal with it before it gets into your archives?" Because maybe somebody else created the archive; and I believe there have been cases where the archive itself exploited a flaw in the code used to expand the archive. On Fri, 25 Jan 2019 19:38:27 +

Re: [clamav-users] Input Stream Scanning for very large files

2019-01-25 Thread Paul Kosinski
I understand that it's impractical for ClamAV to scan exceedingly large files, as it could fill up RAM and/or page forever. But the current 4GB hard limit is overly restrictive, especially since 32-bit addresses and numbers are ancient history in current OSes. In particular, scanning big archives

Re: [clamav-users] Problem with /usr/share/clamav/freshclam-sleep

2018-12-31 Thread Paul Kosinski
Perhaps the distro people (RH/Centos) tried to compensate for the missing headers -- so other software could use libclamav etc. -- and introduced a bug? On Mon, 31 Dec 2018 14:49:51 + "Micah Snyder (micasnyd)" wrote: > I don't think this would be related. The missing headers issue would >

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-22 Thread Paul Kosinski
a check once > an hour is probably fine. > > Sent from my  iPhone > > > On Dec 20, 2018, at 09:55, Paul Kosinski > > wrote: > > > > Only DNS TXT queries are done 3-5 times per hour. Freshclam itself > > is only run whenever that reports that there i

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-20 Thread Paul Kosinski
When talking about averages, I agree. But what I am worried about is the "worst case" malicious payload: for example, a brand new and particularly effective piece of ransomware. It's like car, life or medical insurance. The probability of needing it is low, but when you do, you don't want your

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-20 Thread Paul Kosinski
t; > Replace XY with your country code. If you don’t have that option, > > then you must stick with 1 check per hour. > > -Al- > > On Wed, Dec 19, 2018 at 12:26 PM, Paul Kosinski wrote: > > They all do DNS TXT queries 3-5 times per hour... >

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-19 Thread Paul Kosinski
800 Dennis Peterson wrote: > The TTL of the TXT record is 30 minutes so unless you are directly > polling one of the clamav.net dns servers you are going to get what > ever is in your local NSCD cache. > > dp > > On 12/19/18 12:26 PM, Paul Kosinski wrote: > > > > s

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-19 Thread Paul Kosinski
Yeah, I know that the CDIFFs will/may be cached, but it shouldn't matter. The file daily-25221.cdiff has the same contents no matter when you download it via freshclam or whatever (assuming its contents hasn't been munged by "HTTP-Transform"). But daily.cvd changes over time, as it should. Thus

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-19 Thread Paul Kosinski
quot;Joel Esler (jesler)" wrote: > Inline: > > > On Dec 15, 2018, at 6:23 PM, Paul Kosinski > > wrote: > > > > I don't know if flushing the daily.cvd cache would be adequate, > > since there are probably some downstream caches that wouldn't > > fol

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-15 Thread Paul Kosinski
I don't know if flushing the daily.cvd cache would be adequate, since there are probably some downstream caches that wouldn't follow suit. Pointing *everyone* directly at Cloudflare might be expensive, if that meant millions (or even thousands) of new clients. How does Cloudflare charge Talos

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-15 Thread Paul Kosinski
Indeed, Scripted Update via cdiffs is far more efficient until one has *lots* of machines running ClamAV on one's LAN. This tradeoff should be (and have been) documented. Better yet, the current Local Mirror mechanism should be either fixed to support cld files (if it doesn't already) or removed

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-15 Thread Paul Kosinski
g, that is by design because each > cdiff file brings the local cld file to the cdiff version, and > because it can't be known how many cdiffs have been created between > user updates, they are retained for a period of time and freshclam > applies them in order until the final cdiff m

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-15 Thread Paul Kosinski
Our Comcast account in in MA and is not a business account (which I presume would cost more). My view is that Comcast tech support is on the level of "try restarting your modem" or "try restarting Windows", so I doubt asking about transparent caching would get very far. I don't think it's

[clamav-users] No good deed goes unpunished, or, why CVD files don't work

2018-12-14 Thread Paul Kosinski
The Good Deed When we started using ClamAV, we wanted to distribute the database to the several machines on our LAN in order to reduce the load on the volunteer servers and minimize the load on our old DSL (now gone). The best way to do this, it seemed, was to set up a trivial HTTP server to

Re: [clamav-users] ClamAV installation is OUTDATED! as reported by freshclam utility on CentOS Linux release 7.6.1810 (Core)

2018-12-13 Thread Paul Kosinski
Yeah, I tried to build the latest (i.e. rather old) HAVP against 0.101.0 and it failed due to missing cltypes.h. I haven't had time to look into this -- rather expected -- problem. On Thu, 13 Dec 2018 02:54:08 -0500 Scott Kitterman wrote: > A larger issue in this case is that 0.100.0, as

Re: [clamav-users] Question about LLVM...

2018-12-12 Thread Paul Kosinski
I've always been leery of executable code that gets downloaded "behind the scenes" and then executed for whatever purpose. In the "old days", people were warned against downloading random software and then executing it. How that's become at least half of what we do on a daily basis -- in our

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

2018-12-12 Thread Paul Kosinski
used to use a proxy when many systems were co-located > and it was very effective and was also being used for other purposes. > Life is much simpler now that I'm retired. > > dp > > On 12/11/18 11:45 AM, Paul Kosinski wrote: > > Ever since we set up a local mi

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

2018-12-11 Thread Paul Kosinski
so ignored cdiff files that may > have been available in a version that matched the TXT record. The > purpose of the cdiff files is to cut down on bandwidth. > > dp > > On 12/10/18 6:34 PM, Paul Kosinski wrote: > > We ARE using freshclam to perform the actual update. And

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

2018-12-10 Thread Paul Kosinski
We ARE using freshclam to perform the actual update. And always have been! We've only been using curl (not wget, if that matters) to pull the first few bytes of the cvd to see if its version number matches what the DNS TXT query said. We do this because, after the conversion to Cloudflare, we

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

2018-12-08 Thread Paul Kosinski
t; So my guess, is probably not anycast, but a caching DNS server that > is still giving older records. > > Sincerely, > > Eric Tykwinski > TrueNet, Inc. > P: 610-429-8300 > > > On Dec 7, 2018, at 6:20 PM, Paul Kosinski > > wrote: > > > > As some o

[clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

2018-12-07 Thread Paul Kosinski
As some of you may be aware, ever since ClamAV began using Cloudflare, we have seen many occasions when files like daily.cvd were not available to our LAN until well after the DNS TXT record implied they should be. However, we discovered that these same files *are* available to our Web/email

Re: [clamav-users] is clamav.securiteinfo.com no more?

2018-12-04 Thread Paul Kosinski
I agree. I don't like undecipherable short URLs either: you can't tell what they lead to. (And a preview feature just slows you down.) On Tue, 4 Dec 2018 21:38:14 -0800 Al Varnell wrote: > Not official, but it's a pretty standard response from those of us in > the computer security business

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-26 Thread Paul Kosinski
I believe that the delays we have been observing are due to some problem with the Boston Cloudflare servers, or, perhaps, Comcast has a "transparent" caching proxy which is causing us trouble. I recently installed the same build and configuration of ClamAV 0.100.2 on our Web server, a virtual

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-23 Thread Paul Kosinski
rk over IPv4, I have too many other things to do.) P.S. I think we've been using ClamAV since 0.86.2, back in July 2005 (how time flies), and I've generally been very happy with it. On Fri, 23 Nov 2018 18:32:00 + (GMT) "G.W. Haywood" wrote: > Hi there, > > On Thu, 22

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-23 Thread Paul Kosinski
midt" wrote: > On 23/11/2018 22:45, Gene Heskett wrote: > > On Friday 23 November 2018 03:43:40 Dennis Peterson wrote: > > > >> On 11/22/18 8:51 PM, Paul Kosinski wrote: > >>> I wonder how many users of ClamAV actually log their freshclam >

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-22 Thread Paul Kosinski
I was just looking at freshclam.conf.sample in 0.101.2, and it looks like *all* logging is disabled by default (back to 0.98.6, at least). I wonder how many users of ClamAV actually log their freshclam updates. Those who don't likely won't notice freshclam temporary failures due to an out-of-sync

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-20 Thread Paul Kosinski
evant. > > -- > Joel Esler > Manager, Communities Division > Cisco Talos Intelligence Group > http://www.talosintelligence.com > > > On Nov 19, 2018, at 9:25 PM, Paul Kosinski > > wrote: > > > > Our Internet-facing ClamAV sits on our gateway/firewall an

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-20 Thread Paul Kosinski
here freshclam tries to get updates from. On Tue, 20 Nov 2018 03:39:12 +0100 Benny Pedersen wrote: > Paul Kosinski skrev den 2018-11-20 03:25: > > > # Use aaa.bbb.ccc.ddd as client address for downloading databases. > > # Useful for multi-homed systems. > > # De

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-19 Thread Paul Kosinski
his regard. On Thu, 15 Nov 2018 19:40:43 + "Joel Esler (jesler)" wrote: > Judging by the 60+TB of traffic we are transferring a day, it's > working for at least 3M+ users. > > > On Nov 15, 2018, at 1:34 PM, Dennis Peterson > > wrote: > > > > On 11/1

Re: [clamav-users] ClamAV® blog: The ClamAV 0.101.0 release candidate is here!

2018-11-19 Thread Paul Kosinski
I have long been using HAVP with ClamAV to scan HTTP traffic (inbound). HAVP uses libclamav directly (rather than e.g., clamd) so it doesn't have an excessive performance impact. (Cf. http://www.havp.org/) Unfortunately, HAVP hasn't seen any development for a bit over 2 years. In the past, simply

Re: [clamav-users] ClamAV mirrors have gotten worse!

2018-11-13 Thread Paul Kosinski
er 2018 at 10:33:08 -- On Tue, 13 Nov 2018 09:49:54 -0800 Dennis Peterson wrote: > On 11/12/18 6:28 PM, Paul Kosinski wrote: > > As some of you may remember, I "solved" the problems of the > > Cloudflare mirrors being out of sync by not r

[clamav-users] ClamAV mirrors have gotten worse!

2018-11-12 Thread Paul Kosinski
As some of you may remember, I "solved" the problems of the Cloudflare mirrors being out of sync by not relying on what version the DNS TXT record reports, but double checking it by retrieving the head of the CVD file via curl. Now that I have replaced our dead (hardware, 32-bit) Web and email

Re: [clamav-users] Latest report on update "delays"

2018-10-23 Thread Paul Kosinski
wrote: > On 23/10/2018 16:17, Paul Kosinski wrote: > > Two observations: First, a smoothly working freshclam mechanism > > shouldn't require workarounds. > Well, yes, but it works smoothly for a very large number of people, > myself included. > > > And I suspect man

Re: [clamav-users] Latest report on update "delays"

2018-10-22 Thread Paul Kosinski
(although I suppose using cdiffs would significantly reduce the useless data transfer). Plus there is useless load on the client machine and its LAN. On Tue, 23 Oct 2018 14:16:58 +1100 "Gary R. Schmidt" wrote: > On 23/10/2018 13:28, Paul Kosinski wrote: > > "I'm convince

Re: [clamav-users] Latest report on update "delays"

2018-10-22 Thread Paul Kosinski
; of infestations and is data that can be used in setting priorities. > How to collect that? How to collect any metrics? So far it is largely > buzz generated by responders and which is largely anecdotal. > > To be honest, many problems would be solved if all outbound mail were

Re: [clamav-users] Latest report on update "delays"

2018-10-20 Thread Paul Kosinski
tools to request > signatures and people that do so should have no expectation of > consistent high reliability, and support requests should go in the > bit bucket. The risk associated with self-service falls on the > operator, not the vendor. > > dp > > On 10/19/18 2:19

Re: [clamav-users] Latest report on update "delays"

2018-10-19 Thread Paul Kosinski
> Sincerely, > > Eric Tykwinski > TrueNet, Inc. > P: 610-429-8300 > > -Original Message- > From: clamav-users [mailto:clamav-users-boun...@lists.clamav.net] On > Behalf Of Paul Kosinski > Sent: Thursday, October 18, 2018 1:23 PM > To: > clamav-users@li

Re: [clamav-users] /bin/mkdir: cannot create directory ‘/run/clamav’: File exists

2018-10-18 Thread Paul Kosinski
A sad situation. Reindl is a knowledgeable person, and he seemed to have become less difficult after having been expelled from another list I subscribe to. I had hoped he had reformed. On Thu, 18 Oct 2018 16:31:36 + "Joel Esler (jesler)" wrote: > After several complaints in this thread and

Re: [clamav-users] Latest report on update "delays"

2018-10-18 Thread Paul Kosinski
ate to say that > the DNS announcement is premature. > > The change to freshclam is an effort to ignore potentially premature > database version numbers listed via DNS. > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > On Oct 15, 2018, at 2

Re: [clamav-users] Latest report on update "delays"

2018-10-15 Thread Paul Kosinski
the > mirror is 1 behind what was advertised. My hope is that this > alleviates the issue. > > Respectfully, > Micah > > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > On Oct 4, 2018, at 4:47 PM, Paul Kosinski > mailto:

[clamav-users] Latest report on update "delays"

2018-10-04 Thread Paul Kosinski
At Joel's suggestion, i have changed our sampling rate looking for ClamAV cvd updates from 15 minutes down to 1 minute. This gives a more precise measurement of how long it takes for the cvd file(s) to actually become available from Cloudflare after its presence is "advertised" by the CNS TXT

Re: [clamav-users] ClamAV 0.100.2 has been released!

2018-10-04 Thread Paul Kosinski
I hope it won't be in the style of Mozilla's "significant change" to Firefox, which has just about destroyed it (IMHO, anyway). On Thu, 4 Oct 2018 07:00:00 + "Joel Esler (jesler)" wrote: > :) > > We have some thoughts around 1.0. We want it to be a significant > change, not just an

[clamav-users] Latest delay reports

2018-09-30 Thread Paul Kosinski
Here is our latest report of delays between DNS TXT records indicating ClamAV signatures are available and quick curl(s) confirming file(s) are actually available for freshclam download from Cloudflare. (As before, our sampling is done every 15 minutes.) 2018-09-13 13:03:01 No delay

Re: [clamav-users] updates

2018-09-13 Thread Paul Kosinski
ng? > > > On Sep 13, 2018, at 2:16 AM, Paul Kosinski > > wrote: > > > > "What is the interval that you run this?" > > > > Every 15 minutes by cron, specifically: > > > > OCBG='/opt/clamav/bin/getfreshclam' > > > >

Re: [clamav-users] updates

2018-09-13 Thread Paul Kosinski
is the code we use to update ClamAV: 'getfreshclam' is run > > by cron under userid clamav (same as clamd) every so often > > (currently every 15 mins) to determine if there are any relevant -- On Wed, 12 Sep 2018 20:59:45 +0000 "Joel Esler (jesler)" wrote: > What

Re: [clamav-users] updates

2018-09-12 Thread Paul Kosinski
, as in: /opt/clamav -> /opt/clamav.d/clamav.0.100.1 Enjoy! Paul Kosinski On Wed, 12 Sep 2018 15:41:23 + "Joel Esler (jesler)" wrote: > Paul, > > Can you give me some more information on how you do this? How often > is the check ran, etc. > > I am working w

Re: [clamav-users] updates

2018-09-07 Thread Paul Kosinski
Here is our recent CVD delay report showing how long the actual daily.cvd (and sometimes bytcode.cvd) file(s) lag behind the DNS TXT record. We are located near Boston, and the data comes via Comcast cable, but our DNS queries use our old, slow static-IP DSL. I keep it this way because there were

Re: [clamav-users] ClamAV signature update sync errors have gotten worse

2018-08-20 Thread Paul Kosinski
2 PB in the last month, delivering > updates an average of 39% faster. We are seeing excellent results. > > > On Aug 18, 2018, at 1:09 AM, Paul Kosinski > > wrote: > > > > Joel, > > > > Still lots of delays since "2018-08-11 13:18:02 No delay"

Re: [clamav-users] ClamAV signature update sync errors have gotten worse

2018-08-17 Thread Paul Kosinski
t will resolve > the issues. Please keep these coming?! > > Sent from my iPad > > On Aug 11, 2018, at 2:10 PM, Paul Kosinski > mailto:clamav-us...@iment.com>> wrote: > > Here is the latest report for ClamAV virus update mirror delays since > the end of July.

[clamav-users] ClamAV signature update sync errors have gotten worse

2018-08-11 Thread Paul Kosinski
-10 05:48:01 01:00:00 delay 2018-08-10 13:48:02 00:30:00 delay 2018-08-11 00:48:02 03:30:00 delay 2018-08-11 05:33:02 No delay 2018-08-11 13:18:02 No delay On Tue, 31 Jul 2018 13:47:39 -0400 Paul Kosinski wrote: > There are still over 1/3 signature update sync errors with the new >

Re: [clamav-users] After 0.100.1 Update, clamd crashes

2018-07-31 Thread Paul Kosinski
I must say that I agree. To have ClamAV crash on a badly formed signature is as bad (or worse) as having it crash while scanning. Since ClamAV tends to be run with automatic updates to its DB, having a bad signature cause it to crash can result in email blockage or a total lack of AV service

[clamav-users] Still over 1/3 signature update sync errors

2018-07-31 Thread Paul Kosinski
There are still over 1/3 signature update sync errors with the new ClamAV mirrors. You may remember that I previously added code to our ClamAV update protocol to verify that the actually available daily.cvd etc. matched the version number reported by the DNS TXT record. (This is done by using

Re: [clamav-users] Yet another synchronization failure!

2018-07-18 Thread Paul Kosinski
P.S. Maybe I should only try to update those databases that really are available (i.e., use "freshclam --update-db=whatever"). ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users

[clamav-users] Yet another synchronization failure!

2018-07-18 Thread Paul Kosinski
A few days ago, I programmed some pre-tests so as to avoid running freshclam until *both* the DNS TXT record and the first few bytes of daily.cvd (obtained via curl) agree that there is a new version which *actually* is available for download via freshclam. It turns out that doesn't cover all the

[clamav-users] No virus updates for 31 hours?

2018-07-18 Thread Paul Kosinski
Judging by the DNS TXT record, we have seen no virus updates since 01:33 (EDT) yesterday. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-11 Thread Paul Kosinski
More sync delays (which our new curl pretest scheme mitigates). First, a 2 hour 15 minute delay: -- Wednesday 11 July 2018 at 01:03:01 -- /opt/clamav/bin/testclam-external --> EXT D 24741/24742/24741 B 324/324/324 M 58/58/58 # 4

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-10 Thread Paul Kosinski
I looked at a bunch of pages on Cloudflare's site. What they offer is quite impressive -- way beyond "mere" distributed/anycast CDN. On Tue, 10 Jul 2018 22:13:49 -0400 Eric Tykwinski wrote: > They have some documentation on their site: >

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-10 Thread Paul Kosinski
ot; wrote: > Thanks for this feedback everyone. This is extremely useful. > > > > On Jul 10, 2018, at 11:26 AM, Paul Kosinski > > wrote: > > > > Last night our new method of getting cvd updates showed that it was > > *one hour* from the time the D

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-10 Thread Paul Kosinski
Last night our new method of getting cvd updates showed that it was *one hour* from the time the DNS TXT record claimed a new cvd was available to the time when our quick curl said it was really available! In particular at 1:03 AM (EDT), DNS said version 24739 was available, but a curl of the

[clamav-users] PrivateMirror vs HTTPproxy

2018-07-09 Thread Paul Kosinski
For several years now, we've been using HTTPproxy to reduce the load on the public ClamAV servers. We don't use Squid or anything general purpose like that, but rather a simple-minded ClamAV-only server that listens on a private port, only supports the HTTP that freshclam actually uses to get the

Re: [clamav-users] max file size & system damage

2018-07-09 Thread Paul Kosinski
A Linux process that exhausts physical memory may cause problems, depending on how the kernel is configured. Look up "linux oom" with your favorite search engine. On Mon, 9 Jul 2018 21:59:46 + "Rovan, Jim (IMS)" wrote: > Hello, everyone. > > I understand how I can increase the max file

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-09 Thread Paul Kosinski
I have changed the way we use freshclam to mitigate the sync problem with the new Cloudflare mirror regime -- which, by the way, *still* seems to lag what the DNS TXT record reports. What I have done is to introduce a pretesting phase before invoking freshclam. Our new update method operates in

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-05 Thread Paul Kosinski
Mirrors should support a well-defined protocol. Using an ill-defined protocol which only works with a particular tool is not, in my mind, consistent with the spirit of Open Source. I've been perfectly happy (until the recent sync failures, at least) using freshclam, which is Open Source like the

Re: [clamav-users] Is ClamAV available on the hypervisor?

2018-07-05 Thread Paul Kosinski
"* If the question is about using ClamAV to analyze traffic then no, that is not the function of ClamAV. ClamAV analyzes files, not traffic." I use HAVP to scan HTTP traffic, and it uses libclamav and thus ClamAV signatures etc. The future development of HAVP is uncertain,but it still seems to

Re: [clamav-users] Freshclam IPv6 error messages on IPv4-only systems

2018-07-04 Thread Paul Kosinski
I always build ClamAV for our systems, and use the "--disable-ipv6" option (among others) when building. Part of the reason I build locally is so that I always have an old version around in case something goes horribly wrong. Also, I can run more realistic tests on the new build before cutting

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-04 Thread Paul Kosinski
Using DNS TXT records is great when they work, but a bandwidth disaster when they don't. I don't think Cloudflare per se is the problem -- I think having different computers serving the DNS vs the big files is the problem. Back in the old days of ClamAV, they probably were the same computers.

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-03 Thread Paul Kosinski
We used to check once every 90 minutes (16 per day). Plus, we run a local proxy/mirror so the updates can be served to other machines on our LAN without extra load on the ClamAV servers. That was before the new mirroring scheme. Now we're checking several times per hour in the (vain?) hope of

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-03 Thread Paul Kosinski
You are right! Maybe it only rejects browser-ish headers. On Tue, 3 Jul 2018 08:12:47 -0700 Dennis Peterson wrote: > If you run that curl command I provided it will return only the > signature serial number. > > dp > > On 7/3/18 6:59 AM, Paul Kosinski wrote: > >

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-03 Thread Paul Kosinski
The way Linux updates are done in practice is significantly different from ClamAV virus signature updates. With ClamAV, freshclam is automatically run periodically, sees (by some low-cost means) that a new file version is *supposed* to be available and tries to download it. If either it can't,

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-03 Thread Paul Kosinski
Determining what version a *mirror* has is a bit tricky. Looking at the capture of the entire HTTP session with the new mirrors, they seem to require some header magic to be acceptable: Host: db.us.clamav.net User-Agent: ClamAV/0.99.4 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Simply trying

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-02 Thread Paul Kosinski
> mirrors are synched (push) quickly from the repository and the next > tier of mirrors can now update from this block of mirrors rather than > the repository alone, and this will distribute the load and minimize > bandwidth induced lag. NIS works in this fashion. > > Another o

[clamav-users] Proposals for more reliable updates

2018-07-02 Thread Paul Kosinski
Currently, when a daily.cvd is downloaded, its version and other such info is in the first N bytes of the whole file, which is quite big. How about repeating that information in the HTTP response header, so it could be retrieved by an HTTP HEAD command, rather than having to do a massive GET,

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-02 Thread Paul Kosinski
a result of > > the move from ClamAV mirrors to the ClamAV CDN. > > > > Sent from my iPad > > > > -Al- > > > >> On Jul 1, 2018, at 20:38, Dennis Peterson > >> wrote: > >> > >>> On 7/1/18 8:24 PM, Paul Kosinski wrote: > &

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-07-01 Thread Paul Kosinski
ation lookup. Helps us see what > versions people are running out there and what version of ClamAV > people are using. It’s failure shouldn’t stop the update process. > Please give us a debug. > > Sent from my iPhone > > > On Jun 30, 2018, at 19:28, Paul Kosinski > &g

[clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

2018-06-30 Thread Paul Kosinski
We are *still* failing to get ClamAV cvd files updates reliably -- even after deleting mirrors.dat before each attempt! The basic problem seems to be that the query to (e.g.): daily.24710.85.1.0.6810BB8A.ping.clamav.net fails as often as not (e.g.): Querying

Re: [clamav-users] Tweet by ClamAV - Cloudflare

2018-06-27 Thread Paul Kosinski
o it, is because for some reason, there are > a bunch of people in Italy attempting to fetch from the Ireland > mirror. Maybe because of unreliability in that region at some point, > and they left it that way... In any case, everyone is being served > out of their closest POP now

Re: [clamav-users] Tweet by ClamAV - Cloudflare

2018-06-27 Thread Paul Kosinski
Assuming my map reading is correct, it looks like the San Francisco area saw the biggest improvement. Why wouldn't they have had really good service to begin with? On Wed, 27 Jun 2018 14:25:47 + "Joel Esler (jesler)" wrote: > I generally wouldn’t copy a Tweet over to the mailing list, but

Re: [clamav-users] VirusDB Updates Broken?

2018-06-26 Thread Paul Kosinski
Esler (jesler)" wrote: > I just purged db.us’s cache. Can you try? > > Sent from my iPhone > > > On Jun 26, 2018, at 20:24, Paul Kosinski > > wrote: > > > > Joel, > > > > Sorry to have been somewhat cryptic: I assumed the context of the > &

Re: [clamav-users] VirusDB Updates Broken?

2018-06-26 Thread Paul Kosinski
enever]? On Tue, 26 Jun 2018 20:01:09 + "Joel Esler (jesler)" wrote: > Define broken in your context? Doesn't have the file? (Humor me, so > I understand from your parlance) > > > > > On Jun 26, 2018, at 2:59 PM, Pa

Re: [clamav-users] VirusDB Updates Broken?

2018-06-26 Thread Paul Kosinski
Looking into it. > >> > >> Sent from my iPhone > >> > >> > >> > On Jun 24, 2018, at 23:12, Al Varnell wrote: > >> > > >> > Yes, but all but one was empty. > >> > > >> > Sent from my iPad > >> >

Re: [clamav-users] VirusDB Updates Broken?

2018-06-24 Thread Paul Kosinski
I've gotten several daily.cvd updates in that period. They came from several IP addresses associated with http://db.us.clamav.net/. On Sun, 24 Jun 2018 18:08:59 -0700 Al Varnell wrote: > Just wanted to point out that there has only been one signature added > to the VirusDB by daily updates in

Re: [clamav-users] off topic Re: clamav list spf problem

2018-06-24 Thread Paul Kosinski
This reminds me of one of the reasons I dropped commercial AV software in favor of Open Source ClamAV: I decided that I would prefer somewhat less comprehensive AV rather than "full featured" AV that does things you can't control (or sometimes even know about). P.S. We also have internal email

Re: [clamav-users] FW:

2018-04-25 Thread Paul Kosinski
When I tried to specify a limit beyond 4 GB using the "--max-filesize" or "--max-scansize" options, clamscan didn't allow it. Has that been fixed in the new ".100" release? On Wed, 25 Apr 2018 10:53:28 + Richard Tappenden wrote: > Hey guys - can you answer a

Re: [clamav-users] Errors connecting to mirrors

2018-03-28 Thread Paul Kosinski
it's saved that in the mirror.dat file. Clear that file, and it > potentially, could remove those messages. (unless those mirrors are > messed up.) > > > > On 3/28/18, 8:42 PM, "clamav-users on behalf of Paul Kosinski" > <clamav-users-boun...@lists.clamav.net on beh

Re: [clamav-users] Errors connecting to mirrors

2018-03-28 Thread Paul Kosinski
Here is a recent freshclam log that details our local mirror -- we try to save bandwidth! -- downloading from ClamAV servers. It shows lots of errors followed by success. The total elapsed time, on our 100+ Mb/s cable connection, is about 1 minute in spite of the failures.

[clamav-users] Another Open Source anti-malware project

2018-03-23 Thread Paul Kosinski
I just came across this Open Source anti-malware project called "Linux Malware Detect". Anybody know anything about this? https://www.rfxn.com/projects/linux-malware-detect/ ___ clamav-users mailing list clamav-users@lists.clamav.net

Re: [clamav-users] Question about the clamdscan

2018-03-21 Thread Paul Kosinski
A few years ago, when Tripwire was no longer free, I set up a "scan once" environment for ClamAV, identifying files using SHA1 hashing (with a few 'stat' results like inode and timestamp for good measure). I gave up when I realized that even if a file had already been scanned, it might have

Re: [clamav-users] ClamAV performance overhead on RHEL & Solaris

2018-03-17 Thread Paul Kosinski
1. With regard to separating signature sets: Isn't there always a danger that scanning with a reduced signature set misses malware that was not "expected" in a particular context? For example, except in highly restrictive work environments, people tend to visit all sorts of Web sites, any of

Re: [clamav-users] using clamav to detect unwanted data exfiltration programs (e.g., Dropbox)?

2018-02-15 Thread Paul Kosinski
Perhaps it would be more effective simply to block access (in the firewall) to sites like Dropbox. In any case, it might improve "legal security" to add such blocking to the firewall, or perhaps your local DNS forwarding server. P.S. Note that site blocking works even for encrypted connections,

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 beta has been released!

2018-02-05 Thread Paul Kosinski
Any hope for eliminating the 4 GB file size limit soon? It blocks scanning many package files (ZIP, et al) and, of course, video files. On Mon, 5 Feb 2018 23:03:44 + "Joel Esler (jesler)" wrote: > > >

Re: [clamav-users] Daily version 24256

2018-01-30 Thread Paul Kosinski
If anyone still wants 24256, I have made it available at http://iment.com/clamav/daily.cvd.24256 On Mon, 29 Jan 2018 13:24:45 +0100 Carlos García Gómez wrote: > Hi, > > I´m thinking about >

Re: [clamav-users] GPG key where? (was: Re: GPG signature problem with clamav-0.99.2.tar.gz)

2018-01-29 Thread Paul Kosinski
I tend to get keys via GPG's "--recv-key" command, since it often is not clear from the Web site where to get the key. E.g., when "gpg --verify" reports the key is missing, the command below will usually retrieve it (when it is provided, of course, with the right fingerprint in place of

[clamav-users] False positive -- I hope

2018-01-28 Thread Paul Kosinski
Using clamav.0.99.3 to scan the latest Firefox ESR (52.6.0), and using various extra signatures from Sane Security, I get: firefox-52.6.0-esr-32.tar.bz2: Sanesecurity.Foxhole.Zip_Js_Js.UNOFFICIAL FOUND firefox-52.6.0-esr-64.tar.bz2: Sanesecurity.Foxhole.Zip_Js_Js.UNOFFICIAL FOUND I get the

  1   2   >