[clamav-users] An example of why ClamAV should be able to scan disk images (which are typically over 2 GB)

[Paul Kosinski via clamav-users]

CVE-2021-44879

Wenqing Liu reported a NULL pointer dereference in the f2fs
implementation. An attacker able to mount a specially crafted image
^^^
can take advantage of this flaw for denial of service.

>From "Debian Security Advisory DSA-5594-1"
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Question About MaxFileSize / news of upcoming Large Archive Scanner tool

[Paul Kosinski via clamav-users]

Large archive files may be the most obvious case, especially if things like 
disk images and installation images are included, but make sure that large 
multimedia files are also handled.

In today's Internet environment, there are probably far, far more large video 
files floating around than traditional archives. And in some sense multimedia 
"container" files (like MP4, MOV, AVI etc.) are archives of their media streams 
(like H.264/5, AAC, etc.) -- but these archives are, of course, interleaved for 
real-time playback.

I might add that there have been recent reports of malformed (perhaps 
malicious) multimedia files causing crashes or unwanted code execution in 
software such as FFMPEG.


On Mon, 13 Nov 2023 20:32:38 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> In case anyone else is looking into this, I wanted to share some news.
> 
> We have been getting some help to create a tool to recursively unpack (or 
> mount) and scan large archives (greater than 2000MB).
> 
> This effort has progressed to the point where we've started code review and 
> writing documentation. I'm not entirely sure how we will package it for 
> people to use.  I'll share more when we go to open source it. I wanted to 
> share the news now in case anyone else was going to work on it and so they're 
> not as frustrated when it turns out we've done the same.
> 
> I don't have a specific release date in mind.  It likely won't be until early 
> next year.  While we've started code review and testing, the developer that 
> has built the tool for us is now working on adding the allmatch-mode feature 
> support.
> 
> Best regards,
> Micah
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
> 
> 
> From: Andrew C Aitchison 
> Sent: Thursday, June 8, 2023 6:25 PM
> To: Micah Snyder (micasnyd) 
> Cc: ClamAV users ML 
> Subject: Re: [clamav-users] Question About MaxFileSize
> 
> On Thu, 8 Jun 2023, Micah Snyder (micasnyd) wrote:
> 
> > I agree with you.  I suspect the majority of cases today is when
> > people have a large archive of files to scan.
> >
> > I think best case scenario for people with a need to scan files
> > larger than the present internal 2GB limit is that archives larger
> > than 2GB are decompressed and then the files inside are scanned, but
> > without actually scanning the very large outer archive.
> >
> > The way to do this as things work today is to script something
> > around clamscan or clamdscan that if the file is too large, handle
> > some assorted file types:
> >
> >  1.  if file is a tar.gz, un-tar.gz it and then scan the files within.
> >  2.  if file is a zip, un-zip it and then scan the files within.
> >  3.  etc.
> >
> > I think everyone would like if clamav could do this automatically
> > for select archive types. And I think the advantage would be that we
> > would perhaps keep the extracted files in memory, or else at least
> > delete the temp files as we go without extracting all of it to disk
> > before starting to scan.
> >
> > However, it would be far easier to make a shell script or a python
> > script that wraps clamscan/clamdscan and uses native tools like
> > "tar", "unzip", etc.  
> 
> Good idea.
> 
> Simply untarring or unzipping into a pipe does not separate the packed files.
> However at least tar does have an option which allow us to write a one-liner:
> (tar xf ~/viruses.tar --to-command='clamdscan -v - || echo "  found in 
> $TAR_REALNAME\n\n---"' ) |& egrep -i found
> stream: Eicar-Signature FOUND
>found in viruses/EICAR.COM.TAR
> stream: Eicar-Signature FOUND
>found in viruses/eicar.com.txt
> stream: Eicar-Signature FOUND
>found in viruses/URLEICAR.COM.TAR
> stream: Eicar-Signature FOUND
>found in viruses/4DOSBOX/EICAR.COM
> stream: Eicar-Signature FOUND
>found in viruses/EICAR.COM
> 
> The echo is needed to show the name of the file inside the archive.
> 
> This appears not to write the unpacked files to disk.
> 
> --
> Andrew C. Aitchison  Kendal, UK
> and...@aitchison.me.uk
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] first questioon????

[Paul Kosinski via clamav-users]

On Wed, 25 Oct 2023 17:18:46 +0100 (BST)
Andrew C Aitchison via clamav-users  wrote:

> On Sun, 22 Oct 2023, Rahim Fakir via clamav-users wrote:
> 
> > I would like to know if it is possible to have clamav on the desktop and
> > remotely scan the phone.
> > for example: clamscan -r -i remove=yes ipaddress root.of.cellphone  
> 
> For Android it is likely you can use
> https://play.google.com/store/apps/details?id=net.xnano.android.sshserver=en_US=US
> to share you phone as an sftp server, and
>   sshfs
> on the desktop to make the phone appear as a drive on the desktop.
> 
> Once that is working ClamAV could scan the mounted phone drive.
> I have no idea how fast or slow this would be.
> 
> More importantly, I suspect that SSHServer would not have access to all of 
> the files onyour phone, so there would be files you could not scan.
> 
> Please do not use "remove=yes". ClamAV is rarely tested on phones
> so false-positives are more likely than usual.


It may be worse than that.

A few years ago, I had installed a a nice FTP server (WiFi FTP Pro) on my 
wife's Android phone, to make it easy to download photos etc. Then Google 
forcibly installed Storage Access Framework (SAF) on Android: it blocks access 
to all but a few directories (e.g., can't even get at the SD card!), and there 
seems to be no update for this FTP server to coexist with SAF.

P.S. Google claims this was done for security reasons (which it does somewhat 
improve), but I wonder if it isn't also to trap you into their ecosystem 
(Google Drive et all).
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Question About MaxFileSize

[Paul Kosinski via clamav-users]

You are right. But more than that, merely *reading* a file will exercise such 
code. I wonder if anybody has devised a file which exploits such a kernel bug? 
(Shudder.)

After I wrote my objection, I realized that to be even more safe, one should 
scan removable disks at the block level before mounting them. But given the 
capacity these days of even USB thumb drives, this approach is pretty much 
impractical. Beside, what looks like a USB thumb drive might actually act as a 
USB keyboard! (In fact, I think somebody built a prototype.)


On Fri, 09 Jun 2023 18:15:39 -0700
Kenneth Porter  wrote:

> Filesystems are also files, interpreted by kernel-level filesystem drivers. 
> Some filesystems have a compression feature. Scanning ANY file exercises 
> such code.
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Question About MaxFileSize

[Paul Kosinski via clamav-users]

I must say I strongly disagree with the approach of feeding files contained in 
a big archive file one at a time to ClamAV. That's because an archive is 
*itself* a file.

I have on occasion heard of vulnerabilities in some archiving software, where 
the mere act of decompressing and extracting an archive can result in malicious 
code execution due to a bug in the archiving software. After all, such software 
can itself have the all too common lack of bounds checking (etc.) that could be 
exploited by a maliciously malformed archive.

It could also be that lower level archive-like files such as ISOs and disk 
images could, by means of malicious structuring, trigger a total system 
compromise, because it might well involve the kernel. The way an ISO or disk 
image is typically used (on Linux, at least) is to create a "loop" device from 
the file, and then *mount* it as block device -- a clear kernel involvement.

Of course, scanning any file might conceivably trigger a ClamAV bug, and thus a 
compromise, but that is no reason to add another layer of vulnerability to 
things. (But it is a good reason not to run ClamAV as root.)

Paul Kosinski



On Thu, 8 Jun 2023 20:55:25 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> I agree with you.  I suspect the majority of cases today is when people have 
> a large archive of files to scan.
> 
> I think best case scenario for people with a need to scan files larger than 
> the present internal 2GB limit is that archives larger than 2GB are 
> decompressed and then the files inside are scanned, but without actually 
> scanning the very large outer archive.
> 
> The way to do this as things work today is to script something around 
> clamscan or clamdscan that if the file is too large, handle some assorted 
> file types:
> 
>   1.  if file is a tar.gz, un-tar.gz it and then scan the files within.
>   2.  if file is a zip, un-zip it and then scan the files within.
>   3.  etc.
> 
> I think everyone would like if clamav could do this automatically for select 
> archive types. And I think the advantage would be that we would perhaps keep 
> the extracted files in memory, or else at least delete the temp files as we 
> go without extracting all of it to disk before starting to scan.
> 
> However, it would be far easier to make a shell script or a python script 
> that wraps clamscan/clamdscan and uses native tools like "tar", "unzip", etc.
> 
> Regards,
> Micah
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] End of life (EOL) policy change, 0.103 one year extension, 0.105 past end of life

[Paul Kosinski via clamav-users]

On Tue, 16 May 2023 20:32:56 +
"Micah Snyder (micasnyd)"  wrote:

> Hi Paul,
> 
> Unlike Java or C#, Rust does not have any additional runtime library 
> requirement.
> 
> Regards,
> Micah
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.



I'm somewhat surprised Rust doesn't need an extra library. Since one of its 
goals is safety, esp. avoiding the all-too-common memory misuse problems (use 
after free, double free, bad pointers, etc.), I would have guessed that they 
would have created some library functions to encapsulate some of their safety 
paradigms.
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] End of life (EOL) policy change, 0.103 one year extension, 0.105 past end of life

[Paul Kosinski via clamav-users]

Micah,

Great decision!

I was worried about needing Rust on some of our systems. Not only for 
compiling, but doesn't Rust also need its own run time libraries?

I'm still trying to figure out how to move from iptables to nftables, so not 
having also to use Rust "immediately" is a relief. 

(They claim nftables is better, but their automatic translator doesn't handle 
all of the options iptables supported. This is probably because it looks that 
they just arbitrarily dropped some matchers, like 'u32', from the underlying 
engine.)



On Mon, 8 May 2023 17:55:57 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> Read this online at 
> https://blog.clamav.net/2023/05/end-of-life-eol-policy-change-0103-one.htm
> 
> 
> End of life (EOL) policy change
> ClamAV is making a minor change to our EOL 
> policy.
> 
> The original EOL policy stated that Long Term Support (LTS) versions will 
> lose access to signature updates on the same date that we end support for 
> additional patch versions.
> 
> We are changing the policy to allow signature updates for at least one year 
> after we stop supporting the release with patch versions.
> 
> 0.103 support extension
> We are also announcing a one-year extension of support for ClamAV 0.103 LTS.
> 
> We decided to extend the life of the 0.103 LTS release because of the 
> significant changes to the build system in 0.104 and the change in 0.105 
> requiring the Rust programming language toolchain to compile ClamAV.
> 
> The one-year support extension does not apply to future LTS releases.
> 
> ClamAV 0.103.0 was initially released on Sept. 14, 2020. With the additional 
> year of support, and considering the change in the EOL Policy that allows one 
> additional year of access for signature updates, this means that EOL dates 
> for ClamAV 0.103 LTS are as follows:
> 
>   *   Expected End of Life (EOL): Sept. 14, 2024
>   *   Patch versions continue until: Sept. 14, 2024
>   *   Internal signature load testing until: Sept. 14, 2024
>   *   Database downloads allowed until: Sept. 14, 2025
> 
> 0.105 EOL
> Finally, we would like to remind everyone that as per the EOL Policy, the 
> release of ClamAV 1.1 heralds the end of patch versions supporting ClamAV 
> 0.105. There will no more patch versions for ClamAV 0.105.
> 
> ClamAV 0.105 will continue to have access to signature updates for an 
> additional four months after the 1.1 release, which was on May 1, 2023. This 
> means that we may block 0.105 from further updates after Sept. 1, 2023.
> 
> Posted by
> Micah Snyder  at 1:24 
> PM
>  [https://img1.blogblog.com/img/icon18_email.gif]  
> 
> Email 
> ThisBlogThis!Share
>  to 
> TwitterShare
>  to 
> FacebookShare
>  to 
> Pinterest
> Labels: 0.103, 
> 0.105, 
> eol, 
> LTS
> 
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Be wary of emails with attachments targeting clamav-users list members

[Paul Kosinski via clamav-users]

I have just started getting these claiming to be relevant to ClamAV, but I have 
*also* been receiving this sort of thing claiming to be from the Firefox ESR 
list for months now.

I am posting (one of) the HTMLs "about" ClamAV to 
https://www.clamav.net/reports/malware. Should I also post (one of) the Firefox 
phishes? (In fact, I have several of each, but it quickly gets tedious.)



On Wed, 22 Mar 2023 16:48:32 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> All,
> 
> Some users have reported receiving emails that appear to be a reply to a 
> clamav-users mailing list thread but are in fact a phishing attempt have 
> attached malware.
> 
> Most recently, Marc reported receiving an email that appeared to be a reply 
> to an older clamav-users mailing list thread but was in fact a direct email 
> targeting him.  It had this fairly generic phishing text:
> 
> "Would you please look through the last agreement? I have attached some extra 
> details about it."
> 
> The attached file was some small HTML file containing malicious obfuscated 
> javascript.
> 
> This isn't the first time we've heard of this type of phishing using our 
> mailing list archives. Please be careful when you see any sort of attachment, 
> even if it appears to be from this community.
> 
> If you receive this sort of phishing email, please report the attached HTML 
> file to https://www.clamav.net/reports/malware
> 
> Regards,
> Micah
> 
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] The database server doesn't have the latest patch

[Paul Kosinski via clamav-users]

My main point (which wasn't emphasized enough) was that one of the Cloudflare 
"anycast" mirrors (my local one, "BOS"), which host the ClamAV files, was often 
missing the *latest* version of the daily signature file. So I wondered if the 
same kind of Cloudflare problem might be affecting you.

I wasn't suggesting direct download for each ClamAV machine of yours. That's 
just what I had to do to keep up to date, since the much smaller CDIFF files 
were always up to date on my local Cloudflare mirror. (And it still works fine, 
since I have only a few ClamAV machines.)




On Thu, 16 Mar 2023 06:02:45 -0300
Jorge Elissalde via clamav-users  wrote:

> Hi,
> 
> Thank you for sharing your experience.
> In my case I have around 500 machines (and growing) and I cannot allow them
> to directly download updates.
> Everything was working well, until yesterday. A proxy like Squid or similar
> is not an option.
> The truth is I need this working using my own mirror.
> 
> Thank you
> 
> Jorge
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] The database server doesn't have the latest patch

[Paul Kosinski via clamav-users]

A few years ago, when I was attempting local mirroring, I was having a 
recurring problem with my local Cloudflare "anycast" server ("BOS"). I wonder 
if you might be having a similar problem.

I was running a crontab triggered procedure a few times an hour which would do 
a DNS TXT query to see what the latest versions of everything were, and then 
download the CVDs if necessary. These new versions were then put in the local 
mirror. I did this frequent querying -- perhaps followed by downloading -- 
because AV providers occasionally issue "emergency" releases of new signatures.

I found that very frequently the BOS Cloudflare server was many hours behind 
the other Cloudflare servers, and didn't have the latest signature CVD. This 
made the attempted download futile. (Actually, I would download the first N 
bytes of the CVD file to see if it was indeed the version the DNS TXT query 
reported -- before downloading the whole file. This is how I discovered the 
Cloudflare's BOS server was often not up to date.)

I gave up on the local mirror approach when somebody pointed out that I didn't 
have enough local ClamAV instances to actually save bandwidth compared to 
having each instance just use the normal direct approach separately. (I don't 
think that local mirroring supported CDIFFs at the time.)

Now, of course, the CVDs are much bigger, and, unlike then, one has to be very 
careful not to download too much too often, lest one get blocked. (I was 
careful even back then, and never did silly things like downloading identical 
copies of an unchanged CVD 10^N times per day.)




On Wed, 15 Mar 2023 19:22:09 +
newcomer01 via clamav-users  wrote:

> Hi,
> 
> I have a similar problem, i found out, that it might be problems with the ip, 
> but i have no further details.
> I got the same error if i set "--local-address=" for freshclam, as result of 
> this, i have removed this option from my freshclam call again and let clamav 
> do what he want to do.
> 
> 
> kind greetings
> Marc
> 
> Von / From: Clamav User Mailinglist 
> An / To: Newcomer01 
> CC / CC: Jorge Elissalde 
> Gesendet / Sent: Mittwoch, März 15, 2023 um 19:29 (at 07:29 PM) +0100
> Betreff / Subject: [clamav-users] The database server doesn't have the latest 
> patch
> > Hi,
> >
> > I'm using my own mirror for Database update.
> > Configuration in Freshclam correctly points to my server:
> >
> > DatabaseMirror http://myserver.info/clamav
> >
> > The Database Server is a Linux, running cvd command from a crontab:
> >
> > cvd update
> > 2023-03-15 18:20:10 cvdupdate-1.1.1 INFO Using system configured nameservers
> > 2023-03-15 18:20:10 cvdupdate-1.1.1 INFO main.cvd is up-to-date. Version: 62
> > 2023-03-15 18:20:10 cvdupdate-1.1.1 INFO daily.cvd is up-to-date. Version: 
> > 26842
> > 2023-03-15 18:20:10 cvdupdate-1.1.1 INFO bytecode.cvd is up-to-date. 
> > Version: 334
> >
> > But for some reason, Freshclam gives an error today:
> >
> > ClamAV update process started at Wed Mar 15 15:21:07 2023
> > daily.cld database is up-to-date (version: 26842, sigs: 2025908, f-level: 
> > 90, builder: raynman)
> > main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, 
> > builder: sigmgr)
> > bytecode database available for update (local version: 333, remote version: 
> > 334)
> > Current database is 1 version behind.
> > Downloading database patch # 334...
> > Time:    1.3s, ETA:    0.0s [>] 801B/801B
> > ERROR: downloadPatch: Can't apply patch
> > The database server doesn't have the latest patch for the bytecode database 
> > (version 334). The server will likely have updated if you check again in a 
> > few hours.
> >
> >
> >
> > I have updated Linux server with cvd update several times, but nothing 
> > happens.
> > If I change the mirror directive to get files from database.clamav.net 
> >  everything works fine.
> >
> > I'm using Clamav 1.0.1:
> >
> > freshclam --version
> > ClamAV 1.0.1/26842/Wed Mar 15 04:22:42 2023
> >
> >
> > What could be happening to my own mirror?
> > Thank you in advance,
> >
> > Jorge
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] ClamAV 0.103.8, 0.105.2 and 1.0.1 patch versions published

[Paul Kosinski via clamav-users]

I am using ClamAV 0.103.6 on Buster, but I have always built it from source 
(since way before Cisco and even SourceFire), hence I'm a bit obsolete.

I did this -- and still do it -- because ClamAV has always been a bit 
experimental. Thus I install each version under "/opt/clamav.d/version" so I 
can easily revert if there is a problem. (And I have similarly disabled the 
systemd linkage for more control).



On Mon, 20 Feb 2023 14:11:10 +0200
Brent Clark via clamav-users  wrote:

> Good day Guys
> 
> Anyone on Debian Buster and Bullseye?
> 
> How serious is this?
> Does anyone have any suggestions. Cause there is no packages available.
> 
> If anyone can share their thoughts / experiences.
> 
> Regards
> Brent
> 
> On 2023/02/18 21:13, unison.subject_0t--- via clamav-users wrote:
> > Vulnerabilities*
> >
> > —
> > Sent from my iPhone
> >  
> >> On Feb 18, 2023, at 13:54, Joel Esler  wrote:
> >>
> >> 100.3 hasn’t been supported in years.  There’s lots of our abilities that 
> >> affect the version.
> >>  
> >>> On Feb 18, 2023, at 13:36, George.G via clamav-users 
> >>>  wrote:
> >>>
> >>> 
> >>> Hello,
> >>>
> >>> I would like to ask whether these two new vulnerabilities affect the 
> >>> version 0.100.3.
> >>>
> >>> Thank you
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Subject: behaviour of clamAV with password protected pdf file.

[Paul Kosinski via clamav-users]

Compared to the following, encrypted PDFs are a very minor issue (in my 
opinion).

Most websites these days use HTTPS ("for security"), and make extensive use of 
Javascript (find a site that doesn't). This means that browsers are always 
executing code that can't be scanned (at least by ClamAV).

This flies in the face of the advice that we used to get in the days of DOS and 
early Windows -- don't download and execute code from random sources. Yet 
modern websites tend to pull Javascript from all over (as can be seen if you 
use NoScript). This is especially problematic with financial sites. (Do they 
screen their Javascript partners?)

I still use HAVP (which uses the ClamAV library), but it doesn't do anything 
really useful with HTTPS traffic. HTTPS traffic is like an endless stream of 
encrypted PDFs -- PDFs can optionally execute code, but Javascript always does.

I presume that some kind of browser modification could be devised to scan 
Javascript, but Firefox (for one) made that much more difficult when they 
radically changed their internal architecture a few years ago (partly for 
"security", they say).


On Tue, 14 Feb 2023 13:49:48 +0700
Olivier via clamav-users  wrote:

> > Hi team ,
> > We are using clamAVClient for scanning pdf and xlsx files in our Java
> > program. We came across the query,
> > does clamAV scan password protected pdf file or not? If yes ,
> > how we can restrict it? Kindly suggest. Best regards, Nahin Bagwan  
> 
> How do you expect ClamAV to know the password to decode the encrypted
> files?
> 
> No it does not because it cannot.
> 
> If you are concerned that encrypted files could be a security,
> quarantine these emails.
> 
> Best regards,
> 
> Olivier
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] About scanning files larger than 2 GB in size

[Paul Kosinski via clamav-users]

I don't think I implied that the 2 GiB limit was "artificial" in the sense of 
trivial, or made up. I think I very clearly stated that
"It's a holdover from when 32-bit numbers were all that CPUs supported" and now 
"the 2 GiB limit is quite an anachronism".

Note that this question has been around for at least 7 years: 
  
https://security.stackexchange.com/questions/107132/linux-antivirus-and-files-bigger-than-4gb

Clearly, much code review would have to be done. But Linux file I/O interfaces 
were successfully updated from 32-bit to 64-bit sizes and offsets some years 
ago, so the infrastructure is there. Also, the analogous Y2038 problem, which 
requires going from 32 to 64 bit as well (for time-stamps), is being seriously 
worked on. (And note that the Y2K problem, which was a *much* bigger issue, was 
indeed fixed.)

Paul

P.S. Do many current commercial AV suites for Windows have this limit?



On Thu, 26 Jan 2023 00:14:27 +
"Micah Snyder (micasnyd)"  wrote:

> Paul is sort-of correct but the 2GB limit isn't artificial as he has implied.
> 
> ClamAV code contains a lot of signed and unsigned 32bit variables that must 
> be upgraded to 64bit variables to support larger files.  Before raising the 
> limit, a tedious audit process must be completed to ensure that all variables 
> are upgraded in all modules.  We cannot simply remove the limit and cross our 
> fingers.
> 
> Regards,
> Micah
>
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] About scanning files larger than 2 GB in size

[Paul Kosinski via clamav-users]

On Sun, 22 Jan 2023 05:40:18 +0900
Tsutomu Oyamada  wrote:

> How do I set up clamd?
> Setting MaxFileSize to "0" is unlimited, but internally files larger than 2GB 
> in size cannot be scanned. 
> In this case, do you treat the file as clean without scanning it at all?


I've complained about the 2 GiB limit now and then for several years. It's a 
holdover from when 32-bit numbers were all that CPUs supported, and lots of 
code used C's signed "int" for file size and offset.

Way back in 1996, FAT32 had this limit, but even it was extended to 4 GiB (via 
unsigned 32-bit numbers) when Large File Support was added.

These days, media files are often bigger than 2 GiB, as are some archive files 
(esp. disk images). Furthermore, almost all recent versions of standard OSes 
are 64-bit, and very few 32-bit CPUs are even being sold. In other words, the 2 
GiB limit is quite an anachronism.

___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Anyone else having trouble reaching the ClamAV website?

[Paul Kosinski via clamav-users]

I occasionally see a similar message from sites other than clamav.net saying 
something equivalent to Cloudflare's "review the security of your connection".

The phrasing is pure gaslighting. It isn't for *connection* security -- HTTPS 
provides *that*. What it really means is that the site is trying to search your 
computer by running some Javascript (which I block by default via NoScript, 
thus causing the message). They assume, probably correctly, that most visitors 
will think it's for *their* benefit  After all, security is good, isn't it? 

Why can't Cloudflare et al be honest and say that they're trying to avoid 
Denial of Service attacks and other bandwidth overload?

  

On Thu, 5 Jan 2023 10:18:38 -0500
Kris Deugau  wrote:

> I went to load a semi-bookmarked page for signature writing 
> (https://docs.clamav.net/manual/Signatures.html), but it failed and kept 
> reloading Cloudflare's "security check" voodoo.
> 
> (Side question to pass up the chain at Cisco/Talos - is there a knob 
> that can be twisted somewhere to force that check to run exactly once, 
> then stop?  I can't imagine any scenario where running it over and over 
> and over has any benefit to anyone.  [And for bonus points, display an 
> error message that gives some sliver of a hint what 
> beyond-the-bleeding-edge headacheware the site or its security provider 
> insist on relying on this week.])
> 
> I then tried to load the main site, https://www.clamav.net, which also 
> went into the same loop.
> 
> I usually use Seamonkey (all-in-one Mozilla suite).  I tried Konqueror 
> which seemed to load things up fine.
> 
> Since starting to write this and putting it aside, I've come across a 
> small handful of other sites with the same issue, including one case 
> where the base site triggered the issue but a directory under the base 
> site did not.  Since I'm *not* seeing it across a large number of sites, 
> it's pretty clearly some specific security option in Cloudflare causing 
> the failure.
> 
> -kgd
___

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Inquire about clamav latest stable version -

[Paul Kosinski via clamav-users]

On Thu, 28 Jul 2022 17:38:20 -0400
Joel Esler  wrote:

> ClamAV is a Cisco project.  There’s no arguing that. 
> 
> All of the original team are observed here: https://www.clamav.net/about
> 
> So, not sure what you’re getting at.  

The phrase "*the* authors of the software" rather implies that Cisco's Talos 
are the only authors of the software. And G.W. Haywood seems to have agreed 
with me on this that the phrasing could be misinterpreted.


Cisco's Talos has indeed made ClamAV a lot better than it was years ago, but 
they have kept much of the basic structure and, I would guess, some of the 
original code.

I have attached a file list of the contents of the source code that comprised 
clamav-0.88.4 (from the tar of August 2006) and it indicates that the current 
structure is quite similar to the original.

P.S. The reason I have 0.88.4 is that whenever I download a new ClamAV, I keep 
the old one just in case I need to revert (it is security software, after all). 
I don't think I ever have, but disk storage has gotten so cheap that I haven't 
bothered to cull what are, by modern standards, fairly small files.


clamav-0.88.4.list
Description: Binary data
___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] No daily sig since July 28th

[Paul Kosinski via clamav-users]

On Mon, 1 Aug 2022 16:24:50 +0100 (BST)
Andrew C Aitchison via clamav-users  wrote:

> On Mon, 1 Aug 2022, Shawn Iverson via clamav-users wrote:
> 
> > Hello,
> >
> > I've noticed that a daily hasn't been posted since the 28th of July. Are
> > daily sigs being posted?  
> 
> #  clamscan --version
> ClamAV 0.103.7/26615/Thu Jul 28 08:58:07 2022
> 
> # host -t txt current.cvd.clamav.net.
> current.cvd.clamav.net descriptive text 
> "0.103.7:62:26615:1659362400:1:90:49192:333"
> 
> # date -u -d "1970-01-01 UTC 1659362400 seconds"
> Mon Aug  1 14:00:00 UTC 2022
> 
> ... so the magic DNS timestamp is being updated,
> but the daily version number has not changed since Thursday.

=

Same here on the Cloudflare 'BOS' anycast mirror.


--  Thursday 28 July 2022 at 04:43:01  
--

/opt/clamav/bin/testclam-dns
-->  UPD   D 26615/26614 M 62/62 B 333/333

/opt/clamav/bin/freshclam -v --stdout --on-update-execute=EXIT_1
...

...

--  Monday 01 August 2022 at 12:43:01  
--

/opt/clamav/bin/testclam-dns
-->  DNS   D 26615/26615 M 62/62 B 333/333


P.S. Testclam-dns is something I created a few years ago (before the bandwidth 
abuse) when the BOS mirror was often out of date in serving the latest CVD file 
(which I then mirrored locally). It reports the latest vs the currently 
installed versions of the 3 principal signature files ("daily", "main" & 
"bytecode"), and whether freshclam should be invoked. I still use it for its 
detailed reporting, but now each freshclam instance simply uses the CDIFFs 
directly. This saves bandwidth compared to locally mirrored CVDs -- unless one 
has *lots* of ClamAV instances.

 
___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] Inquire about clamav latest stable version -

[Paul Kosinski via clamav-users]

> At the moment three versions are officially supported by Cisco's Talos, the 
> authors of the software.

Cisco's Talos are the *current* authors of the software.

ClamAV was started in 2001 by Tomasz Kojm and a group of Open Source 
enthusiasts. In 2007, they sold the software to Sourcefire (of Snort fame), and 
the principal developers joined Sourcefire as employees.

Cisco acquired Sourcefire in 2013. Since the original software was covered by 
the GPLv2 license, Cisco has kept the source code open (as they must), 
including the many improvements they have made.


The Wikipedia article on ClamAV barely mentions its origin, but it does have 
two links:

  
https://web.archive.org/web/20120206053729/http://www.emailbattles.com/2005/08/31/virus_aabejfhaib_ag/
 
  (Tomasz Kojm interview)

  https://web.archive.org/web/20080828173858/http://www.clamav.net/about/

The latter in turn links to the original developer team:

  https://web.archive.org/web/20080828173858/http://www.clamav.net/about/team/


Disclaimer: I have never been associated with the development of ClamAV, but I 
have used it since well before the Sourcefire acquisition. (I even have a copy 
of the 0.88.4 source code from 2006!) 

In any case, I think the originators of ClamAV should get proper credit.
___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

[clamav-users] ClamAV's 'configure' doesn't seem to complain about invalid options

[Paul Kosinski via clamav-users]

Building 0.103.6, I ran 'configure' with the option "--disable-clamonaccess" 
(instead of "--disable-clamonacc") and got no error or warning that the option 
was not recognized. 

I did this because I realized that I had still been using the old 
"--disable-clamuko", which also had no effect, and gave no warning that it was 
ignored.

Since I never tried to use either feature (and divert all the systemd stuff to 
a dummy directory), the fact that these options were ignored just wasted some 
disk space.

But there might be situations where misspelled options being silently ignored 
could cause serious problems.
___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

Re: [clamav-users] clamdscan: Output detailed scan results to STDOUT or to configurable file?

[Paul Kosinski via clamav-users]

On Thu, 17 Feb 2022 14:08:45 +0100
An Schall via clamav-users  wrote:

> When using clamdscan, I would like to have verbose output logged to a
> file. Specifically, the timestamp, file path and file name as well as
> the scan results should be logged to a specified file.
> 
> In comparison, clamscan outputs this information to STDOUT per default
> and I could simply pipe it to "tee -a $LOG_FILE".
> 
> Unfortunately, clamdscan does not output this information but logs
> this kind of information to /var/log/clamd.scan. However, given that I
> would like to use it within a script, I would like to log this kind of
> output to a configurable file.
> 
> While there is a -L switch, it does not include such detailed
> information (only the summary). Also, there seems to be a --stdout
> switch but it seems it does not help either.
> 
> How can I get verbose information from /var/log/clamd.scan to a
> configurable file in the first place?
> 
> Thanks in advance!


Just off the top of my head, maybe a symlink, or, if you have to make it 
dynamic (e.g., for each clamdscan execution), a pipe/FIFO to a listener 
process? I've never tried this with clamd, so details would need to be filled 
in (and it might not even be practical).

You still would be limited to what clamd is willing to report, of course.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV 0.103.5 and 0.104.2 security patch release; 0.102 past EOL

[Paul Kosinski via clamav-users]

On Wed, 12 Jan 2022 20:12:42 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> Find this announcement online at: 
> https://blog.clamav.net/2022/01/clamav-01035-and-01042-security-patch.html
> 
> 
> ClamAV versions 0.103.5 and 0.104.2 are now available for download on the 
> clamav.net Downloads page.
>

=

Since 0.103.x is supposed to be a Long Term Support Release, 0.103.5 shouldn't 
be hidden under "Previous Stable Releases" along with myriad versions that are 
End Of Life (and beyond).

It was surprisingly hard find. I went to the Downloads page (above) and went 
back and forth among the alternatives, finally deciding that "Previous Stable 
Releases" was the only remaining hope.

There should be a separate category for Long Term Support source files, 
preferably right after "The latest stable release is: ...".

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Problem installing ClamAV 104.1 on CentOS 7

[Paul Kosinski via clamav-users]

On Mon, 6 Dec 2021 16:41:51 -0500
Bowie Bailey via clamav-users  wrote:

> I followed the instructions to install the prerequisites and then went 
> through the 
> steps for the default build.  Everything went fine until I got to the last 
> step.
> 
> $ sudo cmake --build . --target install
> sudo: cmake: command not found
> 
> The instructions have me install cmake under my user directory (python3 -m 
> pip 
> install --user cmake pytest).  That works fine for the build and testing 
> steps, 
> however, it becomes a problem when I try to do the install under sudo.
> 
> Do I need to re-install cmake system-wide, or is there a way I can specify to 
> use the 
> local version when running sudo?
> 
> Could I log in as root, add the /home/username/.local/bin directory to the 
> path, and 
> then do the install?  If so, do I need to add the /home/username/.local/lib 
> directory 
> to a lib variable?
> 
> I could just experiment with the settings, but that's a bit dangerous when 
> running an 
> install as root.
> 

-

I also don't like running "install" as root.

I have always built ClamAV from source and installed it under
"/opt/clamav-VERSION". This was originally because I didn't want to
wipe out a previous working version until I tested the new version.

But it turns out to have another advantage as well: I can do *all* the
"make" stuff, including the "make install", as non-root. Then, at the
last minute, I do a "chmod -R 0.0" of the new installation, followed by
a "chmod -R clamuid.clamgid" of the "share/clamav/" subdirectory,
followed by setting a symbolic link of "/opt/clamav" to the new
version's subdirectory under "/opt".

Of course this also requires creating the new "/opt/clamav-VERSION"
directory with the proper permissions, changing some of the "make"
options to non-standard values, and now, changing an option or two to
put the systemd stuff into a different place to avoid tying ClamAV to
systemd. (I continue to use cron to trigger updates.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Fail to download source archive with 403 forbitten

[Paul Kosinski via clamav-users]

On Mon, 15 Nov 2021 13:23:49 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> On Nov 14, 2021, at 19:11, Yasuhiro Kimura 
> mailto:y...@utahime.org>> wrote:
> 
> These results means server checks User-Agent header of HTTP request
> and returns 403 forbitten if the value doesn't look like that of web
> browser.
> 
> Then is it intened change?
> 
> Yes, and it has been this way for over two years.
> 
> --
> Joel Esler
> Strategy, Cisco Talos Intelligence Group


Does anyone do automated updating of ClamAV from source when new fixes become
available (e.g., www.clamav.net/downloads/production/clamav-0.104.N.tar.gz)?

This sort of restriction could make it awkward.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clam updates failing

[Paul Kosinski via clamav-users]

On Fri, 22 Oct 2021 18:47:01 +
"Joel Esler (jesler)"  wrote:

> > On Oct 22, 2021, at 14:16, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > On Fri, 22 Oct 2021 13:27:46 +
> > "Joel Esler \(jesler\) via clamav-users"  
> > wrote:
> >   
> >>> On Oct 21, 2021, at 18:55, Kenneth Porter  wrote:
> >>> 
> >>> On 10/21/2021 10:14 AM, Paul Kosinski via clamav-users wrote:
> >>>> I've never seen a DNS age warning, but that might be because, for 
> >>>> several years now, I only run freshclam when the DNS TXT record (which I 
> >>>> check hourly) says there is a new signature available compared to a 
> >>>> local file's version number (in its header).
> >>> 
> >>> I thought freshclam did the DNS check itself. Why do it again before 
> >>> running freshclam?
> >> 
> >> It does.  No need to do an extra check.  
> > 
> > 
> > Since checking the DNS TXT record costs almost nothing (and is UDP), I 
> > figure I can do it more often than running freshclam without ever risking 
> > triggering Cloudflare's bandwidth limits. And, although I currently do it 
> > only once per hour, if there ever was something like a SANS Threat Level 
> > RED, I could up the frequency to get the latest sigs ASAP.
> > 
> >   
> 
> DNS is unrestricted.  That’s why I am saying it’s unnecessary.  The 
> restrictions are on the files themselves.  


So you're saying that if -- because I wanted to get an update ASAP in the face 
of a severe virus alert -- I upped the running of freshclam to every 5 minutes 
on each of my 3 systems, there is no chance that I would be blocked, because 
freshclam doesn't do any actual (restricted) file access until after it checks 
the DNS TXT record?

Even if that's the case, I think it would generate a lot more junk in the log 
files than my current approach does (since I run freshclam with the "-v" 
option).

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clam updates failing

[Paul Kosinski via clamav-users]

On Fri, 22 Oct 2021 13:27:46 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> > On Oct 21, 2021, at 18:55, Kenneth Porter  wrote:
> > 
> > On 10/21/2021 10:14 AM, Paul Kosinski via clamav-users wrote:  
> >> I've never seen a DNS age warning, but that might be because, for several 
> >> years now, I only run freshclam when the DNS TXT record (which I check 
> >> hourly) says there is a new signature available compared to a local file's 
> >> version number (in its header).  
> > 
> > I thought freshclam did the DNS check itself. Why do it again before 
> > running freshclam?  
> 
> It does.  No need to do an extra check.


Since checking the DNS TXT record costs almost nothing (and is UDP), I figure I 
can do it more often than running freshclam without ever risking triggering 
Cloudflare's bandwidth limits. And, although I currently do it only once per 
hour, if there ever was something like a SANS Threat Level RED, I could up the 
frequency to get the latest sigs ASAP.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clam updates failing

[Paul Kosinski via clamav-users]

On Thu, 21 Oct 2021 15:55:54 -0700
Kenneth Porter  wrote:

> On 10/21/2021 10:14 AM, Paul Kosinski via clamav-users wrote:
> > I've never seen a DNS age warning, but that might be because, for several 
> > years now, I only run freshclam when the DNS TXT record (which I check 
> > hourly) says there is a new signature available compared to a local file's 
> > version number (in its header).  
> 
> I thought freshclam did the DNS check itself. Why do it again before 
> running freshclam?


Because a couple of years ago I was running a local mirror using full CVDs, but 
Cloudflare's BOS POP/server was often out of date compared to the claimed 
current CVD version. So I was trying to reduce bandwidth consumption by not 
directly downloading the whole CVD. Now I use freshclam directly on our 3 
ClamAV systems, but I kept the DNS TXT check as it still reduces bandwidth a 
bit (compared to hourly freshclam runs) and also provides a nice summary of 
available vs installed DB files.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clam updates failing

[Paul Kosinski via clamav-users]

On Thu, 21 Oct 2021 10:20:58 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Thu, 21 Oct 2021, Ben Argyle via clamav-users wrote:
> 
> > Has anyone been having trouble downloading updates for the last 20
> > hours or so?  ...  
> 
> Yesterday I saw a couple of warnings about the DNS record being old,
> but apart from that things seem to have been working OK here:


I've never seen a DNS age warning, but that might be because, for several years 
now, I only run freshclam when the DNS TXT record (which I check hourly) says 
there is a new signature available compared to a local file's version number 
(in its header).

Maybe I should also look at the DNS record's TS field. What is considered "old" 
for this?

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] QNAP Antivirus Updates

[Paul Kosinski via clamav-users]

"how's this different from what Joel said?"

My reading of the following (based on normal English convention)

  > >  104.16.218.84
  > >  104.16.219.84  
  > That’s what they are for you.  Cloudflare routes you to the closest pop to 
your network.  Your mileage may vary  

is that "they" refers to the IP addresses, NOT the DNS names (which hadn't even 
been mentioned in my email at this point).

Thus, what I inferred from Joel's statement is that "database.clamav.net" might 
resolve to different IPs for other users (which would be weird, given the use 
of Anycast). So I tested it the best I could (without traveling a lot, or 
setting up VMs in different countries).


On Tue, 21 Sep 2021 13:21:20 +0200
Matus UHLAR - fantomas  wrote:

> >On Mon, 20 Sep 2021 17:17:34 +
> >"Joel Esler (jesler)"  wrote:
> >  
> >> > On Sep 20, 2021, at 13:08, Paul Kosinski via clamav-users 
> >> >  wrote:
> >> >
> >> > These two IPs are Anycast addresses, and have been unchanged for well 
> >> > over 2 years. (Anycast addresses don't have to change even if the 
> >> > physical servers change, that's their point!) They are:
> >> >
> >> >  104.16.218.84
> >> >  104.16.219.84  
> >> That’s what they are for you.  Cloudflare routes you to the closest pop to 
> >> your network.  Your mileage may vary  
> 
> On 20.09.21 20:16, Paul Kosinski via clamav-users wrote:
> >I thought the IP addresses, being Anycast, were what are routed to the 
> >closest POP.  
> 
> how's this different from what Joel said?
> 
> > No matter, when I resolve "database.clamav.net" via various DNS servers,
> > using TCP to bypass the default local DNS server (as our firewall blocks
> > outbound UDP port 53 otherwise), I always get these same two IP addresses
> > as results (see below)  
> 
> yes, becaue those two IP are anycast... they are router to the nearest POP.
> 
> > Given that the servers at 1.1.1.1, 8.8.8.8 and 9.9.9.9 are "public", and
> > likely Anycast, while 71.243.0.12 is local Verizon/FIOS, I suppose that
> > the Authoritative server and the public (Anycast) servers could
> > conceivably be distributing different IP addresses depending on who is
> > querying.  (BIND/named has become incredibly complicated these days.) But
> > since the two IP addresses are themselves Anycast, what would be the
> > point?  
> 
> the point is, not to provide different IPs via anycast DNS but to provide
> anycast IPs via any DNS.
> 
> > In any case, does anyone, anywhere, get IP addresses other than
> >
> >  104.16.218.84
> >  104.16.219.84
> >
> > when resolving "database.clamav.net"?  
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] QNAP Antivirus Updates

[Paul Kosinski via clamav-users]

On Mon, 20 Sep 2021 17:17:34 +
"Joel Esler (jesler)"  wrote:

> > On Sep 20, 2021, at 13:08, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > These two IPs are Anycast addresses, and have been unchanged for well over 
> > 2 years. (Anycast addresses don't have to change even if the physical 
> > servers change, that's their point!) They are:
> > 
> >  104.16.218.84
> >  104.16.219.84  
> That’s what they are for you.  Cloudflare routes you to the closest pop to 
> your network.  Your mileage may vary

===

I thought the IP addresses, being Anycast, were what are routed to the closest 
POP.

No matter, when I resolve "database.clamav.net" via various DNS servers, using 
TCP to bypass the default local DNS server (as our firewall blocks outbound UDP 
port 53 otherwise), I always get these same two IP addresses as results (see 
below) 

Given that the servers at 1.1.1.1, 8.8.8.8 and 9.9.9.9 are "public", and likely 
Anycast, while 71.243.0.12 is local Verizon/FIOS, I suppose that the 
Authoritative server and the public (Anycast) servers could conceivably be 
distributing different IP addresses depending on who is querying. (BIND/named 
has become incredibly complicated these days.) But since the two IP addresses 
are themselves Anycast, what would be the point?

In any case, does anyone, anywhere, get IP addresses other than

  104.16.218.84
  104.16.219.84

when resolving "database.clamav.net"?
  

  
  $ dig +tcp +all @1.1.1.1 database.clamav.net
  
  ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> +nocomments +nostats +nocmd +tcp +all 
@1.1.1.1 database.clamav.net
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5920
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;database.clamav.net. IN  A
  
  ;; ANSWER SECTION:
  database.clamav.net.  31  IN  CNAME   
database.clamav.net.cdn.cloudflare.net.
  database.clamav.net.cdn.cloudflare.net.   271 IN A 104.16.219.84
  database.clamav.net.cdn.cloudflare.net.   271 IN A 104.16.218.84
  
  ;; Query time: 11 msec
  ;; SERVER: 1.1.1.1#53(1.1.1.1)
  ;; WHEN: Mon Sep 20 15:28:17 2021
  ;; MSG SIZE  rcvd: 118
  
  ---
  
  $ dig +tcp +all @8.8.8.8 database.clamav.net
  
  ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> +nocomments +nostats +nocmd +tcp +all 
@8.8.8.8 database.clamav.net
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49012
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;database.clamav.net. IN  A
  
  ;; ANSWER SECTION:
  database.clamav.net.  19  IN  CNAME   
database.clamav.net.cdn.cloudflare.net.
  database.clamav.net.cdn.cloudflare.net.   300 IN A 104.16.218.84
  database.clamav.net.cdn.cloudflare.net.   300 IN A 104.16.219.84
  
  ;; Query time: 31 msec
  ;; SERVER: 8.8.8.8#53(8.8.8.8)
  ;; WHEN: Mon Sep 20 15:21:13 2021
  ;; MSG SIZE  rcvd: 118
  
  ---
  
  $ dig +tcp +all @9.9.9.9 database.clamav.net
  
  ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> +nocomments +nostats +nocmd +tcp +all 
@9.9.9.9 database.clamav.net
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29165
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;database.clamav.net. IN  A
  
  ;; ANSWER SECTION:
  database.clamav.net.  60  IN  CNAME   
database.clamav.net.cdn.cloudflare.net.
  database.clamav.net.cdn.cloudflare.net.   300 IN A 104.16.218.84
  database.clamav.net.cdn.cloudflare.net.   300 IN A 104.16.219.84
  
  ;; Query time: 91 msec
  ;; SERVER: 9.9.9.9#53(9.9.9.9)
  ;; WHEN: Mon Sep 20 15:30:17 2021
  ;; MSG SIZE  rcvd: 118
  
  ---
  
  $ dig +tcp +all @71.243.0.12 database.clamav.net
  
  ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> +nocomments +nostats +nocmd +tcp +all 
@71.243.0.12 database.clamav.net
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12056
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;database.clamav.net. IN  A
  
  ;; ANSWER SECTION:
  database.clamav.net.  60  IN  CNAME   
database.clamav.net.cdn.cloudflare.net.
  database.clamav.net.cdn.cloudflare.net.   144 IN A 104.16.218.84
  database.clamav.net.cdn.cloudflare.net.   144 IN A 104.16.219.84
  
  ;; Query time: 16 msec
  ;; SERVER: 71.243.0.12#53(71.243.0.12)
  ;; WHEN: Mon Sep 20 15

Re: [clamav-users] QNAP Antivirus Updates

[Paul Kosinski via clamav-users]

On Mon, 20 Sep 2021 08:18:01 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Sun, 19 Sep 2021, Gregory Poveda via clamav-users wrote:
> 
> > I have several QNAPs  
> 
> It might be worth searching for 'QNAP' in the list archives.  At least
> some of those devices will struggle to run ClamAV - or rather, ClamAV
> out of the box - for lack of memory.
> 
> > on a locked down network that have the Clamav.net antivirus package/
> > software installed. Something changed on the 16th and I have been
> > unable to get updates. I have an ACL that blocks all traffic on this
> > network unless I define its IPs/DNS addresses. I had set the two DNS
> > addresses that I had detected back in March in the ACL, those are as
> > follows: clamav.net (199.62.84.153) which appears to check if the
> > database as an update and database.clamav.net (198.148.79.54) which
> > has the update file.  
> 
> If you don't mind my saying so, that's a fragile setup.  IPs can and
> do change without notice.
> 
> > Did the DNS names change or has the database stopped providing
> > updates?  
> 
> Check the very recent thread  "Virus DB  updates?".

=

Using an ACL mechanism that uses DNS names to allow outbound traffic strikes me 
as also a setup that is either fragile or very slow. Either it does a DNS 
lookup when started, so if the DNS->IP map changes while it's running, you 
lose. Or it does a reverse DNS (PTR) lookup for every outbound SYN to see if 
it's OK, and it's slow.

In my case, I use iptables (on Linux) to block almost all outbound TCP from 
select servers, and I use two IP addresses (only) to allow ClamAV update 
traffic, from/to freshclam.

These two IPs are Anycast addresses, and have been unchanged for well over 2 
years. (Anycast addresses don't have to change even if the physical servers 
change, that's their point!) They are:

  104.16.218.84
  104.16.219.84

I don't know if they are appropriate for non-freshclam ways of obtaining the 
updates, e.g., updating a mirror. (And I don't know if they work world-wide.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Virus DB updates?

[Paul Kosinski via clamav-users]

I haven't seen any virus database update since the afternoon of Thu 16 Sep 
2021, when it was updated to 26297.

Are updates really this stagnant, or does the DNS TXT record at 
"current.cvd.clamav.net" no longer reflect the state of things?

(For a bit more bandwidth savings, I only run freshclam when the DNS TXT record 
says there is a newer version of one of the 3 files.)


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] IP List for Virus Definition Domain

[Paul Kosinski via clamav-users]

When I do a DNS lookup I also get:

  104.16.218.84
  104.16.219.84

This is the same result that I got well over a year ago, when I had to add 
these IP addresses as holes in my firewall so that my normally isolated 
internal server could update its ClamAV instance.

These are Anycast addresses, so the physical servers can move around without 
needing to change IP addresses. (There might even be multi-machine 
load-balancing at a single Cloudflare POP, I suppose.)



On Wed, 15 Sep 2021 18:17:56 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Wed, 15 Sep 2021, James Freeman wrote:
> 
> > Is there a list of IPs that the ClamAV domain used to download virus 
> > definition resolves to?  
> 
> Here's the (very short) list that it resolves to from my location:
> 
> $ dig +short database.clamav.net
> database.clamav.net.cdn.cloudflare.net.
> 104.16.218.84
> 104.16.219.84
> 
> It's a content delivery network - do the same query where you are and
> you'll probably get different answers.  But you won't get a complete
> list unless you qeury from locations all over the planet.
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] error code 429

[Paul Kosinski via clamav-users]

On Sun, 5 Sep 2021 18:27:09 +
"Joel Esler (jesler)"  wrote:

> Now?

-

All 3 systems updated successfully as soon as our DNS TXT test said the 26285 
update was available (see below).

This is again as it is almost every time since the download limiting mechanism 
stabilized.

P.S. The DNS TXT record test is performed once per hour (via cron) at a 
different minute for each system, so as to spread out the load originating from 
our NAT-shared (DHCP-allocated) external IP address.

=

Freshclam log excerpt from one of 3 systems:

--  Sunday 05 September 2021 at 05:05:02  
--

/opt/clamav/bin/testclam-dns
-->  UPD   D 26285/26284 M 61/61 B 333/333

/opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
ClamAV update process started at Sun Sep  5 05:05:05 2021
daily database available for update (local version: 26284, remote version: 
26285)
Testing database: 
'/opt/clamav.d/clamav.0.103.3/share/clamav/tmp.483d8f034e/clamav-121d7a6b001da0b17cc1d5ec5c709f57.tmp-daily.cld'
 ...
Database test passed.
daily.cld updated (version: 26285, sigs: 1970840, f-level: 90, builder: raynman)
main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)

--  Sunday 05 September 2021 at 05:05:11  
--

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] error code 429

[Paul Kosinski via clamav-users]

On Sun, 5 Sep 2021 02:45:25 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> We are experimenting with a feature that we’ve been working with Cloudflare 
> on, trying to isolate violators on a per host basis for the newest versions 
> of ClamAV, instead of IP.

-

Maybe what we have seen  today is the old problem that the "BOS" mirror is 
wy behind again? We finally got the 26284 update about 17 hours later than 
the TXT record claimed it was available.

Is it possible to find out from Cloudflare why "BOS" has this problem? Some 
time ago, when we were downloading full-blown CVDs (not just CDIFFs), I was 
able to use another mirror which was up to date on the same day "BOS" was 
behind. Now even the small CDIFFs are behind?

Thanks,
Paul Kosinski

  --  Saturday 04 September 2021 at 22:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  UPD   D 26284/26283 M 61/61 B 333/333
  
  /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
  ClamAV update process started at Sat Sep  4 22:05:04 2021
  daily database available for update (local version: 26283, remote version: 
26284)
  WARNING: downloadPatch: Can't download daily-26284.cdiff from 
https://database.clamav.net/daily-26284.cdiff
  The database server doesn't have the latest patch for the daily database 
(version 26284). The server will likely have updated if you check again in a 
few hours.
  main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
  bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)
  
  --  Saturday 04 September 2021 at 22:05:04  
--
  
  
  --  Saturday 04 September 2021 at 23:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  UPD   D 26284/26283 M 61/61 B 333/333
  
  /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
  ClamAV update process started at Sat Sep  4 23:05:03 2021
  daily database available for update (local version: 26283, remote version: 
26284)
  Testing database: 
'/opt/clamav.d/clamav.0.103.3/share/clamav/tmp.d80b44a62a/clamav-8a28103bbb9da5d3289b9e7252001905.tmp-daily.cld'
 ...
  Database test passed.
  daily.cld updated (version: 26284, sigs: 1970546, f-level: 90, builder: 
raynman)
  main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
  bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)
  
  --  Saturday 04 September 2021 at 23:05:10  
--




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] error code 429

[Paul Kosinski via clamav-users]

On Sat, 4 Sep 2021 15:01:00 +0100
Paul Netpresto via clamav-users  wrote:

> Hi all
> 
> Similar issue from Manchester UK. 4 mx's  all failing to collect today's 
> update apparently first available 9:50 am today


Not rate limited (as we only check about once per hour, from each of 3 
systems), but we're not getting updates.

In the past, I would have blamed it on Cloudflare's "BOS" mirror being uniquely 
slow to get updated (as was often the case), but with these other reports, it 
sounds like something more.

Here is what we've seen today -- 8 hours with no update actually available. 
(Testclam-DNS retrieves the TXT record to see when an update is allegedly 
available; 26284 was supposed to be available at 6:05 AM.)

  
  --  Saturday 04 September 2021 at 05:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  DNS   D 26283/26283 M 61/61 B 333/333
  
  
  --  Saturday 04 September 2021 at 06:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  UPD   D 26284/26283 M 61/61 B 333/333
  
  /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
  ClamAV update process started at Sat Sep  4 06:05:05 2021
  daily database available for update (local version: 26283, remote version: 
26284)
  WARNING: downloadPatch: Can't download daily-26284.cdiff from 
https://database.clamav.net/daily-26284.cdiff
  The database server doesn't have the latest patch for the daily database 
(version 26284). The server will likely have updated if you check again in a 
few hours.
  main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
  bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)
  
  --  Saturday 04 September 2021 at 06:05:06  
--
  
  
  --  Saturday 04 September 2021 at 07:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  UPD   D 26284/26283 M 61/61 B 333/333
  
  /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
  ClamAV update process started at Sat Sep  4 07:05:03 2021
  daily database available for update (local version: 26283, remote version: 
26284)
  WARNING: downloadPatch: Can't download daily-26284.cdiff from 
https://database.clamav.net/daily-26284.cdiff
  The database server doesn't have the latest patch for the daily database 
(version 26284). The server will likely have updated if you check again in a 
few hours.
  main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
  bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)
  
  --  Saturday 04 September 2021 at 07:05:04  
--

  ... [removed for brevity]  
  
  --  Saturday 04 September 2021 at 14:05:01  
--
  
  /opt/clamav/bin/testclam-dns
  -->  UPD   D 26284/26283 M 61/61 B 333/333
  
  /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1
  ClamAV update process started at Sat Sep  4 14:05:03 2021
  daily database available for update (local version: 26283, remote version: 
26284)
  WARNING: downloadPatch: Can't download daily-26284.cdiff from 
https://database.clamav.net/daily-26284.cdiff
  The database server doesn't have the latest patch for the daily database 
(version 26284). The server will likely have updated if you check again in a 
few hours.
  main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, 
builder: sigmgr)
  bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, 
builder: awillia2)
  
  --  Saturday 04 September 2021 at 14:05:04  
--
  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: Changes to ClamAV end-of-life policy and a new Long Term Support policy

[Paul Kosinski via clamav-users]

LTS is great! Earlier this year it seemed like I was spending 1 day per week 
trying to keep up with ClamAV updates, lockouts etc. Now I have time to do more 
forward looking software work.


On Fri, 3 Sep 2021 15:52:10 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> > 
> > https://blog.clamav.net/2021/09/changes-to-clamav-end-of-life-policy.html 
> > 
> > 
> > Changes to ClamAV end-of-life policy and a new Long Term Support policy
> > 
> > Today, we're announcing changes to the ClamAV End-of-Life (EOL) policy to 
> > include a new Long Term Support program.
> > 
> >  <>These are the main points of the updated EOL policy:
> > 
> > Long Term Support (LTS) Feature Releases
> > 
> > ClamAV 0.103 is the first Long Term Support (LTS) feature release.
> > 
> > LTS feature releases will be supported for at least three years from the 
> > initial publication date of that LTS feature version. In other words, 
> > support for the LTS release "X.Y" starts when version "X.Y.0" is published 
> > and ends three years after.
> > 
> > Each LTS feature release will be supported with critical patch versions and 
> > access to download signatures for the duration of the three-year support 
> > period.
> > 
> > A new LTS feature release will be identified approximately every two years.
> > 
> > Users must stay up-to-date with the latest patch versions for continued 
> > support. As of Aug. 28, that means version 0.103.3.
> > 
> > Regular (non-LTS) Feature Releases
> > 
> > Non-LTS feature releases will be supported with critical patch versions for 
> > at least four months from the initial publication date of the next feature 
> > release or until the feature release after that is published.
> > 
> > Non-LTS feature releases will be allowed access to download signatures 
> > until at least four months after the feature release after that is 
> > published.
> > 
> > If the end-of-life for a version has to change due to a compatibility 
> > problem, that prohibits the use of new detection technology or affects the 
> > stability of ClamAV infrastructure, we will announce the end of life for 
> > those versions four months before they become unsupported.
> > 
> > You can find full details for the updated End-of-Life policy, including a 
> > version-support matrix and definitions for the policy terminology in the 
> > EOL policy page in our online documentation 
> > .  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Second Release Candidate is here!

[Paul Kosinski via clamav-users]

On Tue, 24 Aug 2021 23:08:52 +
"Micah Snyder (micasnyd)"  wrote:

> This conversation is a fun read!  But don't worry really no point removing 
> the docs from the source package or the pre-compiled packages.  Including it 
> is painless at this point.  If you're curious why, here's the process...
> 
> The documentation website source is hosted in our 
> Cisco-Talos/clamav-documentation
>  repo.
> 
> Any time there is a change to the docs, GitHub Actions automatically 
> re-builds the static site using mdBook and force-pushes it to the 
> gh-pages 
> branch to publish it.
> 
> To include the docs in the source tarball, all we do (Jenkins does) is copy 
> the contents of that branch into the 
> clamav/docs/html 
> directory before building the source package.
> 
> >From there, the build system takes care of it.  The docs/html directory is 
> >bundled into the tarball, and when building the pre-compiled packages, the 
> >html directory is marked for installation and so is included in each 
> >package.  
> 
> That also means that if you're not building from the release tarball (i.e. if 
> you're building from a git clone), you won't get an offline copy of the 
> documentation.
> 
> -Micah
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.


Sounds good!

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Second Release Candidate is here!

[Paul Kosinski via clamav-users]

On Tue, 24 Aug 2021 10:48:48 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Mon, 23 Aug 2021, Paul Kosinski via clamav-users wrote:
> 
> > On Sun, 22 Aug 2021 14:42:06 +
> > "Joel Esler via clamav-users" wrote:
> >  
> >> I’m a fan of the thought of removing the user manual completely
> >> from the downloaded packages and including a link to
> >> docs.ClamAV.net.  Since that’s more dynamic.  
> >
> > I think that's a bad idea for three reasons:
> >
> > First, the Website might be (temporarily) inaccessible.
> >
> > Second, the machine running ClamAV may be blocked from accessing the 
> > Internet ...
> >
> > Finally, if the documentation is "dynamic", it presumably is for the latest 
> > release ...  
> 
> +1
> 
> I suspect most ClamAV installations are via packages for the various
> distributions.  Please think about the distribution package managers.
> If the documentation were not included in the release tarballs they'd
> have a great deal more work to do, and the opportunities for errors
> and confusion would be increased out of all proportion.


Good point about the Release Managers.

Maybe the GPL should augmented to require docs to be available in the same way 
as the source code, when GPLed executables are distributed :-)


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Yara regular expression finds only first match in ClamAV ?

[Paul Kosinski via clamav-users]

On Sun, 22 Aug 2021 20:10:00 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Sun, 22 Aug 2021, Richard Graham via clamav-users wrote:
> > On Sun, Aug 22, 2021 at 10:41 AM Zvi Kave wrote:  
> >> On 8/19/2021 9:33 PM, G.W. Haywood via clamav-users wrote:  
> >>> On Thu, 19 Aug 2021, Zvi Kave via clamav-users wrote:  
> 
>  I found that yara strings like this: $re = /[0-9]{9}/
>  find only first 9-digit match in file.
>  This spoils my logic ...  
> >>>
> >>> ... my advice is not to try anything fancy ...  
> >>
> >> I understand that I have to be patient.  
> >
> > I'm wondering if the --allmatch option/switch is useful here.  
> 
> Unfortunately I'm afraid it's a diffferent issue.  Yara rules don't
> necessarily produce a match (one which ClamAV would report as FOUND)
> even if there are strings in the Yara rules which _do_ in fact match.
> The point is that you can (or should be able to) tell Yara things like
> "count the number of times the string is found in the text, and report
> if there are more than 23 of them".  This sort of thing will sometimes
> work with the Yara engine in ClamAV, but my experience is that it's at
> the fancy end of the scale and I've spent hours trying to get things
> to work which would seem to be trivial exercises in regexes and logic.



Maybe ClamAV should support plugins, rather than being constrained to what's 
compiled in. (There are, of course, various plugins that invoke ClamAV, but 
that's not what I mean.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Second Release Candidate is here!

[Paul Kosinski via clamav-users]

On Sun, 22 Aug 2021 14:42:06 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> I’m a fan of the thought of removing the user manual completely from the 
> downloaded packages and including a link to docs.ClamAV.net.   Since that’s 
> more dynamic. 


I think that's a bad idea for three reasons:

First, the Website might be (temporarily) inaccessible.

Second, the machine running ClamAV may be blocked from accessing the Internet 
in general. E.g., our mail server runs ClamAV, but is explicitly blocked from 
general outbound Internet access by IPtables (except for the few anycast IP 
addresses needed for DB updates). It's an application of the "principle of 
least privilege".

Finally, if the documentation is "dynamic", it presumably is for the latest 
release, probably the latest "official" release. If that is the case, how can 
somebody use those docs to diagnose a problem with a slightly older release 
that's still supposed to be usable? Aren't most problems due to 
misunderstanding, bad configuration or other user caused issues that won't be 
solved by simply upgrading? In other words, aren't the docs really specific to 
a particular release? (This is especially important for beta releases like 
0.104.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Long Term Support (LTS) program proposal

[Paul Kosinski via clamav-users]

On Tue, 3 Aug 2021 07:53:24 +0200
Damian via clamav-users  wrote:

> > The current "stable" Debian is 10/Buster. It has ClamAV 0.103.2, patched by 
> > Debian to "deb10u1" (whatever that implies)  
> 
> https://security-tracker.debian.org/tracker/source-package/clamav


Interesting, but *much* more work to figure out how it all relates to 0.103.3 
than simply building 0.103.3 from source. (Has Debian fixed any problems that 
the ClamAV team hasn't fixed? If so, that's scary.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Long Term Support (LTS) program proposal

[Paul Kosinski via clamav-users]

On Sat, 31 Jul 2021 20:32:23 +0200
Matus UHLAR - fantomas  wrote:

> can't count on Debian?

They are very conservative, which is usually nice. But for security software, 
not so nice.

The current "stable" Debian is 10/Buster. It has ClamAV 0.103.2, patched by 
Debian to "deb10u1" (whatever that implies). Since it's not based directly on 
0.103.3, I'd have to analyze it, and that would be more trouble than just 
building from source.

And, according to Debian (https://tracker.debian.org/pkg/clamav), 0.103.3 is 
now "unstable", and thus may need special action to be added to Debian 
11/Bullseye before its Aug 14 release. (Bullseye is in "full freeze" according 
to https://lists.debian.org/debian-devel-announce/2021/07/msg2.html.)


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Long Term Support (LTS) program proposal

[Paul Kosinski via clamav-users]

On Sat, 31 Jul 2021 02:37:53 +
"Joel Esler (jesler)"  wrote:

> > On Jul 30, 2021, at 14:41, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > (I don't see exactly how a LTS would have helped with the bandwidth issue, 
> > but I suppose it wouldn't have made it any more disruptive.)  
> 
> 103.2 and 103.3 are much more respectful to bandwidth than any past version.  
> We’ve been working with Cloudflare on the development of a feature that is 
> exactly optimized to handle these changes, and they are almost ready to 
> deploy it.  We’ve tested it out and it works well, so we’re just waiting on a 
> few more updates. 
> 


What I meant was that if (e.g.) 0.103.1 had been LTS, that alone wouldn't have 
solved the bandwidth overload. Making (e.g.) 0.104.x LTS will (of course) 
mitigate future bandwidth problems.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Opinion wanted: Change default config directory usr/clamav

[Paul Kosinski via clamav-users]

On Sat, 31 Jul 2021 12:03:36 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> Hi all,
> 
> I could use your opinion about a change we'd planned to make in 0.104. By 
> request, I'd made this pull request to change the default directory for the 
> config files from /etc to /etc/clamav. The purpose being to 
> de-clutter /usr/local/etc: https://github.com/Cisco-Talos/clamav/pull/182
> 
> I procrastinated merging it for a long time because while it feels like a 
> good change it will require a change to our test framework and would likely 
> break scripts for other users as well. Some justification for the change at 
> this time is that with the build system change to cmake, users will likely 
> have to change the same scripts anyhow so we don't have any better time.
> 
> Unfortunately, I procrastinated it so hard I forgot to merge it before the 
> first release candidate. Now, I can't decide if it's right to merge it before 
> the second release candidate or throw it away. I would like your opinion.
> 
> Regards,
> Micah


When are started using ClamAV (<= 0.88.4), I built it in "/opt". Since then, 
it's always in "/opt/clamav.d/clamav.0.major.minor" with a symbolic link like 
"/opt/clamav -> /opt/clamav.d/clamav.0.103.3". This allows me to keep previous 
versions easily accessible in case of a problem.

The only real problem I remember having was when the latest version of 
libclamav wasn't compatible with the current version of HAVP (which I also 
build from source into "/opt" in the same manner). Fortunately, HAVP was 
updated soon after, but it allowed me to scan email with the latest ClamAV 
while keeping HAVP working adequately.

P.S. I tend to think "/opt" is preferable to "/usr" for software like ClamAV, 
at least until Linux distributors keep them up to date more diligently (as 
befits security software), so users don't have to build them on their own,

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [OT] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Thu, 29 Jul 2021 23:33:02 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Thu, 29 Jul 2021, Paul Kosinski via clamav-users wrote:
> 
> > ... do any firewall distros address inter-LAN filtering?  
> 
> We're well off-topic here so I think we should stop this now, but I
> thought most of them do.  What you describe is what I think they
> usually call a 'DMZ', very often 'ORANGE', where the LAN is 'GREEN'
> and the public Internet 'RED'.

As I understand it, a DMZ is usually where servers sit, to be accessed *from* 
the Internet; thus must allow inbound TCP connections. What I'm talking about 
is where client computers that are not fully trusted sit: running closed source 
Linux code (e.g., for DRM-ed movies) which might try probing nearby computers 
on the same LAN.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Long Term Support (LTS) program proposal

[Paul Kosinski via clamav-users]

LTS sounds like a great idea!

Recently, the bandwidth hogging episodes have resulted in rapid changes to 
ClamAV versions, followed by EOL of versions that many people (not including 
me) were still using. So recently I have had to spend far more time on updating 
ClamAV than updating anything else I use. And since I can't count on Debian (or 
even update-happy OpenSUSE) keeping up with these (now rapid) changes, I have 
always built ClamAV from source, ever since I started using it 16+ years ago. 

Thus, an LTS ClamAV which allowed me to build from source only a couple of 
times per year would be much appreciated. And I hope that CMAKE building can be 
sorted out before the next Update Now! emergency. 

(I don't see exactly how a LTS would have helped with the bandwidth issue, but 
I suppose it wouldn't have made it any more disruptive.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] can't cmake 1.0.4rc

[Paul Kosinski via clamav-users]

On Thu, 29 Jul 2021 08:52:57 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Maybe there's no need to worry about that.  I've seen cases where the
> build process looks for a shared object, finds a 32 bit version when
> it's building for 64 bit, and then complains that it doesn't exist.
> It does exist, but it's found the one for the wrong architecture and
> doesn't understand what it's found.  If this is the case here, it's a
> little disappointing (after the build-up we've had for cmake) that it
> will get it as badly around its neck as autotools.
> 
> Do you really need the 32-bit stuff?  Do you have mixed 32 bit and 64
> bit binaries on your system?  If so you're going to run into this kind
> of thing more or less randomly when you build anything and you might
> need to dig into it yourself a bit more.  If you don't need the mixed
> architectures you'd be better off without the 32 bit stuff in there.
> 
> You could try using the package manage to try to remove the 32 bit
> version of libpthread.  If it's needed by something it will tell you,
> and you can take a view on what to do abuot it.

=

Are the build tools that deficient? I have both 64 and 32-bit stuff on my 
Debian system, and the 'file' command is able to report what a shared object 
file is (e.g., see below). Maybe CMAKE does it better?

  /usr/lib/x86_64-linux-gnu/libasan.so.5.0.0: ELF 64-bit LSB shared object, 
x86-64, version 1 (SYSV), dynamically linked, 
BuildID[sha1]=3cf2e4b5261216f9a156ed5dc2953d8b6f98987d, stripped

  /usr/libx32/libasan.so.5.0.0: ELF 32-bit LSB shared object, x86-64, version 1 
(SYSV), dynamically linked, 
BuildID[sha1]=1289f4162c3de1fbe87d1f28ee3876ec8467ac2d, stripped

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [OT] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Wed, 28 Jul 2021 12:53:38 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> I'd recommend not using any big distro for your perimiter firewall.
> I use one of the purpose-built stripped-down firewall distributions.

"..our home firewall and gateway -- with iptables, multi-LAN routing (with 
local DNS), a bit of bridging, encrypted tunnels to elsewhere, etc."
I forgot to mention that it also logs to disk all Internet traffic, which is 
handy for occasional historical analysis of events via Wireshark. As far as 
being stripped down goes, the firewall/gatewaay has no X-windows stuff at all 
installed.

I think stripped-down distros are often too focused. And from what I've seen of 
some common firewalls, they're too simple-minded (e.g. firewalld), perhaps 
aimed at people who are terrified of the command line. (I personally found the 
CLI to be a great improvement over punched cards, just as the GUI is a 
wonderful improvement for many -- but not all -- tasks.) Also, Debian, being a 
major distro which is the basis for Ubuntu and others, has long been very 
reliable in providing security and bug fixes. How many smaller distros are as 
future-proof?

Finally, do any firewall distros address inter-LAN filtering? We have two major 
LANs, Black and Red. Black is the trusted LAN, while Red is for Internet TV 
etc. (on physically separate computers, of course). Red can access the Internet 
but is not allowed access to Black. Black has limited access to Red (for SSH, 
VNC and the like). Both are firewalled from the Internet (with Red a bit less 
so).


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [OT] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Wed, 28 Jul 2021 23:31:05 +1000
"Gary R. Schmidt"  wrote:

> I second what Ged is saying here, for firewalls and so on the Raspberry 
> Pi and its ilk are a much better choice than a full-on system, they use 
> /much/ less power, and keeping a spare or three isn't a board- (or 
> wife-) level budget request.  :-)

My current firewall, which also does inter-LAN routing with iptables filtering, 
has six (6) gigabit Ethernet ports on it (including one 4-port Intel card in a 
PCIe-x4 slot). Which model Raspberry Pi should I use?

P.S. I could make do with 5 ports, as my second WAN (a static IP, but slow, 
DSL) was discontinued in late 2019.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Wed, 28 Jul 2021 09:59:14 +0200
Matus UHLAR - fantomas  wrote:

> a bit OT, but I upgrade debian servers for years in a short steps, combining
> 
> "apt-get upgrade" so only safe packages are upgraded
> and manual upgrades a few at once via aptitude
> (so packages with complicated dependencies at the end, e.g. perl)
> 
> with configuration differences (updatedb; locate -e .dpkg- .ucf-) handled
> between those steps.
> 
> it takes a bit more time, but reduces outages.
> 
> Ubuntu can be handled similarly (however, even base ubuntu is uselessly
> bloated and has bit more complicated dependencies).



The question is, would you be willing to use the method you outlined if one bad 
package upgrade could kill your Internet access?

(Until you restored from a backup, at least.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Tue, 27 Jul 2021 16:41:03 +0100
Mark Fortescue via clamav-users  wrote:

> Hi Joel,
> 
> One quick answer to why people do not upgrade the OS is that the 
> hardware does not support the upgrade (mostly due to memory and x86_64).
> 
> I work with embedded systems where the code is very specific to the 
> hardware so new hardware is not an option.
> 
> For others it may just be the hassle of starting setting up a new OS and 
> fixing all the distribution bugs/annoyances that get installed with each 
> new OS all over again.
> 
> Regards
>   Mark.


In my case, I can't simply upgrade to the latest Debian (or any other distro), 
as one of the systems is our home firewall and gateway -- with iptables, 
multi-LAN routing (with local DNS), a bit of bridging, encrypted tunnels to 
elsewhere, etc. This means we would lose *all* Internet connectivity for who 
knows how long if I tried an in-place upgrade.

So the only way to move forward seems to be to rebuild our system on separate 
hardware. I have started this on hardware I already mainly have (being retired, 
and thus without corporate budget or staff). Then I plan get the new build more 
or less working, and hope that I don't have to move cables between the old and 
new (and change IP addresses back and forth) more than a few times, thereby 
only having a few short periods of time without Internet connectivity.

Finally, building this new system is made even more difficult by the fact that 
iptables has recently been replaced by nftables, whose native syntax has been 
"improved" to be quite different. There is, at least, a legacy iptables 
interface to it, and it may actually be behaviorally identical to the old 
iptables for my > 2000 custom rules (built up over a dozen years) which govern 
LAN[i] <=> LAN[j] and LAN[i] <=> Internet routing and firewalling.

P.S. The last time I upgraded our firewall, from x86 to x86_64, at least 
iptables was quite compatible with ipchains, and Linux as a whole was still in 
the early stages of its exponential growth in complexity.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Tue, 27 Jul 2021 15:30:05 +
"Joel Esler (jesler)"  wrote:

> You can’t support everything, forever.

When you are part of critical infrastructure -- as computers have become -- you 
must. (Well, not quite forever.)

Compare the rollout of IPv6 with the rollout of x86_64 (not to mention the 
rollout years ago of Area Codes and Direct Distance Dialing).

IPv6 is better than IPv4 in providing many more IP addresses, but it hasn't 
replaced it: it's incompatible. (Why didn't they simply add more bits, like 
int32 => int64?)

In CPUs, x86_64, which is backward compatible, has taken over, while its 
"replacement", Itanium is gone. (ARM is spreading, of course, but not because 
x86_64 is being dropped upon software upgrade; even Apple can't do that for a 
few years).

And imagine what would have happened 50 years ago if you had needed new 
telephones and a second telephone number to take advantage of DDD.

Of course ClamAV now belongs to Cisco, and there is no money cost to users, but 
the work cost to keep up-to-date has gotten much worse recently (mainly because 
of the response to bandwidth abuse), and some less dedicated users are probably 
giving up.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.104.0 Release Candidate is here!

[Paul Kosinski via clamav-users]

On Mon, 26 Jul 2021 11:35:29 -0400
"Rick Cooper"  wrote:

> And what, exactly, is the reason for moving to cmake? I am sure you know
> it's going to be problematic for thousands of people so I am curious what
> tremendous gain of speed, size, memory usage or seciurity the other users
> get from this change, or if it's just a convenience thing for the
> developers?


I get the impression that *all* recent software development (at least in Open 
Source) has given up any notion of backward compatibility. For example, Firefox 
(even ESR) has been a disaster in the past few years, changing the UI with 
every major release, once totally blowing away users' bookmarks, and of course, 
completely invalidating many, many years of add-on development by many people 
due to switching from XUL to the less powerful WebExtensions.

Now I wonder what will happen when I next try to build ClamAV on my three 
different Debian systems (7, 8 & 10).

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam updates problem

[Paul Kosinski via clamav-users]

On Wed, 14 Jul 2021 23:55:06 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> Hi Paul, all:
> 
> We're triaging this issue now, also reported by a user on Discord.
> 
> We issue a zero-byte CDIFF database patch file whenever we want Freshclam to 
> download a whole CVD instead of doing the incremental/patch update. 
> Today we published a zero-byte patch file for both daily and main, as we've 
> migrated about half of the daily signatures over to the main database.
> 
> Unfortunately it appears we have a bug in 0.102 and 0.103 where Freshclam 
> will complain loudly about the zero-byte CDIFF patch file. I suspect we 
> didn't notice it before because, despite the error message, Freshclam will 
> still do the right thing (at least in 0.102). 
> 
> But it seems 0.103 has a second bug where it will patiently wait until it's 
> at least 2 versions behind before it downloads the whole CVD database. This 
> behavior is supposed to happen when a private mirror doesn't have the latest 
> patch file yet, but wasn't supposed to happen for a zero-byte patch file. So 
> we clearly have 2 bugs to fix ASAP.
> 
> The good news is, I believe that if we publish 1 more version of daily.cvd 
> and main.cvd, we will see the 0.103 Freshclam clients all download the new 
> daily & main and they'll be good again. I will work with our signature 
> publishing team here to get an update for daily & main out as soon as we're 
> able.
> 
> I'm sorry about the bug everyone. Thank you for your patience!
> 
> I will write again a soon as I have an update.
> 
> -Micah


More details on the new 'main' misbehavior. Don't know if it is helpful.

I'm running 0.103.3 on all 3 of my ClamAV installations. On one of the systems 
(where I have very detailed logging of freshclam), main was still at 59 (today 
at 7:43 AM EDT), and freshclam correctly downloaded daily 26231 over 26230. 

Later on, from the same system, main was allegedly (DNS TXT) supposed to be at 
60, and daily was supposed to be at 26232, but *both* failed to download at 
least 7 times (15:43 EDT through 21:43).

This may, of course, be essentially the same problem I had with Cloudflare's 
BOS (Anycast) POP a couple of years ago, which made me give up putting any more 
work into setting up my own mirror.



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] New Main & Daily CVD's are incoming

[Paul Kosinski via clamav-users]

On Tue, 13 Jul 2021 14:05:53 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> Tomorrow, Wednesday July 14th, we are planning on publishing a brand new 
> main.cvd and daily.cvd, as we do periodically to move more of the long term 
> signatures into the main.cvd and make the daily.cvd smaller again.  
> 
> This will have an impact on your downloads of these files (as every ClamAV 
> instance will have to re-download both files), so you may see a spike in your 
> bandwidth usage.
> 
> We will monitor the situation on the mirror side and make any adjustments 
> necessary, but we anticipate no issues.
> 
> https://blog.clamav.net/2021/07/new-main-daily-cvds-are-incoming.html 
> 




I wondered when (and if) you would be able to distribute a new main.cvd, given 
your concerns about Cloudflare bandwidth usage. I assume this means that there 
is (almost) no one still downloading ClamAV updates every second or so.

I also presume that my IP address won't set off any alarms tomorrow by 
downloading 3 complete copies of main.cvd and daily.cvd (since I gave up trying 
to run my own mirror many months ago).

P.S. When I was wondering about when a new 'main' might be released, it 
occurred to me that perhaps the CDIFF mechanism could be used to gradually 
remove permanent signatures from 'daily' and add the same ones to 'main', 
thereby significantly reducing the peak bandwidth load. (Of course this still 
adds to the long term bandwidth consumption -- unless the CDIFF mechanism has a 
"move signature" option added.)

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Fw: openSUSE-SU-2021:2242-1: important: Security update for clamav-database

[Paul Kosinski via clamav-users]

Just FYI: this is the first time I remember seeing openSUSE notifying something 
about ClamAV.


Begin forwarded message:

Date: Mon,  5 Jul 2021 15:17:01 +0200 (CEST)
From: opensuse-secur...@opensuse.org
To: opensuse-security-annou...@opensuse.org
Subject: openSUSE-SU-2021:2242-1: important: Security update for clamav-database


   openSUSE Security Update: Security update for clamav-database
__

Announcement ID:openSUSE-SU-2021:2242-1
Rating: important
References: #1084929 
Affected Products:
openSUSE Leap 15.3
__

   An update that contains security fixes can now be installed.

Description:

   This update for clamav-database fixes the following issues:

   Changes in clamav-database:
   - database refresh on 2021-07-05 (bsc#1084929)


Patch Instructions:

   To install this openSUSE Security Update use the SUSE recommended 
installation methods
   like YaST online_update or "zypper patch".

   Alternatively you can run the command listed for your product:

   - openSUSE Leap 15.3:

  zypper in -t patch openSUSE-SLE-15.3-2021-2242=1



Package List:

   - openSUSE Leap 15.3 (noarch):

  clamav-database-202107050018-3.480.1


References:

   https://bugzilla.suse.com/1084929



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Help about Clamava on QNAP

[Paul Kosinski via clamav-users]

All these stories about QNAP (etc.) make me glad that I build my own servers, 
rather than getting some easy-to-setup, but non-upgradable, box. (E.g., I'm 
running 0.103.2, at the minor cost of having to build it from source.)


On Thu, 6 May 2021 13:18:20 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Thu, 6 May 2021, Matus UHLAR - fantomas wrote:
> > On 06.05.21 12:19, Chellini Stefano via clamav-users wrote:  
> >> My QNAP NAS It is EOL , it is TS419-PII
> >> 
> >> Is it available an option to upgrade the antivirus on it ?  
> >
> > it should be installable through entware package, but as it only has 512MB
> > of RAM, it's largely useless there (may not work properly).  
> 
> QNAP devices have been mentioned several times on this list recently.
> 
> A very little searching will reveal why.
> 
> There seems to be little doubt that the responses to the reports by
> researchers of critical vulnerabilities have left much to be desired:
> 
> https://securingsam.com/new-vulnerabilities-allow-complete-takeover/
> https://portswigger.net/daily-swig/qnap-fixes-critical-rce-vulnerabilities-in-nas-devices
> https://www.zdnet.com/article/hundreds-of-thousands-of-qnap-devices-vulnerable-to-remote-takeover-attacks/
> 
> If you own one of these devices, I guess that these blog posts make
> uncomfortable reading.
> 
> Even if it would be capable of running ClamAV, installing it on any
> vulnerable device would be pointless; this would not magically make
> the device any less vulnerable.  The vulnerabilities can only be fixed
> by security patches or upgrades, or perhaps by some serious hacking
> which is likely to be well beyond the average user.
> 
> My view is that given their dubious history, QNAP devices should be
> taken out of service unless they're in environments protected by
> people who *really* know what they're doing - people who can create a
> demonstrably safe firewall configuration.  Again well beyond average.
> 
> Otherwise, these things are just compromises waiting to happen.
> 
> They're powerful enough to be attractive targets.  They're easy enough
> to find.  Even when up to date with patches, next time around we'll
> probably see the same unsatisfactory response leave more low-hanging
> fruit for the criminals.  They represent risk not just to their users,
> but, after they're taken over for use as part of the extensive and
> ever-growing criminal infrastructure, to the rest of us as well.
> 
> Do us all a favour and get rid of them.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Help, we are still seeing issues

[Paul Kosinski via clamav-users]

You're comparing daily.CLD with main.CVD: as I understand it, CVDs are 
compressed, CLDs aren't.


On Sat, 17 Apr 2021 21:15:29 +0200 (CEST)
"Robert M. Stockmann via clamav-users"  wrote:

> Here's the freshclam virus data files which were first downloaded when
> i upgraded to 0.103.2 :
> 
>[hubble:stock]:(/var/lib/clamav)$ ll 
>total 429572
>-rw-r--r--  1 clamav clamav293670 Apr  8 02:37 bytecode.cvd
>-rw-r--r--  1 clamav clamav 321713152 Apr 17 14:07 daily.cld
>-rw-r--r--  1 clamav clamav 117859675 Apr  8 02:37 main.cvd
>-rw-r--r--  1 clamav clamav69 Apr  8 02:36 mirrors.dat
>[hubble:stock]:(/var/lib/clamav)$ clamdscan --version
>ClamAV 0.103.2/26143/Sat Apr 17 13:06:39 2021
>[hubble:stock]:(/var/lib/clamav)$ 
> 
> As you can see, the daily.cld is from today, Apr 17, and the others
> were downloaded on the day of upgrade. However one would expect the
> daily.cvd to be the smallest file, instead its the biggest
> with 307M in size. 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Heuristics.Broken.Media.JPEG.JFIFdupAppMarker

[Paul Kosinski via clamav-users]

It's worse than that. Not only do almost all users ignore security (as do many 
organizations), it seems that every new piece or version of software or 
hardware *reduces* security. And this applies to some new protocols (remember 
WiFi's WEP debacle?) and some extensions to or uses of existing ones.

It's all done in the name of convenience, and, in particular, striving for 
universal inter-connectedness. For example, I have fairly recently started 
receiving spam TXT messages containing links to who-knows-what. Yet (some) 
Samsung smart phones urge you to "share your contact list with others" -- and 
this admonition can't be disabled, it seems.

Another example is we got a new air-conditioner that has WiFi built-in, but 
it's only usable via the GE/Haier server, not locally (which might actually be 
useful). Luckily, the WiFi can be disabled (supposedly), so maybe this will 
stop it from being part of an IoT botnet, since its tiny computer likely can't 
get security updates.

In other words, securing email may be the least of our problems in the near 
future.


On Sat, 17 Apr 2021 13:14:40 +0100
Pedro Guedes via clamav-users  wrote:

> Hi again.
> 
> Well, the source ...
> .. you known users most of the time have no idea what are doing.
> Seems a usual correspondent but,  who knows.
> Since mail is responsible for 99% of malware and dirt and
> because users hate security, bad for day to day work the only solution
> is using clamav-milter whitelist addresses.
> Mail is a complete anarchy, no way blocking failed SPF, DKIM signatures,
> DMARC, etc
> Because no one does anything and so if you block a lot of important
> emails block.
> 
> Media all around talk about security but no one does nothing.
> Even most important banks don't even have DNSSEC and when they have is
> incorrect.
> 
> Dkim? To much trouble
> Dnssec? To much trouble.
> 
> And with all he monopoly on the clouds things get even worse

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Last ClamAV compatible with x32

[Paul Kosinski via clamav-users]

I have sometimes been able to find older RPMs for various system components at 
rpm.pbone.net, but it can be tedious.

On Mon, 12 Apr 2021 15:10:01 -0500
"J.R. via clamav-users"  wrote:

> > I've made some investigation and the people on google says that this
> > is a BUG with zlib, and the last zlib for RHEL 6.7 x32 fail to correctly
> > decompress the CVD signature database.
> >
> > A solution is to use a newer version of zlib but I'm not able to find a
> > newer version of zlib for this version of RHEL 6.7 x32. Newer than
> > the one for the RHEL repositories.
> >
> > Any advice?  
> 
> There was an EL6 clamav RPM that had included a newer version of zlib
> to build statically with it so it would work with the newer database
> files. Updating the spec and building your own RPM is not hard.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Scanning a large file through HTTP

[Paul Kosinski via clamav-users]

Seems to me that this behavior, advertising a 4GB limit while silently imposing 
a 2GB limit and reporting "OK" for anything in between, is a *major* security 
flaw: ClamAV *must* report that the file was too big to deal with (however 
worded).

Thus I've taken to using clamscan rather than clamdscan (slow though that is), 
because at least it reports how many bytes were read, and how many scanned, so 
I can see what's going on.

P.S. Recently I've downloaded some MP3s from Amazon and scanned them (as I do 
everything I download -- except updates from my Linux distros). But for a 
reason I saw on this list -- but can't remember -- MP3s are fully read, but not 
scanned. Is this going to be remedied?


On Wed, 7 Apr 2021 22:14:39 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> In reality, the file size limit is 2GB.  Anything larger than that will be 
> automatically skipped and marked as “OK”.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Private Mirror Via Artifactory

[Paul Kosinski via clamav-users]

On Fri, 12 Mar 2021 15:47:02 + (GMT)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Fri, 12 Mar 2021, Arjen de Korte via clamav-users wrote:
> 
> > Citeren "G.W. Haywood via clamav-users" :
> >  
> >> I think the OP was saying that he's not allowed to do that. ...  
> >
> > I see no reason why. ...  
> 
> Nor do I.  But he said it was for the government, which says to me
> that rational argument will have precious little to do with it. :/


The same applies to many organizations that are big enough -- and have
been established long enough -- to have Policies.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Unable to download clamav cvd file using google cloud python function

[Paul Kosinski via clamav-users]

"I can’t play wack-a-mole with single IPs or even whole ASNs."

Does Cloudflare have the iptables hashlimit filter (or the equivalent) 
available?



On Wed, 10 Mar 2021 22:29:41 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> To give everyone a frame of reference. This is what a Cdiff release and 
> download cycle should look like:
> 
> 
> [cid:311D041A-A699-48A6-BB74-8523A3927866]
> 
> Big influx right in the morning when we publish, and then peaks on the top 
> and bottom of the hour every hour throughout a 24 hour period, (people having 
> a cron job that runs at the top of every hour throughout the day) 
> Theoretically speaking, at the end of 24 hours, the line should go to zero, 
> it never will, because of new installs that download a bunch of cdiffs right 
> in a row and things like that.  But I I look between the peaks find people 
> like this:
> 
> [cid:B0884332-310A-4C6F-9960-A0A8DB6C2B0D]
> 
> 100 CDIFFs or so behind, and they download it nearly 2k times in a row?  Why? 
>  This is not a partial download either.  It’s the full file.  Stuck cron?
> 
> Or this single IP:
> 
> [cid:AE797960-535D-44D1-AB4F-7C5823B5BBF2]
> 
> Who in the past 24 hours has created 22.17M file downloads all by themselves 
> from a single IP. (The main.cvd btw)
> 
> It’s these bad apples that have ruined the basket for everyone.  I can’t play 
> wack-a-mole with single IPs or even whole ASNs.
> 
> Multiply this one IP above x thousands, and you see the volume I am dealing 
> with.  But that graph at the top there is from yesterday, and it’s much 
> better.  This is what we are aiming for.  We’ve reduced transferred data by 
> 60% by cutting back on abusers.
> 
> Like I said, I’ll be writing a blog post about this, but just to show you 
> guys what I am dealing with:
> 
> [cid:D66E6145-0352-45EA-8579-5353C85C15F1]
> 
> In the past 72 hours, this is what our event graphs look like.  Big drop offs 
> and increases are attributed to the constant adjustment I am doing to find 
> the right balance.
> 
> --
> Joel Esler
> Manager, Communities Division
> Cisco Talos Intelligence Group
> http://www.talosintelligence.com | https://www.snort.org
> 
> On Mar 10, 2021, at 3:30 PM, Joel Esler (jesler) via clamav-users 
> mailto:clamav-users@lists.clamav.net>> wrote:
> 
> 
> 
> On Mar 10, 2021, at 12:31 PM, Paul Smith via clamav-users 
> mailto:clamav-users@lists.clamav.net>> wrote:
> 
> On 10/03/2021 17:00, Paul Kosinski via clamav-users wrote:
> I wonder how many "ordinary" users of ClamAV are giving up on using it after 
> getting permanent 403s. I would imagine there are lots of people who don't 
> pursue the issue. They may even tell others that ClamAV is unreliable (which 
> would tarnish its reputation).
> 
> Indeed. There does seem to be a view from some people here that anyone using 
> ClamAV should be regularly updating, monitoring this list, monitoring blogs, 
> etc. Ordinary people just don't do that.
> 
> I expect many will just be thinking that the database servers are broken, and 
> are waiting for them to recover on their own (as they've done in the past) 
> and they'll eventually go elsewhere.
> 
> The change should really be published everywhere possible - at least in big 
> letters on the ClamAV home page, and possibly including going to popular 
> computer press, etc.
> 
> A blog article (which is actually very hard to find) or announcement list 
> post (which is even harder to find) which vaguely says that databases won't 
> be tested on older versions isn't quite the same as a home page announcement 
> that old versions & wget just won't work any more!
> 
> Of course, people have limited rights to complain - it's not like we're 
> paying for it.
> 
> We are going to be writing a couple blog posts in the coming days.  I haven’t 
> had the time to sit down and do it.
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] looks like I have a problem too

[Paul Kosinski via clamav-users]

I wrote a little script that run off cron every hour or so. But it *only* 
invokes freshclam after querying ClamAV's DNS TXT record to see if any 
advertised versions of 'daily', 'bytecode' or 'main' are newer than the local 
versions of the CVD files, as determined by 'head', not the files' timestamps. 
Only if something newer is supposed to be available (Cloudflare BOS mirror had 
some problems in the past), does it invoke freshclam. (It also keeps backup 
CVD/CLD files just in case.)

This provides a very low bandwidth way of getting ClamAV updates ASAP. (I even 
cron my different ClamAV machines at different odd times during the day.)

If anyone is interested, I could provide my scripts: 'getfreshclam' (Bash) 
which uses 'testclam-dns' (Perl).



On Wed, 10 Mar 2021 12:25:18 -0500
Gene Heskett via clamav-users  wrote:

> Greetings;
> 
> I just reduced my freshclam fetch from 24 to 6 times a day.  But I do see 
> some errors when it does try to update:
> copy/paste, wordwrap off:
> from freshclam.log:
> 
> Wed Mar 10 08:09:24 2021 -> Received signal: wake up
> Wed Mar 10 08:09:24 2021 -> ClamAV update process started at Wed Mar 10 
> 08:09:24 2021
> Wed Mar 10 08:09:24 2021 -> daily database available for update (local 
> version: 26103, remote version: 26104)
> Wed Mar 10 08:09:24 2021 -> Testing 
> database: 
> '/var/lib/clamav/tmp.9230f/clamav-e29c7bfba68291a21e41dbe83fb8c776.tmp-daily.cld'
>  ...
> Wed Mar 10 08:09:29 2021 -> WARNING: [LibClamAV] cli_tgzload: Invalid 
> checksum for file daily.hsb
> Wed Mar 10 08:09:29 2021 -> WARNING: [LibClamAV] Can't 
> load 
> /var/lib/clamav/tmp.9230f/clamav-e29c7bfba68291a21e41dbe83fb8c776.tmp-daily.cld:
>  Malformed database
> Wed Mar 10 08:09:29 2021 -> ERROR: Failed to load new database: Malformed 
> database
> Wed Mar 10 08:09:29 2021 -> Database test passed.
> Wed Mar 10 08:09:31 2021 -> daily.cld updated (version: 26104, sigs: 3958880, 
> f-level: 63, builder: raynman)
> Wed Mar 10 08:09:31 2021 -> main.cld database is up to date (version: 59, 
> sigs: 4564902, f-level: 60, builder: sigmgr)
> Wed Mar 10 08:09:31 2021 -> bytecode.cld database is up to date (version: 
> 333, sigs: 92, f-level: 63, builder: awillia2)
> Wed Mar 10 08:09:31 2021 -> WARNING: Clamd was NOT notified: Can't connect to 
> clamd through /var/run/clamav/clamd.ctl: No such 
> file or directory
> 
> So obviously something is aglay with my config which I haven't touched 
> since debian stretch was installed. But it has been kept uptodate at 
> least weekly. synaptic says I have version 102-4.
> 
> What should I fix?
> 
> Thanks folks.
> 
> Cheers, Gene Heskett

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Unable to download clamav cvd file using google cloud python function

[Paul Kosinski via clamav-users]

I wonder how many "ordinary" users of ClamAV are giving up on using it after 
getting permanent 403s. I would imagine there are lots of people who don't 
pursue the issue. They may even tell others that ClamAV is unreliable (which 
would tarnish its reputation).


On Wed, 10 Mar 2021 11:58:13 + (GMT)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Wed, 10 Mar 2021, Kachare, Ganesh, Vodafone Group (External) via 
> clamav-users wrote:
> 
> > I am getting error  http client 403 ...  
> 
> I suspect that some people are subscribing to the mailing list when
> they belatedly find that the non-preferred way of getting the ClamAV
> signatures that they've been using has suddenly stopped working.
> 
> Is there not an argument for (at least temporarily) putting something
> in the mailing list manager's 'welcome' message which asks them to
> read some of Joel's recent messages?
> 
> Not only could it prevent a lot of unnecessary noise on the mailing
> list, it could also help people to avoid looking dumb - and that's
> hardly ever a bad thing.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam network unreachable

[Paul Kosinski via clamav-users]

"Out of procedural curiosity, why would someone want to disable ipv6?"

Although our FIOS connection supports IPv6, our firewall/gateway complex, which 
I custom built from scratch 16+ years ago using iptables etc., doesn't. Since 
this firewall/gateway also does lots of inter-LAN routing and blocking (not to 
mention some source-based iproute2 stuff), it would have to be rewritten 
extensively. I don't have time to do this, especially given that there is still 
(after all these years) nothing critical that is IPv6 only.

P.S. It would have been nice if the designers of IPv6 hadn't made it almost 
totally incompatible with IPv4 (unlike x64 vs x86). What if, when Ma Bell 
introduced direct distance dialing in the 1960s, they had made the new 
area-code scheme require that every customer who wanted to use area-codes get a 
new phone number with a totally different format, and replace their telephone 
handset?


On Tue, 9 Mar 2021 14:37:59 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> Out of procedural curiosity, why would someone want to disable ipv6?
> 
> > On Mar 8, 2021, at 6:40 PM, G.W. Haywood via clamav-users 
> >  wrote:
> > 
> > Hi there,
> > 
> > On Mon, 8 Mar 2021, Adam Bashore via clamav-users wrote:
> >   
> >> I'm able to telnet to port 80 at db.local.clamav.net without issue. but I
> >> get a 403 forbidden when i try to download main.clv directly with wget 
> >> (wget
> >> http://db.local.clamav.net/main.cvd)  
> > 
> > There's been a flurry of recent activity on the mailing list about the
> > abuse of ClamAV DB service, see the archives for more detail but I
> > think Joel's reply has answered this part.
> >   
> >> I'm not convinced that it's a network issue. Can anyone explain why
> >> freshclam appears to be trying IPv6 even though the host only has an IPv4
> >> address on eth1?  
> > 
> > I think it is a network issue.  Most network software doesn't know
> > what interface it's going to use, it just asks the resolver for an
> > address.  Your resolver provides an IPv6 address and freshclam tries
> > to use it.
> > 
> > To build freshclam (and everything else) from source without IPv6
> > support you could (at least theoretically, I've never tried it myself)
> > use the 'configure' option '--disable-ipv6'.  Alternatively, which I'd
> > suggest is preferable, you can fix the network's IPv6 connectivity.
> > 
> > -- 
> > 
> > 73,
> > Ged.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV not even mentioned in article "The 6 Best Antiviruses for Linux 2021"

[Paul Kosinski via clamav-users]

Or maybe, since ClamAV is free (as well as Free), they don't get any commission 
on a sale.

(E.g., their "Get Kaspersky Now" button goes to 
"https://www.safetydetectives.com/go/vendor/linux/202/?post_id=9609_btn_name=Visit+Button+-+8435;.)


On Fri, 19 Feb 2021 21:02:08 +
"Joel Esler (jesler)"  wrote:

> This is what happens when you don’t pay people for SEO.  
> 
> Sent from my  iPhone
> 
> > On Feb 19, 2021, at 12:10, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > https://www.safetydetectives.com/best-antivirus/linux/

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] ClamAV not even mentioned in article "The 6 Best Antiviruses for Linux 2021"

[Paul Kosinski via clamav-users]

https://www.safetydetectives.com/best-antivirus/linux/

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] What are all the tmp.xyzuvwpqrs subdirs that keep accumulating

[Paul Kosinski via clamav-users]

For ClamAV 0.103.0:
  root@ime1:~# grep -i temporary /opt/clamav.d/clamav.0.103.0/etc/clamd.conf
  # Optional path to the global temporary directory.
  TemporaryDirectory /var/clamav/tmp
  # Do not remove temporary files (for debug purposes).
  LeaveTemporaryFiles 0

For ClamAV 0.102.1 it was the same:
  root@ime1:~# grep -i temporary /opt/clamav.d/clamav.0.102.2/etc/clamd.conf
  # Optional path to the global temporary directory.
  TemporaryDirectory /var/clamav/tmp
  # Do not remove temporary files (for debug purposes).
  LeaveTemporaryFiles 0

But the subdirs are in my "/opt/clamav.d/clamav.0.103.0/share/clamav/" 
directory. (I install each new version under opt, "just in case".)

And there's no "temporary". "tmp" or "temp" (except in the word "attempt") in 
my freshclam.conf file.





On Thu, 11 Feb 2021 23:52:37 + (GMT)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Thu, 11 Feb 2021, Paul Kosinski via clamav-users wrote:
> 
> > in my clamav.0.103.0/share/clamav/ directory?
> >
> > They don't seem to have been there with clamav.0.102.0 and earlier.  
> 
> What's the output of
> 
> grep -i temporary clamd.conf
> 
> ?
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] What are all the tmp.xyzuvwpqrs subdirs that keep accumulating

[Paul Kosinski via clamav-users]

in my clamav.0.103.0/share/clamav/ directory?

They don't seem to have been there with clamav.0.102.0 and earlier.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [When was 0.103.1 announced on *this* list?

[Paul Kosinski via clamav-users]

Thanks.

It's good to know that my mail filtering isn't misbehaving (and that no
"blackhat" is subtly attacking ClamAV).


On Tue, 9 Feb 2021 18:37:26 +
"Joel Esler (jesler)"  wrote:

> I forgot to announce it.  Sorry about that.
> 
> — 
> Sent from my  iPad
> 
> > On Feb 9, 2021, at 10:14, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > I save all the ClamAV mail, and couldn't find an announcement. 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] When was 0.103.1 announced on *this* list?

[Paul Kosinski via clamav-users]

I save all the ClamAV mail, and couldn't find an announcement. 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Terminate clamscan after specific time

[Paul Kosinski via clamav-users]

The problem with only scanning files that have changed since they were
last scanned is that there usually have been virus signature updates in
the meantime. So you could have an "old" file that contains what was a
zero-day virus at the time it was scanned, and now there is a signature
that would detect it.


On Wed, 06 Jan 2021 11:56:47 +0100
"Pierre Dehaen"  wrote:

> Hi,
> 
> On 6 Jan 2021 at 9:58, G.W. Haywood via clamav-users wrote:
> 
> > > My goal is to terminate scan of big number of files like '/' on CPU busy 
> > > hours.  
> > Do not scan everything under the root directory.  
> 
> Use zfs, make regular snapshots, scan once, then use zfs diff to find the 
> new/changed(/removed) files, scan these only.
> 
> Or make a full scan every week if desired, then use a auditing program to 
> regularly search for 
> the files that were added/updated(/removed), scan these only. These auditing 
> programs use 
> hash signatures which are faster to compute than doing full virus scans, but 
> they will anyway 
> make a lot of i/o as they will read all files. If you are really constrained 
> by the i/o you could run 
> a less secure but lighter audit based on the file attributes (size, 
> ownership, mode, dates...) 
> and once a day/week a full audit...
> 
> There are many options...
> 
> HTH,
> Pierre

> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Is there anything to do about encrypted viruses?

[Paul Kosinski via clamav-users]

Since the password has to be included for the victim to be able to
decrypt, it ought to be possible to automatically find the password in
the email. Of course, eventually the criminals will start hiding the
password in some way that a human can easily find it, but non-AI
automation can't.


On Tue, 22 Dec 2020 03:46:13 -0800
Al Varnell via clamav-users  wrote:

> When you submit it, be sure to include the password so that the ClamAV 
> signature team can properly asses it and provide a hash signature for the zip 
> file.
> 
> -Al-
> 
> > On Dec 22, 2020, at 03:32, Alessandro Vesely via clamav-users 
> >  wrote:
> > 
> > Hi all,
> > 
> > 
> > today I received a message with an encrypted zip attachment.  I saved the 
> > attachment and loaded it to VirusTotal, where no scanner detected anything:
> > https://www.virustotal.com/gui/file/2cef2c979e60c1e2892e6a494814dd65db14c2076102279e6e74737d36c115a5/detection
> > 
> > Then I unzipped the file using the password given in the message text, 
> > uploaded the only extracted file and got plenty of VBA / W97M malware:
> > https://www.virustotal.com/gui/file/99b352442e1351334d5e68e7f12469dc7f2790e6ae44b05be7dcd03739211f1f/detection
> > 
> > I spare reporting this malware to ClamAV, as it seems hopeless to me.  Am I 
> > wrong?
> > 
> > 
> > Best
> > Ale  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] local server takes time to update clamav db

[Paul Kosinski via clamav-users]

I did a quick grep on the the source code (and compiled output too) of
ClamAV 0.103.0, and I couldn't find any instance of 'CF-Cache-Status'.
Should freshclam (or somebody) be checking this HTTP header line that
Cloudflare returns? The 'STALE' and 'UPDATING' values sound like they
might be particularly relevant.


On Mon, 14 Dec 2020 02:57:48 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> Both of those things are done as well.  
> 
> Sent from my  iPhone
> 
> > On Dec 13, 2020, at 19:24, Dave Warren via clamav-users 
> >  wrote:
> > 
> > On 2020-12-11 08:51, Paul Kosinski via clamav-users wrote:  
> >> "The whole CVD filename is not versioned (always "daily.cvd") which is
> >> why the CloudFlare caching issue may result in serving the previous
> >> version."
> >> HTML filenames for Web pages are not versioned either. Does this mean
> >> that CDNs like Cloudflare often serve up obsolete Web pages? If so, does
> >> nobody notice (and complain)?
> >> A delay of an hour could have an adverse effect on online commerce,
> >> especially during the busy holiday season.  
> > 
> > By default Cloudflare does not cache HTML. Cloudflare also respects 
> > cache-control headers, which is the normal mechanism used for websites 
> > which want caching, but only to a point.
> > 
> > Cloudflare also has an API to clear the cache (at least by URI, or 
> > everything, and possibly more depending on the particular options offered 
> > by your plan). But in practice clearing the cache is not completely 
> > reliable and seems to be intended for cases where it is strictly needed and 
> > not for every "I updated this file" situation. I have the impression that 
> > this applies when using Cloudflare's tiered caching, my idle speculation 
> > wonders if perhaps this is a timing issue, where server #1 clears the 
> > cache, processes a request for the file which it obtains from server #2 all 
> > before server #2 clears the file from cache and then processes a request by 
> > pulling it from server #1.
> > 
> > From a ClamAV perspective, one solution to solve this would be to call 
> > daily.cvd?version=26013 -- Note that the underlying web server could ignore 
> > the version parameter completely, but this would ensure that each 
> > Cloudflare cache retrieves a fresh version of the file and negates the need 
> > to push a cache clear message at all. If ClamAV's server serves an outdated 
> > version of the file then it would still get cached, but this would defeat 
> > any caching within Cloudflare for new versions as they're released.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] PR: Removing PidFile

[Paul Kosinski via clamav-users]

I agree. I don't run ClamAV from systemd, and I wouldn't be pleased to
have to spend time changing my scripts "just because".

P.S. I do run other things from systemd -- if the OS set them up that
way -- and I do appreciate the parallelism: it saves a few minutes of
start-up time when I reboot every month or twelve. Now if only the
systemd config files weren't scattered among so many directories (with
symlinks thrown in for good measure).


On Sat, 12 Dec 2020 11:26:13 + (GMT)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Sat, 12 Dec 2020, Hanspeter Gosteli via clamav-users wrote:
> 
> > I propose to remove PidFile-Handling from ClamAV and handing it to the init
> > system as it seems already to be the case in most deployments. Please have
> > a look at the following PR and let me know if you oppose or need support in
> > integrating your stack.  
> 
> Please don't ask me to try to use Github.
> 
> Please don't break ClamAV for any of its existing users for cosmetic,
> religious or political reasons.
> 
> Please don't change parts of ClamAV which now work fine when there are
> parts which don't now work properly, or indeed in some cases, at all.
> 
> You have seen the ClamaV Bugzilla.  There are plenty of more important
> things to get your teeth into.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] local server takes time to update clamav db

[Paul Kosinski via clamav-users]

"The whole CVD filename is not versioned (always "daily.cvd") which is
why the CloudFlare caching issue may result in serving the previous
version."

HTML filenames for Web pages are not versioned either. Does this mean
that CDNs like Cloudflare often serve up obsolete Web pages? If so, does
nobody notice (and complain)?

A delay of an hour could have an adverse effect on online commerce,
especially during the busy holiday season.


On Thu, 10 Dec 2020 18:34:36 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> Ged, Joel, Andrew, Paul:
> 
> Ged wrote:
> > As I said earlier to the OP, I've never seen the problem that he's 
> > complaining of and I'm beginning to suspect that he's right - that it's the 
> > use of the `ScriptedUpdates no` option which is at the root of the problem. 
> >
> 
> This is correct -- there is no issue getting the latest patch when using 
> scripted updates.  The issue is when trying to download the whole CVD.  The 
> whole CVD filename is not versioned (always "daily.cvd") which is why  the 
> CloudFlare caching issue may result in serving the previous version.  
> 
> Andrew wrote:
> > Would it be sensible for freshclam to update the file when a newer version 
> > is available, even if it is not the newest ?
> > ...
> > To be clearer, say I have version 26011, the DNS says 26013 is current but 
> > the newest that freshclam can find on any configured mirror is 26012, it 
> > might be better to update to 26012 than wait for 26013.  
> 
> It should already do this.  If you have version 26011 and it says 26013, but 
> only 26012 is available, it should get 26012.  If that's not working -- let 
> me know, we'd have a bug to fix.
> 
> Joel wrote:
> > I think the way to fix this is, freshclam, if it receives an "I'm behind" 
> > error from the PoP, to do a sleep for awhile and then try again.  If the 
> > second attempt still fails then give the error to the user.  
> 
> I want to be clear -- the message that was originally reported is not an 
> error message. It's a verbose (a.k.a debug-level) message.  If you're running 
> freshclam relatively frequently, then this "wait a while and try again" thing 
> is transparent to you.  Disable the `Verbose` option in freshclam.conf and 
> don't worry about it.
> 
> -Micah
> 
> > -Original Message-
> > From: clamav-users  On Behalf Of
> > G.W. Haywood via clamav-users
> > Sent: Thursday, December 10, 2020 9:21 AM
> > To: Joel Esler (jesler) via clamav-users 
> > Cc: G.W. Haywood 
> > Subject: Re: [clamav-users] local server takes time to update clamav db
> > 
> > Hi there,
> > 
> > On Thu, 10 Dec 2020, Joel Esler (jesler) via clamav-users wrote:
> >   
> > >>> I think the way to fix this is, freshclam, if it receives an "I'm
> > >>> behind" error from the PoP, to do a sleep for awhile and then try
> > >>> again. ...  
> > 
> > Maybe the workaround is simpler than that.
> > 
> > The document at
> > 
> > https://www.clamav.net/documents/private-local-mirrors
> > 
> > tells the reader to set the 'ScriptedUpdates' option to 'no' for _both_ the 
> > local
> > mirror _and_ that mirror's clients.
> > 
> > I can understand the logic of setting the option to 'no' for clients of the 
> > local
> > mirror, because a local mirror won't serve '.cdiff' files and if they ask 
> > the local
> > mirror for such a file they'll get a 404.
> > 
> > But the local mirror could grab the .cdiff files from the Cloudflare mirrors
> > using freshclam, just as does any client which does _not_ use a local 
> > mirror,
> > no?
> > 
> > What reason is there for not using 'ScriptedUpdates yes' on the mirror?
> > 
> > As I said earlier to the OP, I've never seen the problem that he's 
> > complaining
> > of and I'm beginning to suspect that he's right - that it's the use of the
> > 
> > ScriptedUpdates no
> > 
> > option which is at the root of the problem.  (Well, that and the fact that
> > Cloudflare apparently isn't providing the service that Cisco has presumably
> > contracted it to provide - if all that's necessary in order for the 
> > Cloudflare PoP
> > to update its copy of the .cvd file is for some random client to request a
> > download of it, then you'd expect that the OP's request would trigger that,
> > and apparently it doesn't).
> > 
> > Most freshclam daemons will be configured to make just a few attempts per
> > day to update, and a failure will mean using outdated databases (on a server
> > which by definition is providing service to many clients) until at least 
> > the time
> > of the next scheduled update.  That and the "try again in an hour or two"
> > suggestion seem to fly in the face of the freshclam man page:
> > 
> > --on-error-execute=COMMAND  Execute COMMAND if error occurred.
> >   Remember, that virus database freshness is the most important thing in
> >   anti-virus system. ...
> > 
> > I wonder if another workaround might be to use the 'DatabaseMirror' or
> > 'PrivateMirror' options in freshclam.conf to avoid 

Re: [clamav-users] local server takes time to update clamav db

[Paul Kosinski via clamav-users]

Does ClamAV (Talos?) check *all* the Cloudflare anycast servers?
I thought it could only check those "near" to ClamAV POPs.


On Thu, 10 Dec 2020 18:00:15 +
"Joel Esler (jesler)"  wrote:

> > On Dec 10, 2020, at 11:58 AM, Paul Kosinski via clamav-users 
> >  wrote:
> > 
> > I would imagine that Cloudflare has a means of fetching a specific file
> > from any of their own mirror servers (via its unique, non-anycast, IP
> > address) to check its operation. If ClamAV DB files could be requested
> > from specific (i.e., all) CF servers, it would cause them to be pulled
> > from ClamAV's master server(s).  
> 
> 
> We already do this.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] local server takes time to update clamav db

[Paul Kosinski via clamav-users]

With regard to "sleep for awhile".

I remember that Cloudflare's BOS server on occasion remained behind the
latest CVD version (according to the DNS TXT record) for more than one
hour!

Might the following be possible instead?

I would imagine that Cloudflare has a means of fetching a specific file
from any of their own mirror servers (via its unique, non-anycast, IP
address) to check its operation. If ClamAV DB files could be requested
from specific (i.e., all) CF servers, it would cause them to be pulled
from ClamAV's master server(s).

Is this something CF could do for ClamAV? AV software helps improve
Internet security, which seems to be part of CF's mission.


On Thu, 10 Dec 2020 13:54:22 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> I think the way to fix this is, freshclam, if it receives an “I’m
> behind” error from the PoP, to do a sleep for awhile and then try
> again.  If the second attempt still fails then give the error to the
> user.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] local server takes time to update clamav db

[Paul Kosinski via clamav-users]

"This is one of the IPs which I was expecting to see.  I wouldn't
expect any problems with it, our ClamAV server updated from it at
1818 GMT last night."

Unfortunately, given the way Cloudflare works, the IP address
(e.g., 104.16.218.84) isn't the whole story. A particular Anycast IP
address such as this will route to the "nearest" server for that IP
address, and different servers may behave differently.

The HTTP(S) response header indicates which of the Cloudflare
servers the IP address actually routed to, for example:

  CF-RAY: 433942cde659ae1a-BOS

But I think you have to pretend you are ClamAV, or the server rejects
you, as in:

  User-Agent: ClamAV/0.103.0 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)

(At least this is the way it was in 2018.)

In the summer of 2018 (just after ClamAV started using Cloudflare) we
were having trouble in that our local BOS server was often behind the
latest ClamAV CVD file which was advertised by the DNS TXT record. I
finally gave up trying to have a local mirror for CVD files, and just
changed all our ClamAV machines to use the "scripted update" (CDIFF)
method individually. There are so few machines that it turned out to
*save* bandwidth in practice. 

P.S. There are a lot of emails about this in the ClamAV list for July
2018 et seq with subject lines: "We STILL cannot reliably get virus
updates (since new mirrors)".





On Wed, 9 Dec 2020 11:12:28 + (GMT)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Wed, 9 Dec 2020, Gal Cohen wrote:
> 
> > 5. here are the full logs of the latest update failure (26011 -> 
> > 26012),freshclam run takes 19 sec
> > Tue Dec  8 22:00:02 2020 -> ClamAV update process started at Tue Dec  8 
> > 22:00:02 2020
> > ...
> > Tue Dec  8 22:00:02 2020 -> *check_for_new_database_version: Local copy of 
> > daily found: daily.cvd.
> > Tue Dec  8 22:00:02 2020 -> *query_remote_database_version: daily.cvd 
> > version from DNS: 26012
> > Tue Dec  8 22:00:02 2020 -> daily database available for update (local 
> > version: 26011, remote version: 26012)
> > Tue Dec  8 22:00:02 2020 -> *Retrieving 
> > https://database.clamav.net/daily.cvd
> > Tue Dec  8 22:00:02 2020 -> *downloadFile: Download source: 
> > https://database.clamav.net/daily.cvd
> > Tue Dec  8 22:00:02 2020 -> *downloadFile: Download destination: 
> > /data/tmp.7624b/clamav-cde3734f56b3b9351a0261c3b140966f.tmp
> > *   Trying 104.16.218.84:443...  
> 
> This is one of the IPs which I was expecting to see.  I wouldn't expect any
> problems with it, our ClamAV server updated from it at 1818 GMT last night.
> 
> Maybe you have a proxy between you and the Cloudflare servers which is caching
> the data downloads?  Try downloading the 'daily' file with 'wget' from several
> different places and check which versions you receive.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

Sorry, forgot to CC the list (sometimes Reply All *is* appropriate).



I think it's important that ClamAV *not* say there were 0.00 MB
scanned/read, while also saying a file *was* scanned & infected.
Contradictions like this [might be interpreted to] suggest a serious
problem in the logic.

Rounding 68 bytes to 0.02 MB also would suggest a bug of some sort.
Perhaps it would be best to be explicit and simply report 16 kb blocks
read and scanned, rather than megabytes read and scanned. Maybe there
is a better word than "blocks" (which suggests disk blocks), such as
"segments", "sequences", "pieces"?


On Wed, 4 Nov 2020 17:49:09 +
"Micah Snyder (micasnyd)"  wrote:

> Do you reckon folks will be less confused if it rounds up?
> 
> -Micah
> 
> On 11/3/20, 1:37 PM, "clamav-users on behalf of Paul Kosinski via 
> clamav-users"  clamav-users@lists.clamav.net> wrote:
> 
> If ClamAV always rounded up when counting the number of 16kb blocks,
> then it should be counting at least 0.016384 MB (or 0.015625 MiB) for
> tiny files. By normal rounding rules this should display as 0.02 MB/MiB.
> 
> 
> On Tue, 3 Nov 2020 17:50:18 +
> Mark Fortescue via clamav-users  wrote:
> 
> > Hi all,
> > 
> > I would call this a bug. Scanning 1 byte is the same as scanning 1 
> block.
> > 
> > When storing things in blocks is is always important to round up or you 
> > get a false impression of reality.
> > 
> > You can't store 100 bytes in 0 disk sectors of 128 bytes. It is always 
> 1 
> > disk sector.
> > 
> > Can you not just round up by adding (BlockSize - 1) bytes when setting 
> > the block variables ?
> > 
> > Regards
> > Mark.
> > 
> > On 03/11/2020 16:07, Paul Kosinski via clamav-users wrote:  
> > > "This is a display problem, not a storage problem."
> > > 
> > > I disagree. When the counts in info.blocks and info.rblocks are counts
> > > of 16kb *blocks*, keeping precise track of the reading and scanning of
> > > small files is impossible, no matter how clever the display code is.
> > > 
> > > 
> > > 
> > > On Tue, 3 Nov 2020 17:44:18 +1100
> > > "Gary R. Schmidt"  wrote:
> > > 
> > >> On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote:
> > >>> "(don't you love C?)"
> > >>>
> > >>> I have never understood why the originators of C didn't give 
> integers
> > >>> explicit widths in bits: their scheme made C code often 
> non-portable.
> > >>>
> > >> Because C is intended to be very, very close to the machine
> > >> architecture, only a step or tow above assembler, or doing the
> > >> bit-twiddling by hand.
> > >>
> > >>> When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 
> bits
> > >>> while longs were 64 (unlike "standard" C). This made Alpha C code 
> not
> > >>> portable to lesser CPUs. On the other hand, when I wrote C on DOS 
> for
> > >>> the IBM PC in the late 1980s, ints were only 8 bits! It took some 
> time
> > >>> to figure out why my C-compliant code failed so badly. In spite of 
> all
> > >>> that, having started programming before C was invented, I can safely
> > >>> say that C is better than its predecessors for software like ClamAV.
> > >>>
> > >> Uh, not a good example, I've written C code that is still in use on
> > >> everything from 80286s (yes, Virginia, there are people who keep them
> > >> alive, not just because they're cheap, sometimes just because they
> > >> *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a
> > >> matter of knowing what you are doing, and sticking to it...
> > >>
> > >>> P.S. Good code these days tends to use typedefs defining things like
> > >>> int32, uint64 etc. A shame the original ClamAV coders didn't do 
> that.
> > >>>
> > >> And none of this has *anything* to do with the original problem - 
> seeing
> > >> 0 when the value is 0.01, or so.
> > >>
> > >> This is a display problem, not a storage problem.  You could declare
> > >> something as PIC(999.99) and you will still only see > 0
> > >> if you told it to display two decimal places.
> > >>
> > >>  Cheers,
> > >>  GaryB-)
> >  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

If ClamAV always rounded up when counting the number of 16kb blocks,
then it should be counting at least 0.016384 MB (or 0.015625 MiB) for
tiny files. By normal rounding rules this should display as 0.02 MB/MiB.


On Tue, 3 Nov 2020 17:50:18 +
Mark Fortescue via clamav-users  wrote:

> Hi all,
> 
> I would call this a bug. Scanning 1 byte is the same as scanning 1 block.
> 
> When storing things in blocks is is always important to round up or you 
> get a false impression of reality.
> 
> You can't store 100 bytes in 0 disk sectors of 128 bytes. It is always 1 
> disk sector.
> 
> Can you not just round up by adding (BlockSize - 1) bytes when setting 
> the block variables ?
> 
> Regards
>   Mark.
> 
> On 03/11/2020 16:07, Paul Kosinski via clamav-users wrote:
> > "This is a display problem, not a storage problem."
> > 
> > I disagree. When the counts in info.blocks and info.rblocks are counts
> > of 16kb *blocks*, keeping precise track of the reading and scanning of
> > small files is impossible, no matter how clever the display code is.
> > 
> > 
> > 
> > On Tue, 3 Nov 2020 17:44:18 +1100
> > "Gary R. Schmidt"  wrote:
> >   
> >> On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote:  
> >>> "(don't you love C?)"
> >>>
> >>> I have never understood why the originators of C didn't give integers
> >>> explicit widths in bits: their scheme made C code often non-portable.
> >>>  
> >> Because C is intended to be very, very close to the machine
> >> architecture, only a step or tow above assembler, or doing the
> >> bit-twiddling by hand.
> >>  
> >>> When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits
> >>> while longs were 64 (unlike "standard" C). This made Alpha C code not
> >>> portable to lesser CPUs. On the other hand, when I wrote C on DOS for
> >>> the IBM PC in the late 1980s, ints were only 8 bits! It took some time
> >>> to figure out why my C-compliant code failed so badly. In spite of all
> >>> that, having started programming before C was invented, I can safely
> >>> say that C is better than its predecessors for software like ClamAV.
> >>>  
> >> Uh, not a good example, I've written C code that is still in use on
> >> everything from 80286s (yes, Virginia, there are people who keep them
> >> alive, not just because they're cheap, sometimes just because they
> >> *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a
> >> matter of knowing what you are doing, and sticking to it...
> >>  
> >>> P.S. Good code these days tends to use typedefs defining things like
> >>> int32, uint64 etc. A shame the original ClamAV coders didn't do that.
> >>>  
> >> And none of this has *anything* to do with the original problem - seeing
> >> 0 when the value is 0.01, or so.
> >>
> >> This is a display problem, not a storage problem.  You could declare
> >> something as PIC(999.99) and you will still only see 0
> >> if you told it to display two decimal places.
> >>
> >>Cheers,
> >>GaryB-)  
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

"This is a display problem, not a storage problem."

I disagree. When the counts in info.blocks and info.rblocks are counts
of 16kb *blocks*, keeping precise track of the reading and scanning of
small files is impossible, no matter how clever the display code is.



On Tue, 3 Nov 2020 17:44:18 +1100
"Gary R. Schmidt"  wrote:

> On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote:
> > "(don't you love C?)"
> > 
> > I have never understood why the originators of C didn't give integers
> > explicit widths in bits: their scheme made C code often non-portable.
> >   
> Because C is intended to be very, very close to the machine 
> architecture, only a step or tow above assembler, or doing the 
> bit-twiddling by hand.
> 
> > When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits
> > while longs were 64 (unlike "standard" C). This made Alpha C code not
> > portable to lesser CPUs. On the other hand, when I wrote C on DOS for
> > the IBM PC in the late 1980s, ints were only 8 bits! It took some time
> > to figure out why my C-compliant code failed so badly. In spite of all
> > that, having started programming before C was invented, I can safely
> > say that C is better than its predecessors for software like ClamAV.
> >   
> Uh, not a good example, I've written C code that is still in use on 
> everything from 80286s (yes, Virginia, there are people who keep them 
> alive, not just because they're cheap, sometimes just because they 
> *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a 
> matter of knowing what you are doing, and sticking to it...
> 
> > P.S. Good code these days tends to use typedefs defining things like
> > int32, uint64 etc. A shame the original ClamAV coders didn't do that.
> >   
> And none of this has *anything* to do with the original problem - seeing 
> 0 when the value is 0.01, or so.
> 
> This is a display problem, not a storage problem.  You could declare 
> something as PIC(999.99) and you will still only see 0 
> if you told it to display two decimal places.
> 
>   Cheers,
>   GaryB-)


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Clamd.exe -- excluding files when scanning

[Paul Kosinski via clamav-users]

I'm not a big Windows fan, but it sounds like ClamAV regexes are rather
unfriendly to Windows since they don't seem to have an "ignore case"
option (unlike most other regex-using programs).

Assuming that is the case (sic), you might try:

  ExcludePath "[Cc]:\\[Ww][Ii][Nn][Dd][Oo][Ww][Ss]"

as a shorter way to exclude all possible spellings of "Windows".

The same rather ugly approach should work in other situations where you
want to ignore case in a ClamAV regex to accommodate Windows' filename
interpretation (which it inherited from DOS).



On Sat, 24 Oct 2020 01:44:06 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hello again,
> 
> On Fri, 23 Oct 2020, Marcy Rogers via clamav-users wrote:
> 
> > ...
> > I followed the instructions for installing Clamav for Windows and placed
> > the clamd.conf file in the c:\program files\clamav.
> > ...
> > In the config file, you will see this.
> > ...
> > ExcludePath "C:\Windows"  
> 
> There are two potential issues there. more below.
> 
> > ...
> > SelfCheck 3600
> >
> > This was set at 600 before I changed it to 3600 minutes.  Clamd.exe is
> > reading to do a selfcheck every 3600 minutes but it is not reading to
> > excludepath "c:\windows"  
> 
> It's good to know that the selfcheck interval has indeed changed from
> the default to what you have set in the config file.  At least that
> shows that you have had some effect on the daemon.  I'd just like to
> be sure that the config file that you think is having that effect is
> actually the file that's doing that, and that you don't have another
> file somewhere with the 3600 second self-check interval set but _not_
> the ExcludePath line.  If you change the interval to something like
> 1200 seconds and wait for twenty minutes you should be able to verify
> that you're working with the right file.  Alternatively you can give
> the config file path explicitly on the command line to make sure.
> 
> A couple of other things:
> 
> 1.
> 
> On Fri, 23 Oct 2020, Mark Fortescue wrote:
> 
> > Have you tried C:\\Windows or C:/Windows.  
> 
> Mr. Fortescue makes good suggestions.  The ExcludePath directive takes
> as its argument a 'regular expression', not just a string of text.
> Regular expressions are kinds of patterns which are _compared_ with a
> string of text - in this case the regex will be compared with a path
> name.  It either matches (and so the path is excluded) or it doesn't
> (so it isn't excluded).  Think about the '*' character that's often
> used when you want to list the files in a directory which all have
> names beginning with the same few characters.  A regex is like that
> with bells on.  This isn't the place to talk about regular expressions
> (if you aren't familiar with them, search for tutorials about them)
> but we do need to mention the backslash I'm afraid.  In most regular
> expression (regex) libraries, the backslash character is 'special'.
> It does not behave literally in a string as ordinary characters do; it
> escapes the following character, if that is another special character,
> thus making the special character _not_ special.  But if the following
> character is _not_ a special character, the non-special character is
> taken literally as if the backslash were not there.  That means that
> the regex
> 
> c:\Windows
> 
> actually matches
> 
> c:Windows
> 
> and if you want to have a literal backslash in a regex you generally
> have to double it, as in Mr. Fortescue's first suggestion.
> 
> Linux, MacOS etc. pathnames use the forward slash character as the
> directory separator.  Windows has a quirk.  On Windows, the directory
> separator in the pathnames is the backslash character.  Sometimes to
> get around this quirk on Windows, tools which use regexes will accept
> a forward slash instead of a backslash for the directory separator,
> avoiding the need to double backslashes everywhere which can be messy
> if there are many directories in the path.
> 
> 2.
> 
> In the config file I notice that you have
> 
> ExcludePath "C:\Windows"
> 
> but you say it continues to scan "c:\windows".  As I said I don't use
> ClamAV on Windows so I don't know if clamd behaves differently there
> from how it behaves on Linux etc., but on the operating systems that
> I'm used to working with ClamAV tools are case sensitive.  That means
> that "C:\Windows" and "c:\windows" would be two different paths, and
> excluding one would not exclude the other.  You can have more than
> one ExcludePath directive in the file so it won't hurt to try several
> 
> ExcludePath "C:\\Windows"
> ExcludePath "C:\\WINDOWS"
> ExcludePath "C:\\windows"
> ExcludePath "C:\Windows"
> ExcludePath "C:\WINDOWS"
> ExcludePath "C:\windows"
> ExcludePath "C:/Windows"
> ExcludePath "C:/WINDOWS"
> ExcludePath "C:/windows"
> 
> and see if that helps.  I'm afraid that I'm guessing here.  Also I
> left out the nine lines with a lower case 'c' but I'd be surprised if
> anything on Windows would treat the drive letter case sensitively.

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

"(don't you love C?)"

I have never understood why the originators of C didn't give integers
explicit widths in bits: their scheme made C code often non-portable.

When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits
while longs were 64 (unlike "standard" C). This made Alpha C code not
portable to lesser CPUs. On the other hand, when I wrote C on DOS for
the IBM PC in the late 1980s, ints were only 8 bits! It took some time
to figure out why my C-compliant code failed so badly. In spite of all
that, having started programming before C was invented, I can safely
say that C is better than its predecessors for software like ClamAV.

P.S. Good code these days tends to use typedefs defining things like
int32, uint64 etc. A shame the original ClamAV coders didn't do that.



On Tue, 3 Nov 2020 01:53:33 +
"Micah Snyder (micasnyd)"  wrote:

> I hadn't really looked at the code. You raise a good point.
> 
> Changing it isn't super simple.  The info.blocks variable is passed through 
> cli_scandesc_callback() and scan_common() where it's placed into the scan 
> context.  When data is scanned, the amount scanned is divided by 
> CL_COUNT_PRECISION (also found in clamav.h), which is what you multiply the 
> number by to get the value in bytes. Provided that all downstream 
> applications use CL_COUNT_PRECISION as clamscan does, we could shrink the 
> count precision from 4k to something lower, but that would also decrease the 
> max amount of data which could be scanned.  
> 
> If the variable were a uint64_t, that'd probably be fine... but it's an 
> unsigned long int... aka maybe 4 bytes or maybe 8 bytes (don't you love C?).  
> On systems where an unsigned long is 4 bytes, then that'd cap the scan limit 
> at 4GB.  Changing the variable to be an uint64_t would be "best", but it 
> would be a non-backwards compatible change to the API which is very much not 
> worth it. 
> 
> Sigh :-/
> 
> > -Original Message-
> > From: clamav-users  On Behalf Of
> > Paul Kosinski via clamav-users
> > Sent: Monday, November 2, 2020 5:23 PM
> > To: clamav-users@lists.clamav.net
> > Cc: Paul Kosinski 
> > Subject: Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned
> > 
> > Can this really be done? I was looking at the code referred to by G.W.
> > Haywood, and I see that it uses "info.blocks" and "info.rblocks".
> > Looking at the definitions in "clamav-0.103.0/clamscan/", I see the
> > following:
> > 
> > struct s_info {
> > unsigned int sigs; /* number of signatures */
> > unsigned int dirs; /* number of scanned directories */
> > unsigned int files;/* number of scanned files */
> > unsigned int ifiles;   /* number of infected files */
> > unsigned int errors;   /* number of errors */
> > unsigned long int blocks;  /* number of *scanned* 16kb blocks */
> > unsigned long int rblocks; /* number of *read* 16kb blocks */ };
> > 
> > This suggests that the counts for "scanned" and "read" are not really byte
> > counts, and EICAR's 68 bytes would always be recorded as 0 (if normal
> > rounding rules are applied).
> > 
> > 
> > 
> > On Mon, 2 Nov 2020 23:59:20 +
> > "Micah Snyder \(micasnyd\) via clamav-users" 
> > wrote:
> >   
> > > I agree.  We already have some logic in freshclam to convert bytes to 
> > > human  
> > readable B / KiB / MiB / GiB format.  It should be pretty much a copypaste
> > effort to improve the data scanned/read output.  
> > >
> > > -Micah
> > >
> > > On 11/2/20, 9:47 AM, "clamav-users on behalf of G.W. Haywood via clamav- 
> > >  
> > users"  > us...@lists.clamav.net> wrote:
> > >
> > > Hi there,
> > >
> > > On Mon, 2 Nov 2020, Paul Kosinski via clamav-users wrote:
> > >  
> > > > ... I still think it is a bad message that should be fixed.  
> > >
> > > +1
> > >
> > > If you want to try a very quick and dirty tweak to get more precise
> > > numbers, change the value of
> > >
> > > 1) CL_COUNT_PRECISION in .../libclamav/clamav.h from 4096 to 1
> > >
> > > 2) replace '1024' with '1' in four places in clamscan/clamscan.c
> > >
> > > 3) change 'MB' to 'Bytes' in two places in clamscan/clamscan.c and
> > >
> > > 4) rebuild.
> > >
> > > 
> > > 8<--
> > >  

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

Can this really be done? I was looking at the code referred to by G.W.
Haywood, and I see that it uses "info.blocks" and "info.rblocks".
Looking at the definitions in "clamav-0.103.0/clamscan/", I see the
following:

struct s_info {
unsigned int sigs; /* number of signatures */
unsigned int dirs; /* number of scanned directories */
unsigned int files;/* number of scanned files */
unsigned int ifiles;   /* number of infected files */
unsigned int errors;   /* number of errors */
unsigned long int blocks;  /* number of *scanned* 16kb blocks */
unsigned long int rblocks; /* number of *read* 16kb blocks */
};

This suggests that the counts for "scanned" and "read" are not really
byte counts, and EICAR's 68 bytes would always be recorded as 0 (if
normal rounding rules are applied).



On Mon, 2 Nov 2020 23:59:20 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> I agree.  We already have some logic in freshclam to convert bytes to human 
> readable B / KiB / MiB / GiB format.  It should be pretty much a copypaste 
> effort to improve the data scanned/read output. 
> 
> -Micah
> 
> On 11/2/20, 9:47 AM, "clamav-users on behalf of G.W. Haywood via 
> clamav-users"  clamav-users@lists.clamav.net> wrote:
> 
> Hi there,
> 
> On Mon, 2 Nov 2020, Paul Kosinski via clamav-users wrote:
> 
> > ... I still think it is a bad message that should be fixed.  
> 
> +1
> 
> If you want to try a very quick and dirty tweak to get more precise
> numbers, change the value of
> 
> 1) CL_COUNT_PRECISION in .../libclamav/clamav.h from 4096 to 1
> 
> 2) replace '1024' with '1' in four places in clamscan/clamscan.c
> 
> 3) change 'MB' to 'Bytes' in two places in clamscan/clamscan.c and
> 
> 4) rebuild.
> 
> 8<--
> ~/clamav-0.103.0-rc2: $ grep -C3 -r CL_COUNT_PRECISION clamscan libclamav 
> | ...
> ...
> ...
> clamscan/clamscan.c:mb = info.blocks * (CL_COUNT_PRECISION / 
> 1024) / 1024.0;
> clamscan/clamscan.c:logg("Data scanned: %2.2lf MB\n", mb);
> clamscan/clamscan.c:rmb = info.rblocks * (CL_COUNT_PRECISION / 
> 1024) / 1024.0;
> clamscan/clamscan.c:logg("Data read: %2.2lf MB (ratio %.2f:1)\n", 
> rmb, info.rblocks ? (double)info.blocks / (double)info.rblocks : 0);
> ...
> ...
> libclamav/clamav.h:#define CL_COUNT_PRECISION 4096
> ...
> ...
> 8<--
> 
> This is untested, YMMV.  Obviously, if you're skilled in the art, this
> can be done better.  Note that 'MB' should in any case be 'MiB' as the
> values printed are the counts divided by 2^20 and not by 10^6.
> 
> -- 
> 
> 73,
> Ged.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned

[Paul Kosinski via clamav-users]

When I first saw this message, I quickly concluded it was a roundoff
behavior. But I still think it is a bad message that should be fixed.

First, most file managers that only display file sizes in "human
readable" form, still display a non-zero size for small files. Second,
it is not logically impossible (in principle) that a file be enumerated
while not having any data read or scanned.

Third, and perhaps most important, it can make users new to ClamAV
doubt its design, implementation or reliability.



On Sun, 1 Nov 2020 19:53:18 -0800
Al Varnell via clamav-users  wrote:

> The eicar test file is 68 bytes long which is .68 MB which rounded to two 
> significant digits is 0.00 MB both scanned and read.
> 
> There are various limits, depending on file and archive types as to how much 
> is read and/or scanned. In most cases they will be exactly the same.
> 
> -Al-
> 
> > On Nov 1, 2020, at 19:40, Ankur Sharma via clamav-users 
> >  wrote:
> > 
> > Hi All,
> > 
> > I tried to scan an eicar test file and got the following scan output:
> > 
> > {'Scanning /tmp/bucket-file-upload/eicar_com.zip!ZIP': 'eicar.com 
> > ', '/tmp/bucket-file-upload/eicar_com.zip': 
> > 'Win.Test.EICAR_HDB-1 FOUND', 
> > '/tmp/bucket-file-upload/eicar_com.zip!(1)ZIP': 'eicar.com 
> > : Win.Test.EICAR_HDB-1 FOUND', 'Known viruses': 
> > '8931107', 'Engine version': '0.102.4', 'Scanned directories': '0', 
> > 'Scanned files': '1', 'Infected files': '1', 'Data scanned': '0.00 MB', 
> > 'Data read': '0.00 MB (ratio 0.00:1)', 'Time': '22.963 sec (0 m 22 s)'}
> > 
> > Though it correctly mentions that the 'Infected files' is '1'. It mentions 
> > that data scanned and data read is 0.00 MB. Can someone please help me and 
> > confirm what is Data read and Data scanned ? How are these different?
> > 
> > Thanks a lot for your time.
> > 
> > -- 
> > regards
> > Ankur  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV - Emotet - Malware not detected

[Paul Kosinski via clamav-users]

"Vaccine for Emotet Malware" at "Schneier on Security":

https://www.schneier.com/crypto-gram/archives/2020/0915.html#cg2



On Wed, 16 Sep 2020 16:27:45 +0200
Brent Clark via clamav-users  wrote:

> Hiya
> 
> Thanks so much.
> 
> I know the community and the internet as a whole, stands to gain from 
> your efforts / work.
> 
> Regards
> Brent

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: Freshclam, cdiffs and bandwidth are your friends

[Paul Kosinski via clamav-users]

"...we also only release updates once a day."

Are there *never* any urgent virus updates released in between? In
other words, is it always useless to check the TXT record more often?



On Mon, 27 Jul 2020 22:09:31 +
"Joel Esler \(jesler\) via clamav-users"  wrote:

> https://blog.clamav.net/2020/07/freshclam-cdiffs-effect-on-bandwidth.html
> 
> Freshclam, cdiffs and bandwidth are your friends
> During a recent review of file downloads from our ClamAV CDN network, we've 
> noticed hundreds of IPs that seem to be downloading the daily.cvd and the 
> main.cvd thousands of times a day.
> 
> There are about a dozen IPs that are downloading those to files more than 
> 40,000 times a day. This is causing us to transfer about 250TB of data a day. 
> We would encourage any users still doing this to cease as soon as possible. 
> Not only does it waste our bandwidth — as we have much more efficient ways of 
> downloading the updates — but it also wastes your bandwidth, as well.
> 
> Freshclam has the ability to download partial files of updates (called 
> cdiffs).  Which are smaller, more incremental updates to the database. This 
> allows users, and us, to manage our downloads in a much more efficient 
> manner. We often receive the complaint, "I have to download the daily.cvd and 
> main.cvd with Python and move the updates to an off-internet system."  That's 
> fine — it's a use case we support. However, you can do the same with 
> freshclam and the small cdiffs.
> 
> Furthermore, we also only release updates once a day.  Reducing the number of 
> updates you check for (and, subsequently, download we assume through a 
> crontab or periodic job of some type) would also alleviate this issue.
> 
> We will be constantly monitoring this in hopes that people migrate to using 
> freshclam.  Over-abusers (for instance, the top 10 IPs that are downloading 
> main.cvd 40,000 times a day), will be immediately blocked.  Further abusers 
> may also be blocked, without notice.
> 
> To mitigate, please complete the following tasks:
> 
> 1. Use Freshclam instead of Python or whatever downloading script you have 
> cron'd.
> 2. Reduce the checks to once or twice a day.
> 
> Thank you for helping keep the ClamAV network healthy.
> 
> Any questions, please see us over on the ClamAV-Users list.
> 
> 
> Sent from my  iPhone

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] clamscan vs clamdscan

[Paul Kosinski via clamav-users]

On milters:

Our email handling is a two stage mechanism. Our rented server at our
public IP address is a small, cheap VM (with no ports blocked, of
course) which runs Postfix and Apache. There is not enough RAM to also
run clamd, so we simply use Postfix's builtin filter mechanisms, like
SMTP protocol checking, sending domain vs sending IP matching, rate
limiting etc., to reject *lots* of obvious spam.

Any email that is actually received by this Postfix is *immediately*
forwarded, via an encrypted tunnel, to our second Postfix, which is on
our LAN. This second Postfix immediately "delivers" the email to our local
Dovecot, which is set up for local IMAP access. On the way, the email
is filtered via clamd and our Bayesian spam checker. We also make
extensive use of Postfix's 'valias' feature to allow the us to create
multiple email names for a given real user. This is handy when setting
up Website logins etc. (This posting from  is
an example of that.)

Only email to valid user names is delivered, the rest is discarded
right away. (You might be surprised how much email to made-up user
names arrives. These are mapped by valias's catch-all to "nobody" and
then flushed.) Email that is deemed to be spam is actually delivered
to a second instance of Dovecot, where it is sorted by year and month
and only then by the ultimate real user. This allows us to keep for
review email that only appears to be spam.

The IMAP access is completely local to our LAN, so our email (as a
whole) is not permanently stored on any physically public servers.
Also, any purely intra-domain email never leaves our LAN.

A final feature in our email handling is that any mail we send out has
its destination address automatically recorded in a local database and
any reply from that email address never gets routed to the spam bucket,
but simply has a header added indicating it's a Reply. (This database
also has a local Web interface to explicitly Query, Forget, Allow or
Block an email address.)

P.S. G.W Haywood ought to consider something like our email reply
database. When I replied to an email he sent me from what seemed to be
his private email address (i.e., not his clamav address), it was held
for a few days at his MTA, "timed out" and was then bounced -- twice.



On Sun, 10 May 2020 09:33:11 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Sat, 9 May 2020, Paul Kosinski via clamav-users wrote:
> 
> > On our mailserver, we run clamdscan, since mail arrives frequently (!).  
> 
> On a mail server most people would use a milter, e.g. clamav-milter,
> which is part of the ClamAV package.
> 
> The use of milters offers many benefits.  It enables a mail server to
> inspect a message during the SMTP conversation, allowing the server
> (for example) to reject unwanted mail at the earliest possible time,
> before accepting the message.  This can avoid wasting resources, and
> leaks of information to the sender such as the fact that a recipient
> address actually exists and accepts mail (valuable information to the
> typical spammer, because it is saleable).
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clamav with VPN

[Paul Kosinski via clamav-users]

My point about letting two IP addresses through our firewall was that
it was also a hack that might eventually fail (according to Joel, if
"cloudflare drastically changes things"). In other words I put in a
hack to work around a external problem that was beyond my ability to
fix (having no clout with Cloudflare).


On Tue, 5 May 2020 19:02:20 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Tue, 5 May 2020, Paul Kosinski via clamav-users wrote:
> 
> >>> To try to solve this issue, i have added this line in my /etc/hosts file :
> >>>
> >>>  * 104.16.218.84 database.clamav.net  
> >>
> >> Don't do things like that.  Sooner or later it will break, and you'll
> >> find yourself back here again asking why.  
> >
> > Our firewall blocks our mail server from issuing requests via ports 80
> > and 443, but, after our failure to set up a private mirror that worked
> > reliably after the switch to Cloudflare (their BOS mirror was usually
> > behind the DNS TXT reported version, as detailed in many previous
> > posts), I had to add exceptions for 104.16.218.84 and 104.16.219.84 ...  
> 
> I'm not sure that I understand your point.  Mine was that hacks like
> tweaking resolv.conf to try to get round a broken name service instead
> of fixing the service are bound to come back and bite you.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Clamav with VPN

[Paul Kosinski via clamav-users]

> > To try to solve this issue, i have added this line in my /etc/hosts file :
> >
> >  * 104.16.218.84 database.clamav.net  
> 
> Don't do things like that.  Sooner or later it will break, and you'll
> find yourself back here again asking why.


Our firewall blocks our mail server from issuing requests via ports 80
and 443, but, after our failure to set up a private mirror that worked
reliably after the switch to Cloudflare (their BOS mirror was usually
behind the DNS TXT reported version, as detailed in many previous
posts), I had to add exceptions for 104.16.218.84 and 104.16.219.84 so
that our mail server could update ClamAV. (And Joel said last July that
these IPs are quite stable for our geo-location "Unless cloudflare
drastically changes things".)

The only other alternative was to set up some sort of on-LAN relay or
proxy (e.g., Squid), which seemed like way overkill.


P.S. Since "G.W. Haywood"  never accepts
incoming mail, why not switch from CC to BCC in your submissions to
clamav-users and save us a lot of frustration. (Also, your private
email address from which you sent me a private email never accepted my
private reply, it just "timed out" -- twice.)



On Tue, 5 May 2020 12:23:10 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Tue, 5 May 2020, 21ch181 via clamav-users wrote:
> 
> > I use ExpressVPN and each time i want to update the database i see a
> > message in the logs files (syslog and freshclam) ...
> > To try to solve this issue, i have added this line in my /etc/hosts file :
> >
> >  * 104.16.218.84 database.clamav.net  
> 
> Don't do things like that.  Sooner or later it will break, and you'll
> find yourself back here again asking why.
> 
> > Please note that the update work well if i switch off my VPN.  
> 
> It's clear from your log messages that your problem is caused by name
> resolution issues.  It isn't clear exactly what they are, but it's
> obviously associated with the DNS service provided when the VPN is
> running.  Since the ExpressVPN sales pitch makes a thing of encrypting
> your DNS traffic as well as other traffic this isn't a great surprise.
> You could try to debug the name resolution using tools like 'dig', but
> it's not necessarily straightforward and in any case I'm not persuaded
> that there's a case for sending ClamAV database traffic over a VPN.
> All the information (including, now that you've posted to this list,
> the fact that you are using it) is in the public domain.
> 
> > Is someone could give me some solutions to solve this issue please ?  
> 
> Send ClamAV traffic over normal routes.  It's possible that Cloudflare
> is blocking ExpressVPN traffic, but I don't know what you'd be able to
> do about that.  Joel (on this list) might have insights to offer.
> 
> I'd never use a VPN service provided by someone else.  You can't trust
> them.  It's very easy to set up your own, then you know what's going
> on, and you aren't providing raw material from which someone probably
> intends to make a profit.
> 
> I'll leave aside the legality or otherwise of using strong encryption
> in your country, but if you can tell us why you think you need ClamAV
> on your Linux box that might be useful.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Server Agent

[Paul Kosinski via clamav-users]

Yes, I would appreciate that.

Thanks,
Paul

P.S. When I copied you on yesterday's ClamAV posting, your mail server said:

: host mail.jubileegroup.co.uk[83.67.166.33] said:
550 5.7.1 Message rejected (in reply to end of DATA command)




On Thu, 23 Apr 2020 10:15:00 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Wed, 22 Apr 2020, Paul Kosinski via clamav-users wrote:
> 
> > Your list includes a number of databases I haven't seen before. Could
> > you provide a list of source sites that provide the DBs that you find
> > most useful?  
> 
> Sorry, I don't keep an organized list but I can privately let you have
> my copy of my unofficial database update script and the configuration,
> if that's any help.  The script is based on Bill Landry's original
> from about a decade ago, although there are much more recent works.
> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV Server Agent

[Paul Kosinski via clamav-users]

Your list includes a number of databases I haven't seen before. Could
you provide a list of source sites that provide the DBs that you find
most useful?

Thanks!



On Wed, 22 Apr 2020 18:43:47 +0100 (BST)
"G.W. Haywood via clamav-users"  wrote:

> Hi there,
> 
> On Wed, 22 Apr 2020, Karmendra Suthar via clamav-users wrote:
> 
> > Actually I never had any antivirus on my  linux we servers, but PCI
> > complaince forced me to install it on my servers. Now a bit of my CPU and
> > RAM is going into running the antivirus, not sure how much, but
> > definitely something is used up.  
> 
> If you have the clamd daemon running, and it is using the 'official'
> databases (which are normally configured by the installation scripts
> for most Linux distributions) then it will use about a gigabyte of
> memory in normal operation and practically no other resources until
> you require ClamAV to scan something.  As has been mentioned you can
> ask ClamAV to scan something in several different ways, and you need
> to become familiar with them in order to use ClamAV effectively.
> 
> > I have 3 ubuntu 18 servers running load balanced nginx webservers (all
> > these servers are on AWS), only ports like 80, 443, 22(ip restricted) are
> > open to these servers. I run OSSEC for intrusion detection in a server
> > agent model a 4th server is used as bastion server that runs  ossec-server,
> > time-server etc and these 3 webservers uses this bastion server.
> > I wanted to mange the anti virus also from this bastion server.  
> 
> You could install clamd on the bastion server and configure it to
> listen on a TCP port for connections only from your other servers.
> Then you would only need to keep a single set of databases and you
> would only have to keep that single set of databases up to date.
> There is one issue which might not be covered in that case; if you
> wish to use on-access scanning then the last I heard from ClamAV's
> development team was that there are still some things to do to get
> a remote clamd to handle on-access scanning.  I'm sure someone from
> Talos will chip in with a comment if there's still an issue there.
> 
> > 1. When I am using freshclam what kind of threat I am getting
> > protection from?  
> 
> If I were going to install something like ClamAV, I would want to know
> the answer to that question before I installed it, not after.  Before
> that I would want to know and in your case probably document carefully
> what threats my systems faced, and also what the likely results of a
> compromise might be.  For example loss of earnings, lawsuits, people
> becoming homeless and/or starving to death, you being sent to prison,
> that kind of thing.
> 
> ClamAV is a kind of tool kit, and it's up to you how you want to use
> it to make scans happen.  It's also up to you what you want to do if
> something is reported as 'FOUND' by the scanning process.  By default
> nothing else happens, and it would be most unwise (for example) simply
> to delete or move the offending object as it you might have discovered
> a 'false positive' (a very common subject on this mailing list).  To
> blithely move (or delete) system files, for example, on a Linux box is
> very dangerous for the system.  It's better just to mount the system
> partition(s) read-only, so that nothing can mess with them unless the
> box is already hopelessly compromised.
> 
> To be clear, 'freshclam' is the thing which updates your databases.
> The things which use the databases when scanning are usually clamd
> (which is the persistent daemon) and clamscan (which does _not_ use
> the daemon).
> 
> The clamd daemon loads the databases into memory when it starts, and
> then waits for some process to ask it to scan things.  The requesting
> process can be clamdscan, clamav-milter, some other milter such as one
> I wrote for use here, or something else.  When a process requests that
> something be scanned it can, depending on how things are configured,
> either give the location of a directory or a file to scan, or it can
> send the data to be scanned directly to the daemon via a socket.
> 
> (I do not know what other signature DB i can use for webserver. there
> > is no mails on these servers)  
> 
> Try searching, for example, for "ClamAV unofficial databases".  It's
> up to you, since ClamAV is a tool kit, to configure which databases
> are to be used by ClamAV, and to ensure that they're kept up to date,
> and, for that matter, that they are appropriate to the tasks that you
> have decided that ClamAV is to do for you.
> 
> > 2. You mentioned clamd scans TCP ports, my question is it by default scans
> > all data on all open ports or we need to configure it to do so.  
> 
> By default TCP ports are not used, and in any case no port scanning
> takes place - ClamAV is not like 'nmap', or 'metasploit', for example.
> TCP ports are only used for communication between a client, which asks
> for something to be scanned, and the server, which scans it.
> 

Re: [clamav-users] ClamAV 0.102.2 needs a "--without-systemd" option

[Paul Kosinski via clamav-users]

Andrew,

Yeah, per your posting, I tried running 'configure' specifying
`--with-systemdsystemunitdir=no` and it seemed to be suppress the
systemd tie-in. (I didn't actually run 'make'.)

It would be nice if this were documented somewhere. The "--help" info
from 'configure' only lists 'DIR' as an argument. The latest reference
to 'systemd' in NEWS.md is for version 0.99.2 (and the other ".md"
files have nothing), and I couldn't find anything about this new
approach in the documentation or faqs on GitHub. (Google doesn't turn
up anything definitive either.)

In any case, the systemd tie-in is a *major* change: it turns ClamAV
from a mere package into a run-at-startup *service*, and needs to be
presented as such.  

-Paul


On Sun, 19 Apr 2020 15:17:51 -0400
Andrew Williams  wrote:

> Paul,
> 
> You should be able to use `--with-systemdsystemunitdir=no` to make it
> so that `make install` won't try to register clamd as a systemd
> service
> 
> -Andrew
> 
> On Sun, Apr 19, 2020 at 1:26 PM Paul Kosinski via clamav-users <
> clamav-users@lists.clamav.net> wrote:
> 
> > I finally built 0.102.2 a few days ago and was rather shocked that
> > it was tightly integrated into systemd. In a point release,
> > converting ClamAV into a mandatory server strikes me as weird,
> > especially since there is no "--without-systemd" option.
> >
> > I am not philosophically opposed to systemd (its partial ordering of
> > dependencies is actually quite elegant), but I have never used
> > ClamAV in conjunction with systemd (although I might consider it in
> > the future).
> >
> > Now for some details...
> >
> > The way I always have built ClamAV is to install each new version
> > in /opt under its version number. This allows me to try out the new
> > version without needing to shut down the running version. Then I
> > switch to the new version almost atomically by changing one symlink
> > (e.g., /opt/clamav -> /opt/clamav.0.102.2) and restarting clamd. So
> > if the new version has some problem, I can switch back (also almost
> > atomically).
> >
> > Luckily, my procedure was not totally wiped out by the systemd
> > issue due to the fact that (for extra security) I never run "make
> > install" as root. I always create the new ClamAV version directory
> > in /opt owned by the build user and install as that user (followed
> > by "chown -R 0.0" etc.). So the install failed without adding weird
> > stuff to my systemd environment.
> >
> > I then worked around the problem by studying the "configure"
> > options and found that there was an option
> > "--with-systemdsystemunitdir". So I pointed that to a harmless new
> > directory (/opt/clamav.0.102.2/systemd) and reran "configure",
> > "make", "make check" and "make install", which then all worked, and
> > showed me what the new systemd files contained.
> >
> > Thus I would strongly recommend adding a "--without-systemd" option
> > to the new "configure". If I hadn't employed my workaround, "make
> > install" (as root) would have added those 3 files to the standard
> > systemd environment. This have totally broken the way I support
> > multiple versions of ClamAV, as those files have *absolute* paths
> > to the new version of ClamAV no matter where installed.
> >
> > P.S. I run freshclam via cron and my own "getfreshclam" wrapper.
> > This allows me to keep older signature files around in case a new
> > version has a serious problem. (It was also quite useful in
> > investigating the multi-hour out-of-date problem with Cloudflare's
> > BOS mirror.)
> >
> > Finally, note that simply using systemd and thus freshclam's builtin
> > periodic update mechanism (instead of cron) wouldn't easily allow
> > keeping previous signature files around as backups.
 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] ClamAV 0.102.2 needs a "--without-systemd" option

[Paul Kosinski via clamav-users]

I finally built 0.102.2 a few days ago and was rather shocked that it was 
tightly integrated into systemd. In a point release, converting ClamAV into a 
mandatory server strikes me as weird, especially since there is no 
"--without-systemd" option.

I am not philosophically opposed to systemd (its partial ordering of 
dependencies is actually quite elegant), but I have never used ClamAV in 
conjunction with systemd (although I might consider it in the future). 

Now for some details...

The way I always have built ClamAV is to install each new version in /opt under 
its version number. This allows me to try out the new version without needing 
to shut down the running version. Then I switch to the new version almost 
atomically by changing one symlink (e.g., /opt/clamav -> /opt/clamav.0.102.2) 
and restarting clamd. So if the new version has some problem, I can switch back 
(also almost atomically). 

Luckily, my procedure was not totally wiped out by the systemd issue due to the 
fact that (for extra security) I never run "make install" as root. I always 
create the new ClamAV version directory in /opt owned by the build user and 
install as that user (followed by "chown -R 0.0" etc.). So the install failed 
without adding weird stuff to my systemd environment.

I then worked around the problem by studying the "configure" options and found 
that there was an option "--with-systemdsystemunitdir". So I pointed that to a 
harmless new directory (/opt/clamav.0.102.2/systemd) and reran "configure", 
"make", "make check" and "make install", which then all worked, and showed me 
what the new systemd files contained.

Thus I would strongly recommend adding a "--without-systemd" option to the new 
"configure". If I hadn't employed my workaround, "make install" (as root) would 
have added those 3 files to the standard systemd environment. This have totally 
broken the way I support multiple versions of ClamAV, as those files have 
*absolute* paths to the new version of ClamAV no matter where installed.

P.S. I run freshclam via cron and my own "getfreshclam" wrapper. This allows me 
to keep older signature files around in case a new version has a serious 
problem. (It was also quite useful in investigating the multi-hour out-of-date 
problem with Cloudflare's BOS mirror.)

Finally, note that simply using systemd and thus freshclam's builtin periodic 
update mechanism (instead of cron) wouldn't easily allow keeping previous 
signature files around as backups.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Heuristics.Limits.Exceeded FOUND

[Paul Kosinski via clamav-users]

Micah,

Yeah, now that you mention it, I remember having read somewhere about
".ja" files as being not quite zip format. On Linux, the (latest)
"file" command identifies ".ja" as "Mozilla archive" format. But
(recent) unzip commands don't seem to have any trouble (unlike ARK,
which can't find a suitable plugin).

But my favorite file manager -- emacs (!) -- expands them nicely into a
virtual file list so you can look inside the individual files.

P.S. Speaking of ISO and GPT formats, proper handling of them will need
going to 64-bit file sizes and offsets.


On Sun, 12 Apr 2020 17:39:22 +
"Micah Snyder (micasnyd)"  wrote:

> Paul,
> 
> I investigated further and realize now that it ISN'T
> double-extracting files from plain zips.  It is double-extracting
> files from zips within other raw image file formats, like TAR or
> image file formats.  For a plain zip, It detects the file entries
> twice, but doesn't extract them if the parent file is a zip. 
> 
> I tested this by making a simple zip with two text files in it, then
> tar.gz'd it.  Scanning the zip.tar.gz file resulted in
> double-extraction of both text files.  
> 
> Funny story, the omni.ja file is not a real zip.  The author of the
> format decided to place the central directory header at the beginning
> of the file instead of at the end, resulting in a new zip-like file
> format.  We're able to parse out the files from omni.ja okay because
> we have self-extracting zip signatures that identify the individual
> file entries and because the omni.ja file itself is detected as
> "binary data" (so the ZIPSFX-in-a-ZIP exclusion rule does not apply). 
> 
> Anyhow, I now suspect that the omni.ja file in a tar.gz file will
> also get double-extracted.  The simplest option would be to disable
> file-type-recognition scans for embedded files file formats in TAR
> files (and also GPT and other non-compressed archive file formats).
> I had been wanting to do this anyways after investigating a closely
> related issue regarding ISO/GPT file formats. This definitely gives
> us more reason to do so.
> 
> -Micah
> 
> On 4/10/20, 6:55 PM, "Paul Kosinski"  wrote:
> 
> Is this a generic problem with compressed archives (like the
> Firefox ".tar.bz2") or is it zip specific? 
> 
> If it is zip specific, there are 2 files in the Firefox
> distribution file that are zip format compressed which might explain
> the slowness. (They are both named omni.ja, but have different
> contents). 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Heuristics.Limits.Exceeded FOUND

[Paul Kosinski via clamav-users]

Is this a generic problem with compressed archives (like the Firefox
".tar.bz2") or is it zip specific? 

If it is zip specific, there are 2 files in the Firefox distribution
file that are zip format compressed which might explain the slowness.
(They are both named omni.ja, but have different contents).



On Fri, 10 Apr 2020 19:58:35 +
"Micah Snyder (micasnyd)"  wrote:

> One issue ClamAV currently has with scanning Zip archives is that
> ClamAV's self-extracting zip detection logic has a flaw wherein it
> detects every file within a zip as a new self-extracting zip.  As a
> result, I believe (and I could be wrong on this), that Clam ends up
> extracting and scanning every file in a zip *twice*.  I'm still
> brainstorming the best way to fix this -- but I suspect this is a
> large part of why zip-based file formats take much longer than
> expected to scan. 
> 
> -Micah
> 
> 
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>  
> 
> 
> 
> On 4/7/20, 1:38 PM, "clamav-users on behalf of Paul Kosinski via
> clamav-users"  clamav-users@lists.clamav.net> wrote:
> 
> I didn't want to screw around with my clamdscan (clamd.conf)
> settings, so I ran my optioned-up clamscan command on a smaller and
> much less complicated file. It took less than 11 seconds total time.
> (My previous guess on clamscan's DB load time was apparently way off.)
> 
> This suggests that the ClamAV scanning process really does take a
> lot of CPU to deal with a big, complicated file like a Firefox
> package: 
>   time clamscan
>--alert-exceeds-max=yes --max-scantime=99
> --max-scansize=4090M --max-filesize=4090M --max-files=3
> --max-recursion=30 --pcre-match-limit=9
> --pcre-max-filesize=9 audiofile.wav 
>   audiofile.wav: OK
> 
>   --- SCAN SUMMARY ---
>   Known viruses: 6804144
>   Engine version: 0.102.1
>   Scanned directories: 0
>   Scanned files: 1
>   Infected files: 0
>   Data scanned: 1.74 MB
>   Data read: 1.73 MB (ratio 1.01:1)
>   Time: 10.836 sec (0 m 10 s)
> 
>   real0m10.851s
>   user0m10.439s
>   sys 0m0.412s
> 
> P.S. This is an actual audio intermediate file, not just random
> bytes. 
> 
> 
> On Mon, 6 Apr 2020 21:50:15 -0700
> Al Varnell via clamav-users  wrote:
> 
> > Much of that time is almost certainly being consumed by loading
> > the signature database into RAM. How long does it take using
> > clamdscan?
> > 
> > Sent from my iPad
> > 
> > -Al-
> > 
> > On Apr 6, 2020, at 12:29, Paul Kosinski via clamav-users
> >  wrote:  
> > > 
> > > It *does* take more than 120 secs for the clamscan command to
> > > fully scan the 62 MB Firefox installation file (.tar.bz2).
> > > Trying the scan with the default clamscan limits results in
> > > 62 MB "Data read" but *zero* "Data scanned"!

> 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Heuristics.Limits.Exceeded FOUND

[Paul Kosinski via clamav-users]

I didn't want to screw around with my clamdscan (clamd.conf) settings,
so I ran my optioned-up clamscan command on a smaller and much less
complicated file. It took less than 11 seconds total time. (My previous
guess on clamscan's DB load time was apparently way off.)

This suggests that the ClamAV scanning process really does take a lot
of CPU to deal with a big, complicated file like a Firefox package:

  time clamscan
   --alert-exceeds-max=yes --max-scantime=99 --max-scansize=4090M 
--max-filesize=4090M --max-files=3
   --max-recursion=30 --pcre-match-limit=9 
--pcre-max-filesize=9
audiofile.wav

  audiofile.wav: OK

  --- SCAN SUMMARY ---
  Known viruses: 6804144
  Engine version: 0.102.1
  Scanned directories: 0
  Scanned files: 1
  Infected files: 0
  Data scanned: 1.74 MB
  Data read: 1.73 MB (ratio 1.01:1)
  Time: 10.836 sec (0 m 10 s)

  real0m10.851s
  user0m10.439s
  sys 0m0.412s

P.S. This is an actual audio intermediate file, not just random bytes.



On Mon, 6 Apr 2020 21:50:15 -0700
Al Varnell via clamav-users  wrote:

> Much of that time is almost certainly being consumed by loading the
> signature database into RAM. How long does it take using clamdscan?
> 
> Sent from my iPad
> 
> -Al-
> 
> On Apr 6, 2020, at 12:29, Paul Kosinski via clamav-users
>  wrote:
> > 
> > It *does* take more than 120 secs for the clamscan command to fully
> > scan the 62 MB Firefox installation file (.tar.bz2). Trying the scan
> > with the default clamscan limits results in 62 MB "Data read" but
> > *zero* "Data scanned"!  

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Heuristics.Limits.Exceeded FOUND

[Paul Kosinski via clamav-users]

Micah,

It *does* take more than 120 secs for the clamscan command to fully
scan the 62 MB Firefox installation file (.tar.bz2). Trying the scan
with the default clamscan limits results in 62 MB "Data read" but
*zero* "Data scanned"!

Since I previously had run afoul of file size limits, I had written a
wrapper script that set all the "--max-*" limits to values that should
not cause any unnecessary failures. The problem I ran into with 0.102.x
was that the "--help" info for the clamscan command's "--max-scantime"
was incomplete.

I had set the "--max-scantime" limit to 999, assuming it was seconds.
It never occurred to me that it would be milliseconds, especially
since the clamscan command can't even load the DB in under a second.
(Milliseconds would be reasonable for clamd usage, I suppose.)

When somebody pointed out that the max scan time was really in msecs, I
updated my wrapper script and everything worked nicely, like 0.101.x.

Now, scanning the big Firefox installation file takes well over 120
secs real time, to wit (expanding the wrapper):

  time clamscan
   --alert-exceeds-max=yes --max-scantime=99 --max-scansize=4090M 
--max-filesize=4090M --max-files=3
   --max-recursion=30 --pcre-match-limit=9 
--pcre-max-filesize=9
firefox-68.6.1-esr-64.tar.bz2

  firefox-68.6.1-esr-64.tar.bz2: OK

  --- SCAN SUMMARY ---
  Known viruses: 6797620
  Engine version: 0.102.1
  Scanned directories: 0
  Scanned files: 1
  Infected files: 0
  Data scanned: 622.26 MB
  Data read: 62.06 MB (ratio 10.03:1)
  Time: 140.191 sec (2 m 20 s)

  real2m20.219s
  user2m17.212s
  sys 0m2.820s

Paul

P.S. This is on an "Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz" with 32 GB RAM.



On Mon, 6 Apr 2020 15:23:42 +
"Micah Snyder (micasnyd)"  wrote:

> Paul,
> 
> Are you seeing many files that take longer than 2 minutes to scan?
> We thought the default scan time limit was already quite high at 2
> minutes.
> 
> -Micah
> 
> On 4/4/20, 1:47 AM, "clamav-users on behalf of Paul Kosinski via
> clamav-users"  clamav-users@lists.clamav.net> wrote:
> 
> "If one is overriding a default value by providing it on the
> command line, you should know what you're doing. Guessing is never a
> good idea, especially if (like here) the documentation is lacking."
> 
> "It was noted in the list of notable changes in 0.102.0 ... which
> Paul *must* have read, otherwise he would *not* have known of the
> existence of this parameter". Really?
> 
> Does issuing "clamscan --help", and reading its output of 700
> words on 103 lines (according to wc), including one line about
> "--max-scantime", constitute guessing?  Who knew?
> 
> P.S. Up until 0.102.0, direct use of the clamscan command worked
> well for files like the Firefox download. Starting with 0.102.0,
> clamscan started giving Heuristic Limit errors. Since there was no
> indication as to *which* Limit was hit, I read the "--help" to see
> what to do. 

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml