Re: [clamav-users] Scanning a large file through HTTP

[Paul Kosinski via clamav-users]

Seems to me that this behavior, advertising a 4GB limit while silently imposing 
a 2GB limit and reporting "OK" for anything in between, is a *major* security 
flaw: ClamAV *must* report that the file was too big to deal with (however 
worded).

Thus I've taken to using clamscan rather than clamdscan (slow though that is), 
because at least it reports how many bytes were read, and how many scanned, so 
I can see what's going on.

P.S. Recently I've downloaded some MP3s from Amazon and scanned them (as I do 
everything I download -- except updates from my Linux distros). But for a 
reason I saw on this list -- but can't remember -- MP3s are fully read, but not 
scanned. Is this going to be remedied?


On Wed, 7 Apr 2021 22:14:39 +
"Micah Snyder \(micasnyd\) via clamav-users"  
wrote:

> In reality, the file size limit is 2GB.  Anything larger than that will be 
> automatically skipped and marked as “OK”.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] vistumbler as false positive

[Al Varnell via clamav-users]

Without knowing the name of the infection I can't provide even a guess as to 
whether it is or not, but the exact answer to your question is for you to 
report it by filling out the form found @https://www.clamav.net/reports/fp 
including the file itself.

Sent from my iPad

-Al-

On Apr 7, 2021, at 18:03, Eero Volotinen  wrote:
> https://raw.github.com/acalcutt/Releases/master/Vistumbler/VistumblerMDB/v10/Vistumbler_v10-7.exe
> 
> Looks like this is (vistumbler) detected as false positive.
> 
> How to fix this?
> 
> Eero

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] vistumbler as false positive

[Eero Volotinen]

https://raw.github.com/acalcutt/Releases/master/Vistumbler/VistumblerMDB/v10/Vistumbler_v10-7.exe

Looks like this is (vistumbler) detected as false positive.

How to fix this?

Eero

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Scanning a large file through HTTP

[Micah Snyder (micasnyd) via clamav-users]

Hi,

Is it 4 GB? Can this size be increased ?

You can increase the maximum file size by setting the MaxFileSize option in 
clamd.conf.  ClamAV’s option parser won’t allow you to set a maximum scan size 
higher than 4GB.  In reality, the file size limit is 2GB.  Anything larger than 
that will be automatically skipped and marked as “OK”.

The reason for the 2GB file size limit is that in the past there were several 
bug reports for files larger than 2GB causing crashes.  Rather than fix the 
parsers, the devs slipped in a 2GB file size limit to prevent crashes.  I only 
just realized it a few weeks ago, and you can see my comments on this here: 
https://github.com/Cisco-Talos/clamav-devel/commit/1a3b784e1954e00b6463000a817da0c5092296cd

There’s a lot of technical work to be done to safely raise that limitation, as 
large files of various file types types have never been tested.  A large TAR, 
for example, may well work fine when a large ZIP might crash the program.  We 
really have no idea.  Basically it’s going to take a bunch of testing when 
someone goes to work on this.

A lot of folks seem to be unhappy with it saying “OK” when a file hasn’t been 
scanned (myself included).  So we have been talking about changing the output 
to something like the following messages when files are not scanned or are only 
partially scanned:

  *   “SKIPPED (exceeded max file size)”
  *   “INCOMPLETE (exceeded max scan size)”

The exact wording is TBD.  If anyone has any specific requests, I’d enjoy some 
help brainstorming.

Is the memory or persistent storage a limit for ClamAV to scan a file ? if it 
is a persistent storage then can i increase the limit by attaching an external 
NFS ?

Sorry, persistent storage is not the concern.

Read somewhere the full file size is mapped to memory. Is it the case for 
INSTREAM command also ?

Yes, INSTREAM is also limited to 4GB (or _really_ 2GB).

If it is the case then even if chunking is supported then the server side must 
have at least 4GB of memory.

Scanning a file in chunks is a waste of CPU cycles.  ClamAV was designed to 
process a whole file all at once.  Some file formats, like PDF, DMG, and ZIP* 
store metadata at the end of the file which is necessary to properly parse the 
file.  Streaming scanners like the one in Snort struggle or can’t process these 
files.  I put a * near ZIP because zips are actually pretty easy to parse 
in-order even if the central directory is missing.  Files like DMG, on the 
other hand, can’t even be identified as DMG’s without reading the end of the 
file first, or trusting the “.dmg” file extension (which is dangerous).

In short, don’t send chunks of files as separate files to be scanned; It 
probably won’t catch any malware that way and may print lots of warnings or 
errors if it gets confused about the type of the file and starts processing it 
with the wrong parser.

Regards,
-Micah


Micah Snyder
ClamAV
Talos
Cisco Systems, Inc.



From: clamav-users  On Behalf Of Saurav 
Sarkar via clamav-users
Sent: Wednesday, April 7, 2021 7:39 AM
To: clamav-users@lists.clamav.net
Cc: Saurav Sarkar 
Subject: [clamav-users] Scanning a large file through HTTP

Hi All,

We are using a HTTP enabled malware scanning service based on Clam AV.

The service is made something like this
https://github.com/solita/clamav-rest

We have files like CAD files which can go in GBs and want to send to this 
malware scanning service.

Is there a possibility to send the file in chunks and get it scanned in the 
server side in chunks.

I observed that there is a INSTREAM command in clamd for this purpose and also 
there is a 4GB size limit.
https://linux.die.net/man/8/clamd


Read somewhere the full file size is mapped to memory. Is it the case for 
INSTREAM command also ?

If it is the case then even if chunking is supported then the server side must 
have at least 4GB of memory.

Best Regards,
Saurav





___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.103.2 security patch release

[Joel Esler (jesler) via clamav-users]

We’ll look into that for a future update.  

Sent from my  iPhone

> On Apr 7, 2021, at 16:58, Arjen de Korte via clamav-users 
>  wrote:
> 
> Citeren "Joel Esler (jesler) via clamav-users" 
> :
> 
>> It’s available on the webpage.
> 
> I already wrote that I know it is available from the website. I need to 
> update the stored keyring in openSUSE Factory, which needs a backlink to the 
> origin. Rather than downloading https://www.clamav.net/downloads and trimming 
> the HTML code, a straight download link for the keyfile would make it easier 
> to verify it.
> 
 On Apr 7, 2021, at 4:29 PM, Arjen de Korte via clamav-users 
  wrote:
>>> 
>>> Citeren "Joel Esler (jesler) via clamav-users" 
>>> :
>>> 
>>> It seems the package is now signed with a different PGP key. Is there a 
>>> location from where I can directly download the public key, rather than 
>>> copying it from the webpage?
>>> 
>>> Best regards, Arjen
>>> 
>>> 
>>> ___
>>> 
>>> clamav-users mailing list
>>> clamav-users@lists.clamav.net
>>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>> 
>>> 
>>> Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq
>>> 
>>> http://www.clamav.net/contact.html#ml
>> 
>> 
>> ___
>> 
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>> 
>> 
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>> 
>> http://www.clamav.net/contact.html#ml
> 
> 
> 
> 
> ___
> 
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.103.2 security patch release

[Arjen de Korte via clamav-users]

Citeren "Joel Esler (jesler) via clamav-users"  
:



It’s available on the webpage.


I already wrote that I know it is available from the website. I need  
to update the stored keyring in openSUSE Factory, which needs a  
backlink to the origin. Rather than downloading  
https://www.clamav.net/downloads and trimming the HTML code, a  
straight download link for the keyfile would make it easier to verify  
it.


On Apr 7, 2021, at 4:29 PM, Arjen de Korte via clamav-users  
 wrote:


Citeren "Joel Esler (jesler) via clamav-users"  
:


It seems the package is now signed with a different PGP key. Is  
there a location from where I can directly download the public key,  
rather than copying it from the webpage?


Best regards, Arjen


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml





___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.103.2 security patch release

[Joel Esler (jesler) via clamav-users]

It’s available on the webpage.

> On Apr 7, 2021, at 4:29 PM, Arjen de Korte via clamav-users 
>  wrote:
> 
> Citeren "Joel Esler (jesler) via clamav-users" 
> :
> 
> It seems the package is now signed with a different PGP key. Is there a 
> location from where I can directly download the public key, rather than 
> copying it from the webpage?
> 
> Best regards, Arjen
> 
> 
> ___
> 
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.103.2 security patch release

[Arjen de Korte via clamav-users]

Citeren "Joel Esler (jesler) via clamav-users"  
:


It seems the package is now signed with a different PGP key. Is there  
a location from where I can directly download the public key, rather  
than copying it from the webpage?


Best regards, Arjen


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: Are you still attempting to download safebrowsing.cvd?

[Joel Esler (jesler) via clamav-users]

On Apr 7, 2021, at 3:04 PM, Vladislav Kurz via clamav-users 
mailto:clamav-users@lists.clamav.net>> wrote:

Dne středa 7. dubna 2021 19:41:34 CEST, Joel Esler (jesler) via clamav-users
napsal(a):
https://blog.clamav.net/2021/04/are-you-still-attempting-to-download.html


Are you still attempting to download safebrowsing.cvd?

It has come to our attention that a few of you (about 515,000 of you, to
be more accurate), are still attempting to download the safebrowsing.cvd
file from the official ClamAV mirrors.  This tells us that these
attempted downloads are an installation of FreshClam (a non-updated
FreshClam.conf or other script) that have not been updated to remove the
safebrowsing database.>

Hello,

These could be Debian users. The debian package offers to enable
safebrowsing.cvd, and there is no indication that it is discontinued. Perhaps,
if you talk to Debian Clamav maintainers, they could release an update that
disables this option without asking ?

Anyway I was one of those, and now disabling it everywhere…

Thank you, we will do.

--
Joel Esler
Manager, Communities Division
Cisco Talos Intelligence Group
https://www.talosintelligence.com | https://www.snort.org | 
https://www.clamav.net

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Cvdupdate user?

[Joel Esler (jesler) via clamav-users]

If you are in Switzerland, and using cvdupdate, and have cvdupdate set to check 
for updates once an hour, please contact me off list.

Your UUID is:

CVDUPDATE/0.3.0 (3d80e05e-0afa-449b-b289-1de3e9c070f1)


Thanks!

--
Joel Esler
Manager, Communities Division
Cisco Talos Intelligence Group
http://www.talosintelligence.com | https://www.snort.org

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: Are you still attempting to download safebrowsing.cvd?

[Vladislav Kurz via clamav-users]

Dne středa 7. dubna 2021 19:41:34 CEST, Joel Esler (jesler) via clamav-users 
napsal(a):
> > https://blog.clamav.net/2021/04/are-you-still-attempting-to-download.html
> >  > l>
> > 
> > Are you still attempting to download safebrowsing.cvd?
> > 
> >  It has come to our attention that a few of you (about 515,000 of you, to
> >  be more accurate), are still attempting to download the safebrowsing.cvd
> >  file from the official ClamAV mirrors.  This tells us that these
> >  attempted downloads are an installation of FreshClam (a non-updated
> >  FreshClam.conf or other script) that have not been updated to remove the
> >  safebrowsing database.> 

Hello,

These could be Debian users. The debian package offers to enable 
safebrowsing.cvd, and there is no indication that it is discontinued. Perhaps, 
if you talk to Debian Clamav maintainers, they could release an update that 
disables this option without asking ?

Anyway I was one of those, and now disabling it everywhere...

-- 
Best Regards
Vladislav Kurz




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] ClamAV® blog: ClamAV 0.103.2 security patch release

[Joel Esler (jesler) via clamav-users]

> 
> https://blog.clamav.net/2021/04/clamav-01032-security-patch-release.html 
> 
> 
> ClamAV 0.103.2 security patch release
> 
> Wednesday, April 7, 2021
> 
>  <>ClamAV 0.103.2 is out now. Users can head over to clamav.net/downloads 
>  to download the release materials.
> 
> ClamAV 0.103.2 is a security patch release with the following fixes:
> 
> CVE-2021-1386 : 
> Fix for UnRAR DLL load privilege escalation. Affects 0.103.1 and prior on 
> Windows only.
> 
> CVE-2021-1252 : 
> Fix for Excel XLM parser infinite loop. Affects 0.103.0 and 0.103.1 only.
> 
> CVE-2021-1404 : 
> Fix for PDF parser buffer over-read; possible crash. Affects 0.103.0 and 
> 0.103.1 only.
> 
> CVE-2021-1405 : 
> Fix for mail parser NULL-dereference crash. Affects 0.103.1 and prior.
> 
> Fix possible memory leak in PNG parser.
> 
> Fix ClamOnAcc scan on file-creation race condition so files are scanned after 
> their contents are written.
> 
> FreshClam: Deprecate the SafeBrowsing config option. The SafeBrowsing option 
> will no longer do anything.
> 
> For more details, see our blog post from last year about the future of the 
> ClamAV Safe Browsing database 
> .
> 
> Tip: If creating and hosting your own safebrowing.gdb database, you can use 
> the DatabaseCustomURL option in freshclam.conf to download it.
> 
> FreshClam: Improved HTTP 304, 403, & 429 handling.
> 
> FreshClam: Added back the mirrors.dat file to the database directory.
> 
> This new mirrors.dat file will store:
> A randomly generated UUID for the FreshClam User-Agent.
> A retry-after timestamp that so FreshClam won't try to update after having 
> received an HTTP 429 response until the Retry-After timeout has expired.
> 
> FreshClam will now exit with a failure in daemon mode if an HTTP 403 
> (Forbidden) was received, because retrying later won't help any. The 
> FreshClam user will have to take actions to get unblocked.
> 
> Fix the FreshClam mirror-sync issue where a downloaded database is "older 
> than the version advertised."
> 
> If a new CVD download gets a version that is older than advertised, FreshClam 
> will keep the older version and retry the update so that the incremental 
> update process (CDIFF patch process) will update to the latest version.
> Labels: 0.103.2 , clamav 
> , release 
> 


signature.asc
Description: Message signed with OpenPGP

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] ClamAV® blog: Are you still attempting to download safebrowsing.cvd?

[Joel Esler (jesler) via clamav-users]

> 
> https://blog.clamav.net/2021/04/are-you-still-attempting-to-download.html 
> 
> 
> Are you still attempting to download safebrowsing.cvd?
> 
>  It has come to our attention that a few of you (about 515,000 of you, to be 
> more accurate), are still attempting to download the safebrowsing.cvd file 
> from the official ClamAV mirrors.  This tells us that these attempted 
> downloads are an installation of FreshClam (a non-updated FreshClam.conf or 
> other script) that have not been updated to remove the safebrowsing database.
> 
> We discontinued the distribution of the SafeBrowsing database in November of 
> 2019, as indicated in our blog post 
>  back 
> in June, however these installations of ClamAV that have no updated and 
> continue to download the safebrowsing.cvd account for nearly 10TB of traffic 
> a month, just for that file.
> 
> As a result, we have put in a block to make any attempts to download the 
> safebrowsing.cvd result in a 403 error.
> 
> Please take a look at your FreshClam.conf file, in our upcoming version of 
> ClamAV (0.103.2 and beyond), we have removed all of the code that attempts to 
> download the safebrowsing db from our servers, (so you should update) 
> however, for those of you can't update right away, please find this line:
> 
> SafeBrowsing yes
> 
> In your Freshclam.conf file, and commenting it out like this:
> 
> #SafeBrowsing yes
> 
> Thank you for your support of ClamAV!


signature.asc
Description: Message signed with OpenPGP

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Scanning a large file through HTTP

[Saurav Sarkar via clamav-users]

Thanks a lot Paul and Ged for your replies.

Perhaps i added too many confusing points in my question :).

If i just consider myself as a developer of the HTTP Service for malware
scanning.

I would just like to know what could be the maximum file size which can be
supported by ClamAV ?

Is it 4 GB ? Can this size be increased ?

Is the memory or persistent storage a limit for ClamAV to scan a file ? if
it is a persistent storage then can i increase the limit by attaching an
external NFS ?

Best Regards,
Saurav

On Wed, Apr 7, 2021 at 8:45 PM Paul Smith via clamav-users <
clamav-users@lists.clamav.net> wrote:

> On 07/04/2021 15:38, Saurav Sarkar via clamav-users wrote:
> >
> > We have files like CAD files which can go in GBs and want to send to
> > this malware scanning service.
>
> Why are you scanning CAD files?
>
> Can your CAD files contain arbitrary executable code which is blindly
> executed by the CAD software? If not, there's no reason to scan them? If
> they can, then I'd consider getting different CAD software...
>
>
> > Is there a possibility to send the file in chunks and get it scanned
> > in the server side in chunks
>
> That would depend on the HTTP scanning service software. Clam AV needs
> the whole file at once to scan it, but the HTTP scanning service may be
> able to upload in chunks and reassemble it before sending it to Clam AV.
>
>
> >
> > I observed that there is a INSTREAM command in clamd for this purpose
> > and also there is a 4GB size limit.
> > https://linux.die.net/man/8/clamd 
>
> INSTREAM basically lets you send a file to clamd, it saves it as a
> temporary file, and then scans it, then deletes it. It lets you scan
> files that don't exist on the same computer as the clamd daemon without
> having to set up network shares etc. So, all the limits (eg the 4GB
> limit) which apply to normal files also apply to INSTREAM
>
> --
>
> Paul
> Paul Smith Computer Services
> supp...@pscs.co.uk - 01484 855800
>
>
> --
>
>
> Paul Smith Computer Services
> Tel: 01484 855800
> Vat No: GB 685 6987 53
>
> Sign up for news & updates at http://www.pscs.co.uk/go/subscribe
>
> ___
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Scanning a large file through HTTP

[Paul Smith via clamav-users]

On 07/04/2021 15:38, Saurav Sarkar via clamav-users wrote:


We have files like CAD files which can go in GBs and want to send to 
this malware scanning service.


Why are you scanning CAD files?

Can your CAD files contain arbitrary executable code which is blindly 
executed by the CAD software? If not, there's no reason to scan them? If 
they can, then I'd consider getting different CAD software...



Is there a possibility to send the file in chunks and get it scanned 
in the server side in chunks


That would depend on the HTTP scanning service software. Clam AV needs 
the whole file at once to scan it, but the HTTP scanning service may be 
able to upload in chunks and reassemble it before sending it to Clam AV.





I observed that there is a INSTREAM command in clamd for this purpose 
and also there is a 4GB size limit.

https://linux.die.net/man/8/clamd 


INSTREAM basically lets you send a file to clamd, it saves it as a 
temporary file, and then scans it, then deletes it. It lets you scan 
files that don't exist on the same computer as the clamd daemon without 
having to set up network shares etc. So, all the limits (eg the 4GB 
limit) which apply to normal files also apply to INSTREAM


--

Paul
Paul Smith Computer Services
supp...@pscs.co.uk - 01484 855800


--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Scanning a large file through HTTP

[G.W. Haywood via clamav-users]

Hi there,

On Wed, 7 Apr 2021, Saurav Sarkar via clamav-users wrote:


We are using a HTTP enabled malware scanning service based on Clam AV.


Perhaps you will get better answers if you address your questions to
the supplier of this service.


We have files like CAD files which can go in GBs and want to send to this
malware scanning service.


Does the service which you are using permit that?


Is there a possibility to send the file in chunks and get it scanned in the
server side in chunks.


Again you should ask your service because we on the mailing list know
nothing about it.  I imagine that it might be possible, but I would
also guess that it would be pointless for your stated purpose.


I observed that there is a INSTREAM command in clamd for this purpose


The clamd 'man' page doesn't exactly say that.  And assuming that this
is related to your use of the service, do you know that your service
actually uses clamd?

The INSTREAM command is available so that you can send a stream of
data to the scanner instead of telling it to scan some file.  If you
will read the clamd.conf 'man' page you will see that the stream of
data must not exceed the value of the configured 'StreamMaxLength'.
The default for that option is 25 Megabytes, a lot less than the GBs
that you're talking about.  If the maximum is exceeded by the length
of the data stream sent after the INSTREAM command, clamd will return
an error and the scan will fail.  If I were running a Web service of
the sort you've dfescribed I'd be very cautious about increasing the
default StreamMaxLength because of the potential for abuse.


and also there is a 4GB size limit.


A number of limits depend on the configuration, and can be much less
than that.


Read somewhere


Where?


the full file size is mapped to memory.


I do not know what that means.

The scanner will use whatever memory is available to it.  It needs
around 1Gbyte for the current 'official' databases, and it can use
considerably more than that if you add some of the various third-party
datasases.  But this memory is used to store signatures (or rather the
compiled versions of them, which is what takes the time to start clamd
or clamscan), not to store the data being scanned.  It is not easy to
predict how much memory will be used to scan a particular data stream.


Is it the case for INSTREAM command also ?


See my previous answer.


If it is the case then even if chunking is supported then the server side
must have at least 4GB of memory.


Somewhere along your chain of logic you seem to have left me behind,
but I would recommend at least 4GB of memory for anything which will
be running the ClamAV scanner unless the user knows what he's doing.

--

73,
Ged.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Scanning a large file through HTTP

[Saurav Sarkar via clamav-users]

Hi All,

We are using a HTTP enabled malware scanning service based on Clam AV.

The service is made something like this
https://github.com/solita/clamav-rest

We have files like CAD files which can go in GBs and want to send to this
malware scanning service.

Is there a possibility to send the file in chunks and get it scanned in the
server side in chunks.

I observed that there is a INSTREAM command in clamd for this purpose and
also there is a 4GB size limit.
https://linux.die.net/man/8/clamd


Read somewhere the full file size is mapped to memory. Is it the case for
INSTREAM command also ?

If it is the case then even if chunking is supported then the server side
must have at least 4GB of memory.

Best Regards,
Saurav

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV 0.103.1 on RHEL 6.7 x32

[G.W. Haywood via clamav-users]

Hi there,

On Wed, 7 Apr 2021, Sorin Petrut Niculae via clamav-users wrote:


A couple of things that I forgot to mentioned:

1.  I cannot upgrade the system, I know that this version of rhel is
EoL but is impossible to upgrade it due too client policy.


Tell the client he's fired? :)


2.  Is a offline system, is impossible to connect it to internet
due to client policy.  ...


Does client policy state, under these circumstances, what are the
threats against which ClamAV is expected to protect?


So the questions are:

1.  I need to do something special when I download the DDBB manually
and copy it on the clamav folder ?


As far as possible manual downloads should be restricted to testing.
There's nothing special about downloading, except that there may be
restrictions imposed because of ongoing abuse of the download servers
- so you need to use one of the two documented methods to fetch the
database (or you risk your IP being blocked by Cloudflare).  In any
case you can't expect manual efforts to be as reliable as this kind of
thing needs to be.  Perhaps you should be looking at

https://www.clamav.net/documents/private-local-mirrors

which specifically mentions for example the case of a scanner which is
not permitted Internet access and the permitted download methods.


2.  The DDBB is the same for both architecture x32 and x64 or is
different ?


Exactly the same.  The databases are identical for all architectures,
and in them there are signatures for threats to all architectures.  Of
course the ClamaV code must be built separately for each architecture
on which it will run.  That may be done by the ClamAV team (for a few
architectures) in which case there will be binaries available that you
can download, although for less mainstream architectures you may need
either to build it yourself or to find/get it built elsewhere.  In the
latter cases you may need to be cautious; criminals will often try to
get you to install their own doctored versions of well-known software.


 3.  Which can be the source of the error "Malformed database" ?


$ grep -r -C1 'Malformed database' ./clamav-0.103.1 | grep -v 'Binary file'
--
libclamav/others.c-case CL_EMALFDB:
libclamav/others.c:return "Malformed database";
libclamav/others.c-case CL_ECVD:

That message appears in one place in the ClamAV code, it associates a
text string with the CL_EMALFDB flag which is used internally.  Most of
the occurrences of this flag are in statements like

return CL_EMALFDB;

Which means that a section of the code has deduced that the database
is malformed and cannot be used.  To count the number of occurrences
of this flag:

$ grep -r 'CL_EMALFDB' ~/clamav-0.103.1 | grep -v 'Binary file' | wc -l 
288


There are almost three hundred places in the code which might give
rise to that error.  More information would be needed to be able to
narrow it down.  Knowing which of the ClamAV tools produces it, how
exactly it was persuaded to do that, and exactly what databases were
in use at the time would be a good start.

Here are the 'sigtool' test results on my local databases today:

$ sigtool -i /EXPORTS/clamav/databases/main.cvd 
File: /EXPORTS/clamav/databases/main.cvd

Build time: 25 Nov 2019 08:56 -0500
Version: 59
Signatures: 4564902
Functionality level: 60
Builder: sigmgr
MD5: af6f9a95b19fcce8be2c84bde73b5db6
Digital signature: 
VeNZg/gIMosAkDvAv5U4IezNpJzBILxyOIbrsmFVrQRpFEULdbLbRK1csHyDHu9nTzNOwX7fiDiZkM7eOoaF91JNtL0Hju3SHrzWzY0K6nV6NV2+y+RohIpjvHJDx98ViAuCou/b2O7ryjD1u31jhBwwckGU+DwdIzmjXNJu3Jb
Verification OK.

$ sigtool -i /EXPORTS/clamav/databases/daily.cld 
File: /EXPORTS/clamav/databases/daily.cld

Build time: 06 Apr 2021 07:06 -0400
Version: 26132
Signatures: 3968913
Functionality level: 63
Builder: raynman
Verification OK.

$ sigtool -i /EXPORTS/clamav/databases/bytecode.cld 
File: /EXPORTS/clamav/databases/bytecode.cld

Build time: 08 Mar 2021 10:21 -0500
Version: 333
Signatures: 92
Functionality level: 63
Builder: awillia2
Verification OK.

A very quick and easy check to test that the databases which you're
using are properly installed is running 'md5sum' on them.  Here are
the three values here today for three primary ('official') databases:

ged@pi4b530214:/EXPORTS/clamav/databases $ md5sum main.cvd daily.cld bytecode.cld 
0fdc6dc2135ebeb8289cca7bd6a69c43  main.cvd

61cd5237377bd670c91c1afcf94b2c51  daily.cld
bbdce24385bd4d715fc2d81d156ae0bb  bytecode.cld

Note that the md5sum produced on the raw file is not the same as that
produced by sigtool.  Obviously sigtools is looking at something else,
I don't know what nor why.

As you can see, only one of the official database files changes often:

pi4b:/EXPORTS/clamav/databases $ l main.cvd daily.cld bytecode.cld 
-rw-r--r-- 1 clamav clamav 117859675 Feb  5  2020 main.cvd

-rw-r--r-- 1 clamav clamav   1438720 Mar  8 18:57 bytecode.cld
-rw-r--r-- 1 clamav clamav 321211904 Apr  6 14:46 daily.cld

Can you confirm that you have these files, that 

Re: [clamav-users] ClamAV 0.103.1 on RHEL 6.7 x32

[Sorin Petrut Niculae via clamav-users]

Good morning,


Thanks to all for your help.


A couple of things that I forgot to mentioned:

  1.  I cannot upgrade the system, I know that this version of rhel is EoL but 
is impossible to upgrade it due too client policy.
  2.  Is a offline system, is impossible to connect it to internet due to 
client policy.
  3.  The SSL connect error that I've got it is on a VM that I replicated on my 
PC to download the DDBB and after that copy it on the main server.
  4.  I've try to download manually the DDBB from 
https://database.clamav.net/daily.cvd /main.cvd /bytecode.cvd and copy it on 
the main server but I got the error "Malformed database"

So the questions are:

  1.  I need to do something special when I download the DDBB manually and copy 
it on the clamav folder ?
  2.  The DDBB is the same for both architecture x32 and x64 or is different ?
  3.  Which can be the source of the error "Malformed database" ?


Regards.



[cid:83b459fd-eb52-4bed-8eea-534c8aebf8e3]


Sorin Petrut Niculae
GMV Aerospace and Defence

Administrador de sistemas e infraestructura /
IT systems & infrastructure administrator


GMV
Isaac Newton, 11
P.T.M. Tres Cantos
28760 Madrid

España
Tel. +34 91 807 21 00
Fax +34 91 807 21 99
www.gmv.com

[cid:7b3ba95d-c9cd-40ba-979f-b3872f88eb4f]


[cid:9f6759dc-f6b6-4ae9-8d60-2c7c683951c3]


[cid:82e2b7d3-b68f-419a-b4b4-279229a5e393]


[cid:93d33053-42e2-4233-8278-f06ebf026d74]


[cid:e39fc9d3-38aa-4be1-9729-dac1be30fa82]


[cid:065492af-15a4-4216-9896-4184d50de762]



P Please consider the environment before printing this e-mail.



De: clamav-users  en nombre de Eero 
Volotinen 
Enviado: martes, 6 de abril de 2021 21:44:17
Para: ClamAV users ML
Cc: Arjen de Korte
Asunto: Re: [clamav-users] ClamAV 0.103.1 on RHEL 6.7 x32

Well,

For some reasons they backported tls 1.2 support to openssl 1.0.1-e-something 
that is shipped with RHEL 6.5(+).

Eero

On Tue, Apr 6, 2021 at 9:34 PM Arjen de Korte via clamav-users 
mailto:clamav-users@lists.clamav.net>> wrote:
Citeren Eero Volotinen mailto:eero.voloti...@iki.fi>>:

> Well redhat backports some fixes usually as you can see:
>
> https://access.redhat.com/blogs/766093/posts/1976123

Backporting fixes/features, doesn't make openssl-1.0.1 equivalent to
openssl-1.0.2. If that was the case, it wouldn't make sense to
backport the fixes/features: you would just update to the newer version.


___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

P Please consider the environment before printing this e-mail.

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml