Re: [clamav-users] About scanning files larger than 2 GB in size
Thank you for the information. I understand that files larger than 2GB will be treated as clean files without the "AlertExceedsMax yes" setting. I want to wait for the day when I can properly scan files larger than 2GB. T.O On Thu, 26 Jan 2023 22:27:12 + "Micah Snyder \(micasnyd\) via clamav-users" wrote: > > Tsutomu Oyamada asked what actually happens when a large file is > > scanned, not why the limit is there. > > The default behavior is to treat the file as clean if any of the scan limits > are exceeded (scan time, scan size, file size, etc). > > If you want an alert if the limits are exceeded, then you can use the > following options: > For ClamD, set "AlertExceedsMax yes" in the "clamd.conf" file. > For ClamScan, use the "--alert-exceeds-max" option on the command line. > > This will cause clamav to report one of the following signatures when the > limits are exceeded: > - Heuristics.Limits.Exceeded.MaxFileSize > - Heuristics.Limits.Exceeded.MaxScanSize > - Heuristics.Limits.Exceeded.MaxFiles > - Heuristics.Limits.Exceeded.MaxRecursion > - Heuristics.Limits.Exceeded.MaxScanTime > - Heuristics.Limits.Exceeded.EmailLineFoldcnt > - Heuristics.Limits.Exceeded.EmailHeaderBytes > - Heuristics.Limits.Exceeded.EmailHeaders > - Heuristics.Limits.Exceeded.EmailMIMEPartsPerMessage > - Heuristics.Limits.Exceeded.EmailMIMEArguments > and possibly more with the "Heuristics.Limits.Exceeded." prefix. > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > From: Andrew C Aitchison > Sent: Wednesday, January 25, 2023 10:59 PM > To: Micah Snyder (micasnyd) via clamav-users > Cc: Micah Snyder (micasnyd) > Subject: Re: [clamav-users] About scanning files larger than 2 GB in size > > On Thu, 26 Jan 2023, Micah Snyder (micasnyd) via clamav-users wrote: > > > Paul is sort-of correct but the 2GB limit isn't artificial as he has > > implied. > > Paul did not answer the original poster's question. > Tsutomu Oyamada asked what actually happens when a large file is > scanned, not why the limit is there. > > > On Sun, 22 Jan 2023 05:40:18 +0900 > > Tsutomu Oyamada wrote: > > > >> How do I set up clamd? > >> Setting MaxFileSize to "0" is unlimited, but internally files > >> larger than 2GB in size cannot be scanned. In this case, do you > >> treat the file as clean without scanning it at all? > > > ClamAV code contains a lot of signed and unsigned 32bit variables > > that must be upgraded to 64bit variables to support larger files. > > Before raising the limit, a tedious audit process must be completed > > to ensure that all variables are upgraded in all modules. We cannot > > simply remove the limit and cross our fingers. > > A static analyzer such as cppcheck, PVS-Studio or the ones built into > gcc and clang may be useful tools in the tedious audit. > > -- > Andrew C. Aitchison Kendal, UK > and...@aitchison.me.uk ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
Re: [clamav-users] About scanning files larger than 2 GB in size
> Tsutomu Oyamada asked what actually happens when a large file is > scanned, not why the limit is there. The default behavior is to treat the file as clean if any of the scan limits are exceeded (scan time, scan size, file size, etc). If you want an alert if the limits are exceeded, then you can use the following options: For ClamD, set "AlertExceedsMax yes" in the "clamd.conf" file. For ClamScan, use the "--alert-exceeds-max" option on the command line. This will cause clamav to report one of the following signatures when the limits are exceeded: - Heuristics.Limits.Exceeded.MaxFileSize - Heuristics.Limits.Exceeded.MaxScanSize - Heuristics.Limits.Exceeded.MaxFiles - Heuristics.Limits.Exceeded.MaxRecursion - Heuristics.Limits.Exceeded.MaxScanTime - Heuristics.Limits.Exceeded.EmailLineFoldcnt - Heuristics.Limits.Exceeded.EmailHeaderBytes - Heuristics.Limits.Exceeded.EmailHeaders - Heuristics.Limits.Exceeded.EmailMIMEPartsPerMessage - Heuristics.Limits.Exceeded.EmailMIMEArguments and possibly more with the "Heuristics.Limits.Exceeded." prefix. Micah Snyder ClamAV Development Talos Cisco Systems, Inc. From: Andrew C Aitchison Sent: Wednesday, January 25, 2023 10:59 PM To: Micah Snyder (micasnyd) via clamav-users Cc: Micah Snyder (micasnyd) Subject: Re: [clamav-users] About scanning files larger than 2 GB in size On Thu, 26 Jan 2023, Micah Snyder (micasnyd) via clamav-users wrote: > Paul is sort-of correct but the 2GB limit isn't artificial as he has implied. Paul did not answer the original poster's question. Tsutomu Oyamada asked what actually happens when a large file is scanned, not why the limit is there. > On Sun, 22 Jan 2023 05:40:18 +0900 > Tsutomu Oyamada wrote: > >> How do I set up clamd? >> Setting MaxFileSize to "0" is unlimited, but internally files >> larger than 2GB in size cannot be scanned. In this case, do you >> treat the file as clean without scanning it at all? > ClamAV code contains a lot of signed and unsigned 32bit variables > that must be upgraded to 64bit variables to support larger files. > Before raising the limit, a tedious audit process must be completed > to ensure that all variables are upgraded in all modules. We cannot > simply remove the limit and cross our fingers. A static analyzer such as cppcheck, PVS-Studio or the ones built into gcc and clang may be useful tools in the tedious audit. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
Re: [clamav-users] About scanning files larger than 2 GB in size
P.S. Do many current commercial AV suites for Windows have this limit? I have no idea. Does anyone else know? Micah Snyder ClamAV Development Talos Cisco Systems, Inc. From: Paul Kosinski Sent: Thursday, January 26, 2023 11:32 AM To: Micah Snyder (micasnyd) ; clamav-users@lists.clamav.net Subject: Re: [clamav-users] About scanning files larger than 2 GB in size I don't think I implied that the 2 GiB limit was "artificial" in the sense of trivial, or made up. I think I very clearly stated that "It's a holdover from when 32-bit numbers were all that CPUs supported" and now "the 2 GiB limit is quite an anachronism". Note that this question has been around for at least 7 years: https://security.stackexchange.com/questions/107132/linux-antivirus-and-files-bigger-than-4gb Clearly, much code review would have to be done. But Linux file I/O interfaces were successfully updated from 32-bit to 64-bit sizes and offsets some years ago, so the infrastructure is there. Also, the analogous Y2038 problem, which requires going from 32 to 64 bit as well (for time-stamps), is being seriously worked on. (And note that the Y2K problem, which was a *much* bigger issue, was indeed fixed.) Paul P.S. Do many current commercial AV suites for Windows have this limit? On Thu, 26 Jan 2023 00:14:27 + "Micah Snyder (micasnyd)" wrote: > Paul is sort-of correct but the 2GB limit isn't artificial as he has implied. > > ClamAV code contains a lot of signed and unsigned 32bit variables that must > be upgraded to 64bit variables to support larger files. Before raising the > limit, a tedious audit process must be completed to ensure that all variables > are upgraded in all modules. We cannot simply remove the limit and cross our > fingers. > > Regards, > Micah > ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
Re: [clamav-users] About scanning files larger than 2 GB in size
I don't think I implied that the 2 GiB limit was "artificial" in the sense of trivial, or made up. I think I very clearly stated that "It's a holdover from when 32-bit numbers were all that CPUs supported" and now "the 2 GiB limit is quite an anachronism". Note that this question has been around for at least 7 years: https://security.stackexchange.com/questions/107132/linux-antivirus-and-files-bigger-than-4gb Clearly, much code review would have to be done. But Linux file I/O interfaces were successfully updated from 32-bit to 64-bit sizes and offsets some years ago, so the infrastructure is there. Also, the analogous Y2038 problem, which requires going from 32 to 64 bit as well (for time-stamps), is being seriously worked on. (And note that the Y2K problem, which was a *much* bigger issue, was indeed fixed.) Paul P.S. Do many current commercial AV suites for Windows have this limit? On Thu, 26 Jan 2023 00:14:27 + "Micah Snyder (micasnyd)" wrote: > Paul is sort-of correct but the 2GB limit isn't artificial as he has implied. > > ClamAV code contains a lot of signed and unsigned 32bit variables that must > be upgraded to 64bit variables to support larger files. Before raising the > limit, a tedious audit process must be completed to ensure that all variables > are upgraded in all modules. We cannot simply remove the limit and cross our > fingers. > > Regards, > Micah > ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
Re: [clamav-users] About scanning files larger than 2 GB in size
Paul is sort-of correct but the 2GB limit isn't artificial as he has implied. ClamAV code contains a lot of signed and unsigned 32bit variables that must be upgraded to 64bit variables to support larger files. Before raising the limit, a tedious audit process must be completed to ensure that all variables are upgraded in all modules. We cannot simply remove the limit and cross our fingers. Regards, Micah Micah Snyder ClamAV Development Talos Cisco Systems, Inc. From: clamav-users on behalf of Paul Kosinski via clamav-users Sent: Saturday, January 21, 2023 8:02 PM To: clamav-users@lists.clamav.net Cc: Paul Kosinski Subject: Re: [clamav-users] About scanning files larger than 2 GB in size On Sun, 22 Jan 2023 05:40:18 +0900 Tsutomu Oyamada wrote: > How do I set up clamd? > Setting MaxFileSize to "0" is unlimited, but internally files larger than 2GB > in size cannot be scanned. > In this case, do you treat the file as clean without scanning it at all? I've complained about the 2 GiB limit now and then for several years. It's a holdover from when 32-bit numbers were all that CPUs supported, and lots of code used C's signed "int" for file size and offset. Way back in 1996, FAT32 had this limit, but even it was extended to 4 GiB (via unsigned 32-bit numbers) when Large File Support was added. These days, media files are often bigger than 2 GiB, as are some archive files (esp. disk images). Furthermore, almost all recent versions of standard OSes are 64-bit, and very few 32-bit CPUs are even being sold. In other words, the 2 GiB limit is quite an anachronism. ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
Re: [clamav-users] About scanning files larger than 2 GB in size
On Sun, 22 Jan 2023 05:40:18 +0900 Tsutomu Oyamada wrote: > How do I set up clamd? > Setting MaxFileSize to "0" is unlimited, but internally files larger than 2GB > in size cannot be scanned. > In this case, do you treat the file as clean without scanning it at all? I've complained about the 2 GiB limit now and then for several years. It's a holdover from when 32-bit numbers were all that CPUs supported, and lots of code used C's signed "int" for file size and offset. Way back in 1996, FAT32 had this limit, but even it was extended to 4 GiB (via unsigned 32-bit numbers) when Large File Support was added. These days, media files are often bigger than 2 GiB, as are some archive files (esp. disk images). Furthermore, almost all recent versions of standard OSes are 64-bit, and very few 32-bit CPUs are even being sold. In other words, the 2 GiB limit is quite an anachronism. ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
[clamav-users] About scanning files larger than 2 GB in size
How do I set up clamd? Setting MaxFileSize to "0" is unlimited, but internally files larger than 2GB in size cannot be scanned. In this case, do you treat the file as clean without scanning it at all? ___ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
