The verbiage of the warning may be a little misleading. It doesn't tell you how much has been scanned prior to the call into cli_scanxz(). It is likely that it scanned additional data prior to the call to cli_scanxz(), which counts towards the scan limit.
If you re-scan with the --debug flag set, it should report an additional debug-level message near the warning you saw, in the form of: "scansize exceeded (initial: %lu, consumed: %lu, needed: %lu)" initial: should be the limit (3999M). consumed: should be the amount already scanned. needed: should be the amount of data that the cli_scanxz() function has on hand and would like to scan. If you run test this, I would be curious to know what is reported. That said, your comments raises the age old question of if or how ClamAV should report that it is unable to finish scanning a file when limits are exceeded. Most clamav scanning code will not report a warning at all like you see with the xz scanner, and the only way to determine if limits have been exceeded is to run with --blockmax or --alert-exceeds-max (v0.101+). Micah Snyder ClamAV Development Talos Cisco Systems, Inc. On Dec 7, 2018, at 3:41 PM, Albert o <[email protected]<mailto:[email protected]>> wrote: There is something I don't get this my clamd.conf user@debian:~/Downloads/clamav-0.101.0$ cat /etc/clamav/clamd.conf #Automatically Generated by clamav-daemon postinst #To reconfigure clamd run #dpkg-reconfigure clamav-daemon #Please read /usr/share/doc/clamav-daemon/README.Debian.gz for details LocalSocket /var/run/clamav/clamd.ctl FixStaleSocket true LocalSocketGroup clamav LocalSocketMode 666 # TemporaryDirectory is not set to its default /tmp here to make overriding # the default with environment variables TMPDIR/TMP/TEMP possible User clamav ScanMail true ScanArchive true ArchiveBlockEncrypted false MaxDirectoryRecursion 100 FollowDirectorySymlinks true FollowFileSymlinks true ReadTimeout 180 MaxThreads 12 MaxConnectionQueueLength 2000 LogSyslog true LogRotate true LogFacility LOG_LOCAL6 LogClean false LogVerbose false PreludeEnable no PreludeAnalyzerName ClamAV DatabaseDirectory /var/lib/clamav OfficialDatabaseOnly false SelfCheck 3600 Foreground true Debug 0 PidFile true MaxEmbeddedPE 3999M ScanOLE2 true ScanPDF true ScanHTML true MaxHTMLNormalize 3999M MaxHTMLNoTags 3999M MaxScriptNormalize 3999M MaxZipTypeRcg 3999M ScanSWF true DetectBrokenExecutables false ExitOnOOM false LeaveTemporaryFiles false AlgorithmicDetection true ScanELF true ScanPE true IdleTimeout 30 CrossFilesystems true PhishingSignatures true PhishingScanURLs true ExtendedDetectionInfo true PhishingAlwaysBlockSSLMismatch false PhishingAlwaysBlockCloak false PartitionIntersection false DetectPUA 1 BlockMax 1 OLE2BlockMacros 1 ArchiveBlockEncrypted 1 ScanPartialMessages 1 PartitionIntersection 1 HeuristicScanPrecedence 1 StructuredDataDetection 1 CommandReadTimeout 5 SendBufTimeout 400 MaxQueue 100 ExtendedDetectionInfo 1 OLE2BlockMacros 1 ScanOnAccess true AllowAllMatchScan true ForceToDisk false DisableCertCheck false DisableCache false MaxScanSize 3999M MaxFileSize 3999M MaxRecursion 160 MaxFiles 20000 MaxPartitions 10000 MaxIconsPE 10000 PCREMatchLimit 10000 PCRERecMatchLimit 10000 PCREMaxFileSize 3999M ScanXMLDOCS true ScanHWP3 true MaxRecHWP3 16 StreamMaxLength 3999M LogFile /var/log/clamav/clamav.log LogTime true LogFileUnlock true Bytecode true BytecodeSecurity TrustSigned BytecodeTimeout 60000 BytecodeUnsigned 1 ScanOnAccess 1 OnAccessMaxFileSize 0 OnAccessMountPath / OnAccessIncludePath / OnAccessMaxFileSize 0 OnAccessPrevention 1 OnAccessDisableDDD 1 but I still get LibClamAV Warning: cli_msxml_parse_document: encountered issue in parsing xml document LibClamAV Warning: cli_scanxz: decompress file size exceeds limits - only scanning 27262976 bytes What am I missing? should't all the files >4G be scanned? On Tue, Dec 4, 2018 at 4:42 PM Ángel <[email protected]<mailto:[email protected]>> wrote: On 2018-12-03 at 09:58 -0800, Dennis Peterson wrote: If it is a big concern you can use the split command to create "splits" of the suspect file. Split accepts various size arguments (bytes, lines...) and will create as many files as it takes to split the entire large file. These can be scanned individually and discarded when done. There is a risk of a split happening in the middle of a section that might match a signature but that is small. A work around is to split a file, scan it, delete the splits, then split it a second time using a different split size and repeat the scan. This is obviously tedious and works best on static files. There's always a way if you don't mind the effort. It is easily scriptable. dp Splitting a file will probably make chunks other than the first to appear as random bytes, rather than having the correct filetype, thus making some signatures not to be applied. (the first chunk will _probably_ be detected properly, still splitting can make it miss what would be found on the full size, eg. splitting a zip file will lose its central directory...) Signatures are generally more complex than looking for a certain substring... Best regards _______________________________________________ clamav-users mailing list [email protected]<mailto:[email protected]> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml _______________________________________________ clamav-users mailing list [email protected]<mailto:[email protected]> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
_______________________________________________ clamav-users mailing list [email protected] http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
