Re: [clamav-users] OnAccess renders system unusable in ~24h
Hello again, On Tue, 24 Sep 2019, Tim Stubbs wrote: What kinds of threats do you care about? If for example you're not expecting your Linux boxes to be attacked by Windows malware you could reduce the size of the ClamAV databases very significantly which might improve scanning performance. Sorry could you point me in the right direction for how to do this? good hasn't helped me so far? Check the archives for this list, it's been discussed recently how to use an empty database. That might be a good start. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] OnAccess renders system unusable in ~24h
Hi > What kinds of threats do you care about? If for example you're not > expecting your Linux boxes to be attacked by Windows malware you could > reduce the size of the ClamAV databases very significantly which might > improve scanning performance. Sorry could you point me in the right direction for how to do this? good hasn't helped me so far? thanks Tim -Original Message- From: Tim Stubbs Reply-To: ClamAV users ML To: clamav-users@lists.clamav.net Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h Date: Tue, 24 Sep 2019 15:12:16 + Hi, thanks for the quick response. We have been asked to run Realtime scans as part of our PCI requirement, otherwise I would agree with you 100%. that wasn't the best worst, example i had a VM this morning 56 49 47, which went back to 1 when I stopped clamd. I do however have other VMs where (with the same config) I've never had an issue. thanks for that, yes we are a linux house, ill try reducing the DB & I will reduce the paths and test that. failing that I will take a look at v102 thank you, I'll update with my findings. Tim -Original Message- From: G.W. Haywood via clamav-users < clamav-users@lists.clamav.net > Reply-To: ClamAV users ML < clamav-users@lists.clamav.net > To: ClamAV users ML < clamav-users@lists.clamav.net > Cc: G.W. Haywood < cla...@jubileegroup.co.uk > Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h Date: Tue, 24 Sep 2019 15:39:22 +0100 Hi there, On Tue, 24 Sep 2019, Tim Stubbs wrote: > I am running clamd with OnAccess enabled, however its causing the > load > on the systems to make them almost unusable within about 24hours. This may be true, but I'd want to know that the suspicion is justified (and front and centre I personally think scanning most Linux boxes with ClamAV is a waste of CPU). > as you can see sys is at 98% ... No, I see CPU 27% idle and three clamd processes doing nothing. But I do see a load average of around seven. On my dual CPU 2.7GHz Opterons I routinely see an average of that sort of figure when they do backups for a bunch of other machines, and Nagios will whine about it when it gets over 8, but I don't usually worry about it until it gets into the double digits. > it seem clamd is stopping other > applications from processing somehow. cannot find anything in the > logs. > not sure what debugging would be helpful? any advice would be helpful > here? My immediate reaction is - if the suspicion is found to be justified - that you should try to reduce, initially to a bare minimum, the amount of work which you're asking the machine to do. > OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \ > "/opt", "/media", "/mnt" For example you could remove most of the directories from this list to see if it helps. There are other things you might try, like limiting the number of threads. But again, I don't see anything in your 'top' output which tells me that clamd is heavily loading your machine. What kinds of threats do you care about? If for example you're not expecting your Linux boxes to be attacked by Windows malware you could reduce the size of the ClamAV databases very significantly which might improve scanning performance. ClamaV version 0.102 has just been released as a candidate for testing and I've been running it for some time before the RC was released. It contains some significant improvements for on-access scanning and, if you do intend to persevere with on-access scanning, I'd recommend that you install the latest version from the source. -- Thank you, Tim [Winner of the 2018 Consumer Credit Awards] ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml -- Thank you, Tim [Winner of the 2018 Consumer Credit Awards] ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] OnAccess renders system unusable in ~24h
Hi, thanks for the quick response. We have been asked to run Realtime scans as part of our PCI requirement, otherwise I would agree with you 100%. that wasn't the best worst, example i had a VM this morning 56 49 47, which went back to 1 when I stopped clamd. I do however have other VMs where (with the same config) I've never had an issue. thanks for that, yes we are a linux house, ill try reducing the DB & I will reduce the paths and test that. failing that I will take a look at v102 thank you, I'll update with my findings. Tim -Original Message- From: G.W. Haywood via clamav-users Reply-To: ClamAV users ML To: ClamAV users ML Cc: G.W. Haywood Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h Date: Tue, 24 Sep 2019 15:39:22 +0100 Hi there, On Tue, 24 Sep 2019, Tim Stubbs wrote: > I am running clamd with OnAccess enabled, however its causing the > load > on the systems to make them almost unusable within about 24hours. This may be true, but I'd want to know that the suspicion is justified (and front and centre I personally think scanning most Linux boxes with ClamAV is a waste of CPU). > as you can see sys is at 98% ... No, I see CPU 27% idle and three clamd processes doing nothing. But I do see a load average of around seven. On my dual CPU 2.7GHz Opterons I routinely see an average of that sort of figure when they do backups for a bunch of other machines, and Nagios will whine about it when it gets over 8, but I don't usually worry about it until it gets into the double digits. > it seem clamd is stopping other > applications from processing somehow. cannot find anything in the > logs. > not sure what debugging would be helpful? any advice would be helpful > here? My immediate reaction is - if the suspicion is found to be justified - that you should try to reduce, initially to a bare minimum, the amount of work which you're asking the machine to do. > OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \ > "/opt", "/media", "/mnt" For example you could remove most of the directories from this list to see if it helps. There are other things you might try, like limiting the number of threads. But again, I don't see anything in your 'top' output which tells me that clamd is heavily loading your machine. What kinds of threats do you care about? If for example you're not expecting your Linux boxes to be attacked by Windows malware you could reduce the size of the ClamAV databases very significantly which might improve scanning performance. ClamaV version 0.102 has just been released as a candidate for testing and I've been running it for some time before the RC was released. It contains some significant improvements for on-access scanning and, if you do intend to persevere with on-access scanning, I'd recommend that you install the latest version from the source. -- Thank you, Tim [Winner of the 2018 Consumer Credit Awards] ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] OnAccess renders system unusable in ~24h
Hi there, On Tue, 24 Sep 2019, Tim Stubbs wrote: I am running clamd with OnAccess enabled, however its causing the load on the systems to make them almost unusable within about 24hours. This may be true, but I'd want to know that the suspicion is justified (and front and centre I personally think scanning most Linux boxes with ClamAV is a waste of CPU). as you can see sys is at 98% ... No, I see CPU 27% idle and three clamd processes doing nothing. But I do see a load average of around seven. On my dual CPU 2.7GHz Opterons I routinely see an average of that sort of figure when they do backups for a bunch of other machines, and Nagios will whine about it when it gets over 8, but I don't usually worry about it until it gets into the double digits. it seem clamd is stopping other applications from processing somehow. cannot find anything in the logs. not sure what debugging would be helpful? any advice would be helpful here? My immediate reaction is - if the suspicion is found to be justified - that you should try to reduce, initially to a bare minimum, the amount of work which you're asking the machine to do. OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \ "/opt", "/media", "/mnt" For example you could remove most of the directories from this list to see if it helps. There are other things you might try, like limiting the number of threads. But again, I don't see anything in your 'top' output which tells me that clamd is heavily loading your machine. What kinds of threats do you care about? If for example you're not expecting your Linux boxes to be attacked by Windows malware you could reduce the size of the ClamAV databases very significantly which might improve scanning performance. ClamaV version 0.102 has just been released as a candidate for testing and I've been running it for some time before the RC was released. It contains some significant improvements for on-access scanning and, if you do intend to persevere with on-access scanning, I'd recommend that you install the latest version from the source. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
[clamav-users] OnAccess renders system unusable in ~24h
Hi, I am running clamd with OnAccess enabled, however its causing the load on the systems to make them almost unusable within about 24hours. as you can see sys is at 98%, it seem clamd is stopping other applications from processing somehow. cannot find anything in the logs. not sure what debugging would be helpful? any advice would be helpful here? # top -bH -d 5 -n 49 -p 134865 top - 12:25:11 up 60 days, 4:29, 1 user, load average: 7.19, 6.92, 6.71 Threads: 3 total, 0 running, 3 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.7 us, 70.3 sy, 0.0 ni, 27.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 5908584 total, 251604 free, 4134244 used, 1522736 buff/cache KiB Swap: 6291452 total, 5947900 free, 343552 used. 1004892 avail Mem PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 134865 root 35 15 1114568 745700612 S 0.0 12.6 3:20.05 clamd 134867 root 20 0 1114568 745700612 S 0.0 12.6 0:00.00 clamd 134868 root 20 0 1114568 745700612 S 0.0 12.6 1:23.39 clamd # clamconf Checking configuration files in /etc Config file: clamd.d/scan.conf -- AlertExceedsMax disabled PreludeEnable disabled PreludeAnalyzerName disabled LogFile = "/var/log/clamav/clamav.log" LogFileUnlock = "yes" LogFileMaxSize = "1048576" LogTime = "yes" LogClean disabled LogSyslog disabled LogFacility = "LOG_LOCAL6" LogVerbose disabled LogRotate disabled ExtendedDetectionInfo = "yes" PidFile = "/var/run/clamd.scan/clamd.pid" TemporaryDirectory disabled DatabaseDirectory = "/var/lib/clamav" OfficialDatabaseOnly disabled LocalSocket = "/var/run/clamd.scan/clamd.sock" LocalSocketGroup = "virusgroup" LocalSocketMode = "666" FixStaleSocket = "yes" TCPSocket disabled TCPAddr disabled MaxConnectionQueueLength = "200" StreamMaxLength = "26214400" StreamMinPort = "1024" StreamMaxPort = "2048" MaxThreads = "10" ReadTimeout = "180" CommandReadTimeout = "30" SendBufTimeout = "200" MaxQueue = "100" IdleTimeout = "30" ExcludePath disabled MaxDirectoryRecursion = "20" FollowDirectorySymlinks disabled FollowFileSymlinks disabled CrossFilesystems = "yes" SelfCheck = "600" DisableCache disabled VirusEvent disabled ExitOnOOM = "yes" AllowAllMatchScan = "yes" Foreground disabled Debug disabled LeaveTemporaryFiles disabled User = "root" Bytecode = "yes" BytecodeSecurity = "TrustSigned" BytecodeTimeout = "5000" BytecodeUnsigned disabled BytecodeMode = "Auto" DetectPUA disabled ExcludePUA disabled IncludePUA disabled ScanPE = "yes" ScanELF = "yes" ScanMail = "yes" ScanPartialMessages disabled PhishingSignatures = "yes" PhishingScanURLs = "yes" HeuristicAlerts = "yes" HeuristicScanPrecedence disabled StructuredDataDetection disabled StructuredMinCreditCardCount = "3" StructuredMinSSNCount = "3" StructuredSSNFormatNormal = "yes" StructuredSSNFormatStripped disabled ScanHTML = "yes" ScanOLE2 = "yes" AlertBrokenExecutables disabled AlertEncrypted disabled AlertEncryptedArchive disabled AlertEncryptedDoc disabled AlertOLE2Macros disabled AlertPhishingSSLMismatch disabled AlertPhishingCloak disabled AlertPartitionIntersection disabled ScanPDF = "yes" ScanSWF = "yes" ScanXMLDOCS = "yes" ScanHWP3 = "yes" ScanArchive = "yes" ForceToDisk disabled MaxScanTime disabled MaxScanSize = "104857600" MaxFileSize = "26214400" MaxRecursion = "16" MaxFiles = "1" MaxEmbeddedPE = "10485760" MaxHTMLNormalize = "10485760" MaxHTMLNoTags = "2097152" MaxScriptNormalize = "5242880" MaxZipTypeRcg = "1048576" MaxPartitions = "50" MaxIconsPE = "100" MaxRecHWP3 = "16" PCREMatchLimit = "10" PCRERecMatchLimit = "2000" PCREMaxFileSize = "26214400" ScanOnAccess = "yes" OnAccessMountPath disabled OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", "/opt", "/media", "/mnt" OnAccessExcludePath disabled OnAccessExcludeRootUID = "yes" OnAccessExcludeUID disabled OnAccessMaxFileSize = "5242880" OnAccessDisableDDD disabled OnAccessPrevention disabled OnAccessExtraScanning = "yes" DevACOnly disabled DevACDepth disabled DevPerformance disabled DevLiblog disabled DisableCertCheck disabled AlgorithmicDetection = "yes" BlockMax disabled PhishingAlwaysBlockSSLMismatch disabled PhishingAlwaysBlockCloak disabled PartitionIntersection disabled OLE2BlockMacros disabled ArchiveBlockEncrypted disabled Config file: freshclam.conf --- LogFileMaxSize = "2097152" LogTime = "yes" LogSyslog = "yes" LogFacility = "LOG_LOCAL6" LogVerbose disabled LogRotate = "yes" PidFile = "/var/run/freshclam.pid" DatabaseDirectory = "/var/lib/clamav" Foreground disabled Debug disabled UpdateLogFile = "/var/log/freshclam.log" DatabaseOwner = "clamupdate" Checks = "24" DNSDatabaseInfo = "current.cvd.clamav.net" DatabaseMirror = "172.16.22.8" PrivateMirror disabled MaxAttempts = "5" ScriptedUpdates = "yes" TestDatabases = "yes" CompressLocalDatabase disabled ExtraDatabase disabled DatabaseCustomURL disabled HTTPProxyServer disabled HTTPProxyPort disabled HTTPProxyUsername disabled
