Thanks for the report, James, and help with the analysis, Al. Hopefully we all can figure out a fix for this. We also received a bug report about a month ago reporting similar behavior with certain yara rule sets. https://bugzilla.clamav.net/show_bug.cgi?id=12077 It looks as though the bug reporter (Max) had the same error line in his clamd log:
LibClamAV Error: yyerror(): /var/lib/clamav/maldoc_somerules.yar line 235 undefined identifier "uint32be" That may be a good first clue, although maybe not since it is failing to load a rule and it sounds like it the crash occurs much later and could be an issue with how it handles one of the signatures it actually /does/ load. Hard to say without some debugging, further investigation. Micah Snyder ClamAV Development Talos Cisco Systems, Inc. On May 10, 2018, at 12:30 AM, James Brown <[email protected]<mailto:[email protected]>> wrote: Yeah, it should just log the error. I put back EMAIL_Cryptowall.yar back in to test and restarted clamd. It didn’t complain about it. The clamav-unofficial-sigs script had since downloaded these yara files: winnow_malwware.yara CVE-2015-5119.yar CVE-2013-0074.yar CVE-2013-0422.yar CVE-2010-0887.yar CVE-2010-1297.yar CVE-2010-0805.yar Maldoc_Hidden_PE_file.yar maldoc_somerules.yar EK_Zerox88.yar EK_Zeus.yar EK_Sakura.yar EK_ZeroAcces.yar EK_Fragus.yar EK_Phoenix.yar EK_BleedingLife.yar EK_Crimepack.yar EK_Eleonore.yar EK_Angler.yar EK_Blackhole.yar And clamd starts with: LibClamAV Error: yyerror(): /usr/local/clamav/maldoc_somerules.yar line 235 undefined identifier "uint32be" LibClamAV Warning: cli_loadyara: failed to parse or load 1 yara rules from file /usr/local/clamav/maldoc_somerules.yar, successfully loaded 14 rules. LibClamAV Error: yyerror(): /usr/local/clamav/winnow_malware.yara line 84 duplicate identifier "CryptoWall_Resume_phish" LibClamAV Warning: cli_loadyara: failed to parse or load 1 yara rules from file /usr/local/clamav/winnow_malware.yara, successfully loaded 8 rules. It seems to be OK, then after about 4 mins clamd has crashed. James. On 10 May 2018, at 1:42 pm, Al Varnell <[email protected]<mailto:[email protected]> <mailto:[email protected]>> wrote: Lots of variables here, but there has to be an actual bug somewhere. A corrupt yara file should just cause it to be ignored with a log entry indicating what's wrong and not crash ClamAV. That's what happens with one of the .yara files I've been using where I get: LibClamAV Error: yyerror(): /usr/local/clamXav/share/clamav/AlienVault.yara line 55 syntax error, unexpected _TEXT_STRING_, expecting _CONDITION_ LibClamAV Error: cli_loadyara: failed to parse rules file /usr/local/clamXav/share/clamav/AlienVault.yara, error count 1 Yara appears to still be evolving since it's introduction maybe four years ago? Apple began to include it as a PrivateFramework with the OS at some point and currently uses it as a supplement to it's XProtect process. But I think that the ClamAV capability is completely self-contained. If all those except for the two Sanesecurity files are old, then it would seem to be a 0.100.0 bug in not being able to parse something. -Al- On Wed, May 09, 2018 at 07:10 PM, James Brown wrote: Yeah, it was all these: packer.yar winnow_malware.yara CVE-2010-0887.yar maldoc_somerules.yar CVE-2010-0805.yar antidebug_antivm.yar CVE-2010-1297.yar CVE-2013-0074.yar CVE-2013-0422.yar CVE-2015-5119.yar Maldoc_Hidden_PE_file.yar EK_Zeus.yar EK_Sakura.yar EK_ZeroAcces.yar EK_Zerox88.yar EK_Fragus.yar EK_Phoenix.yar EK_BleedingLife.yar EK_Crimepack.yar EK_Eleonore.yar EK_Angler.yar EK_Blackhole.yar Zeus_EK.yar ZeroAcces_EK.yar Zerox88_EK.yar Phoenix_EK.yar Sakura_EK.yar Fragus_EK.yar Crimepack_EK.yar Eleonore_EK.yar Blackhole_EK.yar BleedingLife_EK.yar Angler_EK.yar EMAIL_Cryptowall.yar malicious_document.yar Sanesecurity_spam.yara antidebug.yar Sanesecurity_sigtest.yara I don’t know if all of them would cause clamav to crash or just one particular one. I probably downloaded them not long after this came out: https://blog.clamav.net/2015/06/clamav-099b-meets-yara.html <https://blog.clamav.net/2015/06/clamav-099b-meets-yara.html> <https://blog.clamav.net/2015/06/clamav-099b-meets-yara.html <https://blog.clamav.net/2015/06/clamav-099b-meets-yara.html>> The clamav-unofficial-sigs script by eXtremeShok has just re-downloaded Sanesecurity_sigtest.yara and Sanesecurity_spam.yara and clamd is still running, so I presume one of the other files was corrupt? James On 10 May 2018, at 11:50 am, Al Varnell <[email protected]<mailto:[email protected]> <mailto:[email protected]> <mailto:[email protected]<mailto:[email protected]>>> wrote: I'm guessing those came from some Unofficial signature database you subscribe to as I've never seen any included in the Official database. -Al- On Wed, May 09, 2018 at 06:46 PM, James Brown wrote: Thanks for your replay Al. Have just got it working. This was the clue: Application Specific Information: Assertion failed: (sp == 0), function yr_execute_code, file yara_exec.c, line 177.” I deleted all the .yar and .yara files from /usr/local/clamav and it started fine (and is still running). Hope this helps someone else. James. On 10 May 2018, at 11:34 am, Al Varnell <[email protected]<mailto:[email protected]> <mailto:[email protected]> <mailto:[email protected]<mailto:[email protected]>> <mailto:[email protected] <mailto:[email protected]> <mailto:[email protected]<mailto:[email protected]>>>> wrote: OS X 10.7.5 is very old, but I know it's been done successfully for 10.6.8 by using several work-arounds. Looks like you have PCRE working and assume you got over any OpenSSL hurdles. Might help if you posted the output of sudo clamconf -Al- ClamXAV User On Wed, May 09, 2018 at 05:40 PM, James Brown wrote: I upgraded from 0.99.3 (which worked perfectly) to 0.100.0. Everything seemed to work but today I noticed that it wasn’t actually running. No mention of there being a problem in the logs: Thu May 10 10:01:25 2018 -> +++ Started at Thu May 10 10:01:25 2018 Thu May 10 10:01:25 2018 -> Received 0 file descriptor(s) from systemd. Thu May 10 10:01:25 2018 -> clamd daemon 0.100.0 (OS: darwin11.4.2, ARCH: x86_64, CPU: x86_64) Thu May 10 10:01:25 2018 -> Log file size limited to 2097152 bytes. Thu May 10 10:01:25 2018 -> Reading databases from /usr/local/clamav Thu May 10 10:01:25 2018 -> Not loading PUA signatures. Thu May 10 10:01:25 2018 -> Bytecode: Security mode set to "TrustSigned". Thu May 10 10:02:13 2018 -> Loaded 13435987 signatures. Thu May 10 10:02:17 2018 -> LOCAL: Removing stale socket file /tmp/clamd Thu May 10 10:02:17 2018 -> LOCAL: Unix socket file /tmp/clamd Thu May 10 10:02:17 2018 -> LOCAL: Setting connection queue length to 200 Thu May 10 10:02:17 2018 -> Limits: Global size limit set to 104857600 bytes. Thu May 10 10:02:17 2018 -> Limits: File size limit set to 26214400 bytes. Thu May 10 10:02:17 2018 -> Limits: Recursion level limit set to 16. Thu May 10 10:02:17 2018 -> Limits: Files limit set to 10000. Thu May 10 10:02:17 2018 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes. Thu May 10 10:02:17 2018 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes. Thu May 10 10:02:17 2018 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes. Thu May 10 10:02:17 2018 -> Limits: MaxScriptNormalize limit set to 5242880 bytes. Thu May 10 10:02:17 2018 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes. Thu May 10 10:02:17 2018 -> Limits: MaxPartitions limit set to 50. Thu May 10 10:02:17 2018 -> Limits: MaxIconsPE limit set to 100. Thu May 10 10:02:17 2018 -> Limits: MaxRecHWP3 limit set to 16. Thu May 10 10:02:17 2018 -> Limits: PCREMatchLimit limit set to 100000. Thu May 10 10:02:17 2018 -> Limits: PCRERecMatchLimit limit set to 5000. Thu May 10 10:02:17 2018 -> Limits: PCREMaxFileSize limit set to 26214400. Thu May 10 10:02:17 2018 -> Archive support enabled. Thu May 10 10:02:17 2018 -> Archive: Blocking encrypted archives. Thu May 10 10:02:17 2018 -> BlockMax heuristic detection disabled. Thu May 10 10:02:17 2018 -> Algorithmic detection enabled. Thu May 10 10:02:17 2018 -> Portable Executable support enabled. Thu May 10 10:02:17 2018 -> ELF support enabled. Thu May 10 10:02:17 2018 -> Mail files support enabled. Thu May 10 10:02:17 2018 -> Mail: RFC1341 handling enabled. Thu May 10 10:02:17 2018 -> OLE2 support enabled. Thu May 10 10:02:17 2018 -> OLE2: Blocking all VBA macros. Thu May 10 10:02:17 2018 -> PDF support enabled. Thu May 10 10:02:17 2018 -> SWF support enabled. Thu May 10 10:02:17 2018 -> HTML support enabled. Thu May 10 10:02:17 2018 -> XMLDOCS support enabled. Thu May 10 10:02:17 2018 -> HWP3 support enabled. Thu May 10 10:02:17 2018 -> Self checking every 600 seconds. Thu May 10 10:02:17 2018 -> Set stacksize to 1048576 Mac OS cash report: <clamd_2018-05-10-100246_localhost.crash> Most useful part is probably this: "Crashed Thread: 2 Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Application Specific Information: Assertion failed: (sp == 0), function yr_execute_code, file yara_exec.c, line 177." Any suggestions? Thanks, James _______________________________________________ clamav-users mailing list [email protected]<mailto:[email protected]> <mailto:[email protected]> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users <http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users> Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml> _______________________________________________ clamav-users mailing list [email protected]<mailto:[email protected]> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml _______________________________________________ clamav-users mailing list [email protected] http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
