Alright, I've solved the issue: Albeit this is synthetic, running
``` for f in $(seq 0 1000000); do printf "\n\n\n\n\n\n\n\n\n\n" >> text.txt; done ``` And then ``` yara inc.yar . ``` In the directory with the yara files, yields, on my test system: ./misc.yar(9): warning: $newline is slowing down scanning (critical!) fsL ./inc.yar fsL ./global.yar fsL ./misc.yar error scanning ./text.txt: internal error: 30 And while using the newline scan is not a great idea, clearly the file is still being scanned in one way or another, in spite of the global rule. Den torsdag den 17. august 2017 kl. 12.24.11 UTC+2 skrev necrophcodr: > > I'm afraid I cannot post the exact files. I'll create a working > environment that replicates all the variables required, and I'll post it > here when I've gotten this done. > > Den onsdag den 16. august 2017 kl. 16.31.35 UTC+2 skrev Wesley Shields: >> >> I still can not replicate your problem. >> >> A couple of things to note however: >> >> "internal error: 30" is because there are too many matches, which happens >> when a single string matches too many times. It has nothing to do with file >> size like you guessed. >> >> Your "newline_one" rule is marked as private so it should never be >> reported. >> >> At this point I can not replicate your problem so I'm curious if you >> could zip up the files you're using to do this and post them somewhere for >> me to see them exactly? I would need your exact YARA rules and the file you >> are scanning. >> >> -- WXS >> >> > On Aug 16, 2017, at 5:51 AM, necrophcodr <[email protected]> wrote: >> > >> > Alright, so I've returned with a result: >> > >> > If I have `~/inc.yar` with the following content: >> > >> > ``` >> > include "./global.yar" >> > include "./misc.yar" >> > ``` >> > >> > And the content of these files respectively: >> > >> > ``` >> > global rule fsL { condition: filesize < 8MB } >> > ``` >> > >> > And >> > >> > ``` >> > private rule newline_one { >> > meta: >> > description = "Files that contain one newline" >> > author = "Steffen Rytter Postas" >> > >> > strings: >> > $newline = "\n" >> > >> > condition: >> > ( #newline == 1 ) >> > } >> > ``` >> > >> > Then the issue prevails. >> > >> > Note that this requires an actually large file that contains newlines. >> Doing `dd if=/dev/zero bs=4M count=250 of=file.bin` and scanning that won't >> yield usable results. >> > >> > Den onsdag den 16. august 2017 kl. 11.43.17 UTC+2 skrev necrophcodr: >> > Hi Wesley, >> > >> > Sorry for the late reply, vacations and all. >> > >> > So first and foremost: >> > >> > `yara -v` >> > yara 3.5.0 >> > >> > The files getting scanned are reporting ` internal error: 30` which I'm >> reading to be due to files being too large. These files are often larger >> than 500MB too, well above the 8MB margin. >> > >> > I've attempted to replicate it using my own instructions, coupled with >> your misc.yar, and the result is that it works just fine. >> > >> > So I'm guessing the issue is with my own setup, and I'll continue >> evaluating the specifics and return with a response when I've found the >> culprit. >> > >> > Den mandag den 7. august 2017 kl. 16.06.59 UTC+2 skrev Wesley Shields: >> > I can't replicate this behavior using 3.5.0 or latest master. >> > >> > wxs@wxs-mbp yara % cat foo >> > include "./global.yar" >> > include "./misc.yar" >> > wxs@wxs-mbp yara % cat global.yar >> > global rule fileSizeLimit { condition: filesize < 1KB } >> > wxs@wxs-mbp yara % cat misc.yar >> > rule foo { condition: true } >> > wxs@wxs-mbp yara % ls -l /bin/ls >> > -rwxr-xr-x 1 root wheel 38624 Jul 15 00:29 /bin/ls* >> > wxs@wxs-mbp yara % ./yara foo /bin/ls >> > wxs@wxs-mbp yara % >> > >> > When you say regardless of file size are you sure you're above the 8MB? >> Keep in mind that 8MB is 8 * 1048576, which is 8388608. >> > >> > -- WXS >> > >> > > On Jul 28, 2017, at 7:01 AM, necrophcodr <[email protected]> >> wrote: >> > > >> > > So I've got quite a few rules, but it all comes down to this: >> > > >> > > include "./rules/global.yar" >> > > include "./rules/misc.yar" >> > > >> > > >> > > The global.yar file contains >> > > >> > > global rule fileSizeLimit { condition: filesize < 8MB } >> > > >> > > >> > > Any rule defined in rules/misc.yar are matched regardless of file >> size, but this is not what I intend. What am I doing wrong here? >> > > >> > > If this is not the right place to post, that's alright, feel free to >> slap me on the wrist and direct me to the correct location. >> > > >> > > edit: >> > > >> > > I should mention this is using Yara 3.5.0. I don't have a chance to >> upgrade this within the week. >> > > >> > > >> > > -- >> > > You received this message because you are subscribed to the Google >> Groups "YARA" group. >> > > To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> > > For more options, visit https://groups.google.com/d/optout. >> > >> >> -- You received this message because you are subscribed to the Google Groups "YARA" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
