Hello, I was wondering if anyone had success with using yara to scan a centralized file storage solution. We're using S3 and have accumulated a TB+ of files we'd like to scan on a very frequent basis. I've seen Binaryalert and clara which use ETL/lambda functions to scan files as they go through the pipeline, but we're looking to adjust signatures, and rescan the whole repo of files. We tried using a S3FS (FUSE) mount, but saw performance suffer to the point where the yara scans were taking much too long, we're trying to lower the feedback loop for our yara rule creators so they can scan TBs of files quickly, adjust, scan again, repeat. An example is VirusTotal's retrohunt, we are familiar with the service, but would love an option for files not on VT, or a sliver that is much faster.
Other folks I've chatted with have said to ditch S3 altogether and use NFS or just the posix FS with something like ext4 or XFS and a custom file upload service that allows the files to be uploaded, and then use the yara engine and rules on the host where the files are, instead of trying to bring a large amount of files to the rules. We could possibly distribute the files as well, using something like k8s PVCs and then aggregating the results of the scans, but would likely need to build something custom since mquery didn't support all the types of rules we use, and we had issues testing with klara. -- You received this message because you are subscribed to the Google Groups "YARA" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/yara-project/4df99c45-79d0-4c64-a8b1-30096f5ffdcdn%40googlegroups.com.
