Hi Andre,
Thanks for the reply. Agree with you. I’m not implementing a memory resident AV approach…but rather an ingress “kill chain” as we ingest data. We are looking at metascan, but have had some mixed results with it in a couple of other systems it was used in. But for the time being, I have other tools, e.g. clam, mcafee, Symantec, etc that I can use through their API’s or command line tools. Because as you said, the goal is to get a pass/fail assertion from the tool before allowing the content to be ingested. So with that in mind, and as I’m trying to learn more about Nifi… I’m wondering if existing processors would be good for this use case, or if I need to pursue building a custom processor. I haven’t tried the stream command processor yet, but in reading about it, seems like this would require me to use the command line on-demand scanning of these tools. And that I may need to pursue writing a custom processor to interface with external tools like clam, mcafee, metascan in order to get low-latency, higher throughput with these tools. Thanks, Dan M From: Andre <[email protected]> Reply-To: <[email protected]> Date: Sunday, September 4, 2016 at 12:56 AM To: <[email protected]> Subject: Re: AV Scanning a Flowfile Dan, Using a memory resident AV would not provide you with certainty of the result as the file would just fail to write or fail to read, you want something that is telling you, this is bad|good. There used to be many AVs with Java APIs but nowadays they tend to be reasonably rarer... Some people as you noted use a daemon based approach like: http://nsinfra.blogspot.com.au/2011/06/java-api-to-detect-virus-using-clamav.html Having said that, have you considered using an RESTful AV API such as Metascan? It is a paid product but would fit your need. You can also use a combination of Hash processor and multi AV lookup providers like Virustotal (RESTful) and Cymru (QueryDNS) but this would not detect automatically generated malware variants. This is a technique widely used to escape hashing, where the hostile party changes a single bit of the malicious payload via padding or other approach, causing the resulting md5/sha hash to be completely different from the hash of the same malware delivered to another party. Worst case comes, you could use PutEmail followed by ListenSMTP but this is nuclear waste level of dirty... :-) Cheers On Sun, Sep 4, 2016 at 10:41 AM, dgm <[email protected]> wrote: Hello – Has anyone had any experience integrating an AV Scanner with NiFi, such as clamav or mcafee? Both support running as service listening to a port as well command line options… I’m currently thinking of trying to use the ExecuteStreamCommand processor, however, if using command line tools, the external app will have to initialize for each invocation. This is where I think being able to use an external service call/api might be significantly faster. Thanks, Dan M
