Re: [DISCUSS] community view/roadmap of threat intel

2018-02-21 Thread Ali Nazemian
Hi Simon, Thre are three reasons for why I thought a separate topology may help (However, it was just an example). 1- Threat intel model flexibility: I thought there is no Stellar config available for the threat-intel section, so we will end up using normal Stellar enrichment to address threat

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Andre
Otto, Simon How about using MiNiFi Java (or even C++)? Today you have all those data collection scripts going on: The GeoIP loader, the threat intel loader... Perhaps we could replace some of those with a MiNiFi flow (so you don't end up needing a complete NiFi deployment which IMNSHO is

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Otto Fowler
Having a Metron Processor managed by our project would be fine. On February 19, 2018 at 11:13:20, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Agreed, reputation and confidence is not really encoded formally in the data model, but I would expect most people are using them to weight

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Simon Elliston Ball
Agreed, reputation and confidence is not really encoded formally in the data model, but I would expect most people are using them to weight the results of the threat intel now we have threat triage scores built on stellar expressions. There is definitely scope here to provide at least a

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Otto Fowler
There are a couple of use cases here for getting the data. When you _can_ or want to ingest and duplicate the external store 1. Bulk Loading ( from a clean empty state ) 2. Tailing the feed afterwards When you can’t 3. Calling the api ( most likely web ) for reputation or some other thing

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Andre
Simon, I have coded but not merged a STIX / TAXII processor for NiFi that would work perfectly fine with this. But I will take the opportunity to touch the following points: 1. Threat Intel is more frequently than not based on API lookups (e.g. VirusTotal, RBLs and correlated, Umbrella's top

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Simon Elliston Ball
Would it make sense to lean on something like Apache NiFi for this? It seems a good fit to handle getting data from wherever (web service, poll, push etc, streams etc). If we were to build a processor which encapsulated the threat intel loader logic, that would provide a granular route to

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-15 Thread Ali Nazemian
I think one of the challenges is where the scope of threat intel ends from the Metron roadmap? Does it gonna relly on supporting a standard format and a loader to send it to HBase for the later threat intel use cases? In my opinion, it would be better to have a separate topology (sort of similar

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-14 Thread Simon Elliston Ball
We used to install soltra edge in the old ansible builds (which have thankfully now been pared back in the interests of stability in full dev). Soltra has not been a good option since they went proprietary, so since then we’ve included opentaxii (BSD 3) as a discovery and aggregator. Most of

[DISCUSS] community view/roadmap of threat intel

2018-02-13 Thread Ali Nazemian
Hi All, I would like to understand Metron community view on Threat Intel aggregators as well as the roadmap of threat intelligence and threat hunting. There are some open source options available regarding threat intel aggregator such as Minemeld, Hippocampe, etc. Is there any plan to build that