Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Simon Elliston Ball
Would it make sense to lean on something like Apache NiFi for this? It seems a good fit to handle getting data from wherever (web service, poll, push etc, streams etc). If we were to build a processor which encapsulated the threat intel loader logic, that would provide a granular route to

[GitHub] metron issue #939: [BUG-96727] xpack support (for discussion only)

2018-02-19 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/939 Can you put some kind of description in here? What does that BUG refer to? ---

[GitHub] metron issue #939: [BUG-96727] xpack support (for discussion only)

2018-02-19 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/939 BUG- is not a valid title for this project. ---

[GitHub] metron pull request #939: [BUG-96727] xpack support (for discussion only)

2018-02-19 Thread wardbekker
Github user wardbekker closed the pull request at: https://github.com/apache/metron/pull/939 ---

[GitHub] metron issue #939: [BUG-96727] xpack support (for discussion only)

2018-02-19 Thread wardbekker
Github user wardbekker commented on the issue: https://github.com/apache/metron/pull/939 ok, used the wrong id ---

Re: ES mpack to include more ES 5 stack properties

2018-02-19 Thread Otto Fowler
I don’t think there are right now. I would recommend entering jira issues for what you haven in mind On February 19, 2018 at 01:02:32, Ali Nazemian (alinazem...@gmail.com) wrote: Hi All, Is there any plan to include more ES 5+ specific properties to Metron mpack? For example, if we want to

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Otto Fowler
There are a couple of use cases here for getting the data. When you _can_ or want to ingest and duplicate the external store 1. Bulk Loading ( from a clean empty state ) 2. Tailing the feed afterwards When you can’t 3. Calling the api ( most likely web ) for reputation or some other thing

[GitHub] metron issue #939: [BUG-96727] xpack support (for discussion only)

2018-02-19 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/939 It is almost like hw has a different issue tracking system ---

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Andre
Simon, I have coded but not merged a STIX / TAXII processor for NiFi that would work perfectly fine with this. But I will take the opportunity to touch the following points: 1. Threat Intel is more frequently than not based on API lookups (e.g. VirusTotal, RBLs and correlated, Umbrella's top

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Simon Elliston Ball
Agreed, reputation and confidence is not really encoded formally in the data model, but I would expect most people are using them to weight the results of the threat intel now we have threat triage scores built on stellar expressions. There is definitely scope here to provide at least a

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Otto Fowler
Having a Metron Processor managed by our project would be fine. On February 19, 2018 at 11:13:20, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Agreed, reputation and confidence is not really encoded formally in the data model, but I would expect most people are using them to weight

Re: [DISCUSS] community view/roadmap of threat intel

2018-02-19 Thread Andre
Otto, Simon How about using MiNiFi Java (or even C++)? Today you have all those data collection scripts going on: The GeoIP loader, the threat intel loader... Perhaps we could replace some of those with a MiNiFi flow (so you don't end up needing a complete NiFi deployment which IMNSHO is