GitHub user nickwallen opened a pull request:
https://github.com/apache/incubator-metron/pull/208
METRON-309 Create a normalcy profiler
### [METRON-309](https://issues.apache.org/jira/browse/METRON-309)
Created a normalcy profiler, otherwise known as the Metron Profiler.
The Profiler is a feature extraction mechanism that can generate a profile
describing the behavior of an entity on a network. An entity might be a server,
user, subnet or application. Once a profile has been generated defining what
normal behavior looks-like, it can be used to build models that identify
anomalous behavior.
This is achieved by summarizing the streaming telemetry data consumed by
Metron over sliding windows. A summary statistic is applied to the data
received within a given window. Collecting this summary across many windows
results in a time series that is useful for analysis.
### Changes
This PR contains the following high-level changes to Metron.
* A Storm topology that generates Profiles.
`metron-analytics/metron-profiler`
* Changes to Ansible to deploy the Profiler topology. `metron-deployment`
* A new bolt and associated tooling to perform HBase writes from a Storm
topology. `metron-platform/metron-hbase`
* Additions to the Stellar language to provide arithmetic and control flow
logic required to generate Profiles. `metron-platform/metron-common`
### Testing
Follow the instructions contained in the
[README](https://github.com/nickwallen/incubator-metron/tree/METRON-309/metron-analytics/metron-profiler#getting-started)
to get your very first Profile running.
### Documentation
* Documentation for the Profiler is contained within the
[README](https://github.com/nickwallen/incubator-metron/blob/METRON-309/metron-analytics/metron-profiler/README.md).
* The code itself contains plenty of comments and Javadocs.
* The design document is attached to the
[JIRA](https://issues.apache.org/jira/browse/METRON-309).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/incubator-metron METRON-309
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/208.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #208
----
commit a77fe3e91601dcf1698271aad1b00e687038b1ed
Author: Nick Allen <[email protected]>
Date: 2016-07-29T20:53:49Z
METRON-309 Create a normalcy profiler
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
