Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/343#discussion_r86647954
--- Diff: metron-analytics/metron-profiler/README.md ---
@@ -1,16 +1,74 @@
# Metron Profiler
-The Profiler is a feature extraction mechanism that can generate a profile
describing the behavior of an entity on a network. An entity might be a
server, user, subnet or application. Once a profile has been generated defining
what normal behavior looks-like, models can be built that identify anomalous
behavior.
+The Profiler is a feature extraction mechanism that can generate a profile
describing the behavior of an entity. An entity might be a server, user,
subnet or application. Once a profile has been generated defining what normal
behavior looks-like, models can be built that identify anomalous behavior.
This is achieved by summarizing the streaming telemetry data consumed by
Metron over sliding windows. A summary statistic is applied to the data
received within a given window. Collecting this summary across many windows
results in a time series that is useful for analysis.
-## Usage
+Any field contained within a message can be used to generate a profile. A
profile can even be produced by combining fields that originate in different
data sources. A user has considerable power to transform the data used in a
profile by leveraging the Stellar language. A user only need configure the
desired profiles and ensure that the Profiler topology is running.
-Any field contained within a message can be used to generate a profile. A
profile can even be produced from combining fields that originate in different
data sources. A user has considerable power to transform the data used in a
profile by leveraging the Stellar language. A user only need configure the
desired profiles in Zookeeper and ensure that the Profiler topology is running.
+* [Getting Started](#getting-started)
+* [Creating Profiles](#creating-profiles)
+* [Configuring the Profiler](#configuring-the-profiler)
+* [Examples](#examples)
+* [Implementation](#implementation)
-### Configuration
+## Getting Started
+
+This section will describe the steps required to get your first profile
running.
+
+1. Stand-up a Metron environment. For this example, we will use the
'Quick Dev' environment. Follow the instructions included with [Quick
Dev](../../metron-deployment/vagrant/quick-dev-platform) or build your own.
+
+1. Create a table within HBase that will store the profile data. The table
name and column family must match the [Profiler's
configuration](#configuring-the-profiler).
+ ```
+ $ /usr/hdp/current/hbase-client/bin/hbase shell
+ hbase(main):001:0> create 'profiler', 'P'
+ ```
+
+1. Define the profile in a file located at
`$METRON_HOME/config/zookeeper/profiler.json`. The following JSON will create
a profile that simply counts the number of messages.
+ ```
+ {
+ "profiles": [
+ {
+ "profile": "test",
+ "foreach": "ip_src_addr",
+ "init": { "count": 0 },
+ "update": { "count": "count + 1" },
+ "result": "count"
+ }
+ ]
+ }
+ ```
+
+1. Upload the profile definition to Zookeeper.
+ ```
+ $ cd /usr/metron/0.2.1BETA/
+ $ bin/zk_load_configs.sh -m PUSH -i config/zookeeper/ -z node1:2181
+ ```
+
+1. Start the Profiler topology.
+ ```
+ $ bin/start_profiler_topology.sh
+ ```
+
+1. Ensure that test messages are being sent to the Profiler's input topic
in Kafka. The Profiler will consume messages from the `inputTopic` defined in
the [Profiler's configuration](#configuring-the-profiler).
+
+1. Check the HBase table to validate that the Profiler is writing the
profile. Remember that the Profiler is flushing the profile every 15 minutes.
You will need to wait at least this long to start seeing profile data in HBase.
+ ```
+ $ /usr/hdp/current/hbase-client/bin/hbase shell
+ hbase(main):001:0> count 'profiler'
+ ```
+
+1. Use the Profiler Client to read the profile data. Replace the IP
`10.0.0.1` with one specific to your environment. More information on using
the client can be found [here](../metron-profiler-client).
--- End diff --
Suggest edit: Use the Profiler Client to read the profile data. The below
example PROFILE_GET command will read data written by the sample profile given
above, if `10.0.0.1` is one of the input values for `ip_src_addr`. More
information on using the client can be found [here](../metron-profiler-client).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
