[
https://issues.apache.org/jira/browse/METRON-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824572#comment-15824572
]
ASF GitHub Bot commented on METRON-659:
---------------------------------------
GitHub user nickwallen opened a pull request:
https://github.com/apache/incubator-metron/pull/417
METRON-659 Emulate Sensors in Development Environments
A service has been created to simulate the behavior of a sensor by sending
canned telemetry data to a Kafka topic. These "Sensor Stubs" consume fewer
resources than the actual sensor that they replace.
## Testing
I have tested this change by deploying and validating the following
environments. For each environment I manually validated it and ensured that
data flowed end-to-end and was visible within the Metron dashboard.
* Quick Dev - Default with Sensor Stubs
* Quick Dev - Modified to install Sensors
* Full Dev - Default with Sensor Stubs
* Full Dev - Modified to install Sensors
* Amazon EC2 - Default with Sensors (behavior not changed)
## Q&A
### (Q) How do the sensor stubs work?
The stubs are installed with a set of canned data for each sensor type. A
subset of this canned data is randomly selected and sent to the Kafka topic in
batches. The timestamp of each message is updated to match current system
time.
### (Q) How do I configure the message rate?
The number of telemetry messages sent in each batch, along with the time
delay between batches is configurable. Before installation, these values can
be configured by redefining `sensor_stubs_delay` and `sensor_stubs_count`. The
values can also be configured by altering the deployed system service script at
`/etc/init.d/sensor-stubs`.
### (Q) How do I install the sensor stubs?
Using the default playbooks, this role can be installed by using the
Ansible tag `sensor-stubs`. This service is installed on the same hosts where
the sensors would be; defined by the `sensors` host group.
The defaults for the "Quick Dev" and "Full Dev" environments have been
changed so that the Sensor Stubs are installed by default, rather than the
sensors themselves. The Amazon EC2 environment continues to install the
original sensors by default.
### (Q) How do I use the sensor stubs?
Start all sensor stubs. The output includes the PID for each running
sensor stub.
```
$ service sensor-stubs start
Starting sensor-stubs...
bro: Ok [26505]
yaf: Ok [26507]
snort: Ok [26509]
```
Stop all sensor stubs.
```
$ service sensor-stubs stop
Stopping sensor-stubs...
.. bro: Ok [26505]
.. yaf: Ok [26507]
.. snort: Ok [26509]
```
Start only the Bro sensor stub.
```
$ service sensor-stubs start bro
Starting sensor-stubs...
bro: OK [11616]
```
### (Q) How do I install the original sensors?
The default behavior can be changed by skipping the `sensor-stubs` flag and
including the `sensors` flag. For example, to deploy "Quick Dev" with the
original sensors run the following command.
```
cd metron-deployment/vagrant/quick-dev-platform
vagrant --ansible-skip-tags="sensor-stubs,solr" up
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/incubator-metron METRON-659
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/417.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #417
----
commit c155e089c981a786e6874b7f6c5f5d44d3d436dd
Author: Nick Allen <[email protected]>
Date: 2017-01-13T22:02:39Z
METRON-659 Emulate Sensors in Development Environments
----
> Emulate Sensors in Development Environments
> -------------------------------------------
>
> Key: METRON-659
> URL: https://issues.apache.org/jira/browse/METRON-659
> Project: Metron
> Issue Type: Improvement
> Affects Versions: 0.3.0
> Reporter: Nick Allen
> Assignee: Nick Allen
> Fix For: Next + 1
>
>
> Replace the Snort, Bro, and YAF sensors on the "Quick Dev" and "Full Dev"
> environments with a mechanism that consumes less resources.
> These environments are notoriously difficult to work with because the
> installed services consume nearly all available memory. The Bro, YAF, and
> Snort sensors, along with the PCAP Replay service, consume a considerable
> amount of these limited resources.
> Replacing these sensors with a lightweight mechanism should free-up
> additional resources, make the environment easier to work with, and result in
> faster deployments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
