Repository: metron Updated Branches: refs/heads/master 739782ab7 -> 0ba180262
METRON-1826 Update librdkafka and devtoolset (JonZeolla via jonzeolla) closes apache/metron#1238 Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/0ba18026 Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/0ba18026 Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/0ba18026 Branch: refs/heads/master Commit: 0ba18026222f004316b96633780509c71934121e Parents: 739782a Author: JonZeolla <[email protected]> Authored: Mon Oct 22 19:48:56 2018 -0400 Committer: jonzeolla <[email protected]> Committed: Mon Oct 22 19:48:56 2018 -0400 ---------------------------------------------------------------------- .../ansible/roles/bro/tasks/bro.yml | 4 +-- .../ansible/roles/bro/tasks/dependencies.yml | 4 +-- .../roles/bro/tasks/metron-bro-plugin-kafka.yml | 4 +-- .../ansible/roles/librdkafka/defaults/main.yml | 4 +-- metron-deployment/development/centos6/README.md | 26 ++++++++++++++++++-- .../development/ubuntu14/README.md | 26 ++++++++++++++++++-- 6 files changed, 56 insertions(+), 12 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/ansible/roles/bro/tasks/bro.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/ansible/roles/bro/tasks/bro.yml b/metron-deployment/ansible/roles/bro/tasks/bro.yml index 222ef0e..9ba3ffa 100644 --- a/metron-deployment/ansible/roles/bro/tasks/bro.yml +++ b/metron-deployment/ansible/roles/bro/tasks/bro.yml @@ -30,8 +30,8 @@ - name: Compile and Install bro shell: "{{ item }}" environment: - CXX: /opt/rh/devtoolset-4/root/usr/bin/g++ - CC: /opt/rh/devtoolset-4/root/usr/bin/gcc + CXX: /opt/rh/devtoolset-6/root/usr/bin/g++ + CC: /opt/rh/devtoolset-6/root/usr/bin/gcc args: chdir: "/tmp/bro-{{ bro_version }}" creates: "{{ bro_home }}/bin/bro" http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/ansible/roles/bro/tasks/dependencies.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/ansible/roles/bro/tasks/dependencies.yml b/metron-deployment/ansible/roles/bro/tasks/dependencies.yml index 3cd3bae..f8b1e28 100644 --- a/metron-deployment/ansible/roles/bro/tasks/dependencies.yml +++ b/metron-deployment/ansible/roles/bro/tasks/dependencies.yml @@ -38,8 +38,8 @@ - perl - crontabs - net-tools - - devtoolset-4-gcc - - devtoolset-4-gcc-c++ + - devtoolset-6-gcc + - devtoolset-6-gcc-c++ - python27 - rh-git29 register: result http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/ansible/roles/bro/tasks/metron-bro-plugin-kafka.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/ansible/roles/bro/tasks/metron-bro-plugin-kafka.yml b/metron-deployment/ansible/roles/bro/tasks/metron-bro-plugin-kafka.yml index 7043387..ec7af97 100644 --- a/metron-deployment/ansible/roles/bro/tasks/metron-bro-plugin-kafka.yml +++ b/metron-deployment/ansible/roles/bro/tasks/metron-bro-plugin-kafka.yml @@ -30,8 +30,8 @@ environment: PATH: "{{ git29_bin }}:{{ bro_bin }}:{{ ansible_env.PATH }}" LD_LIBRARY_PATH: "{{ httpd24_lib }}:{{ python27_lib }}" - CXX: /opt/rh/devtoolset-4/root/usr/bin/g++ - CC: /opt/rh/devtoolset-4/root/usr/bin/gcc + CXX: /opt/rh/devtoolset-6/root/usr/bin/g++ + CC: /opt/rh/devtoolset-6/root/usr/bin/gcc command: "{{ python27_bin }}/bro-pkg install apache/metron-bro-plugin-kafka --force --version {{ metron_bro_plugin_kafka_version }}" - name: Configure bro-kafka plugin http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/ansible/roles/librdkafka/defaults/main.yml ---------------------------------------------------------------------- diff --git a/metron-deployment/ansible/roles/librdkafka/defaults/main.yml b/metron-deployment/ansible/roles/librdkafka/defaults/main.yml index 063c22f..6e25cb3 100644 --- a/metron-deployment/ansible/roles/librdkafka/defaults/main.yml +++ b/metron-deployment/ansible/roles/librdkafka/defaults/main.yml @@ -15,6 +15,6 @@ # limitations under the License. # --- -librdkafka_version: 0.9.4 -librdkafka_url: https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz +librdkafka_version: 0.11.5 +librdkafka_url: https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz librdkafka_home: /usr/local http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/development/centos6/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/development/centos6/README.md b/metron-deployment/development/centos6/README.md index e873452..99ec967 100644 --- a/metron-deployment/development/centos6/README.md +++ b/metron-deployment/development/centos6/README.md @@ -97,8 +97,30 @@ In addition to re-running the entire provisioning play book, you may now re-run vagrant --ansible-tags="sensor-stubs" provision ``` -Tags are listed in the playbooks, some frequently used tags: +Tags are listed in the playbooks. Here are some frequently used tags: + `hdp-install` - Install HDP + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) -+ `sensors` - Deploy and start the sensors. ++ `sensors` - Deploy the sensors (see [Sensors](#sensors) for more details regarding this tag) + `sensor-stubs` - Deploy and start the sensor stubs. + +#### Sensors + +By default, the Metron development environment uses sensor stubs to mimic the behavior of the full sensors. This is done because the full sensors take a significant amount of time and CPU to build, install, and run. + +From time to time you may want to install the full sensors for testing (see the specifics of what that means [here](../../ansible/playbooks/sensor_install.yml)). This can be done by running the following command: + +``` +vagrant --ansible-skip-tags="sensor-stubs" up +``` + +This will skip only the `sensor-stubs` tag, allowing the ansible roles with the `sensors` tag to be run. This provisions the full sensors in a 'testing mode' so that they are more active, and thus more useful for testing (more details on that [here](../../ansible/roles/sensor-test-mode/)). **However**, when vagrant completes the sensors will NOT be running. In order to start the sensors and simulate traffic through them (which will create a fair amount of load on your test system), complete the below steps: + +``` +vagrant ssh +sudo su - +service pcap-replay restart +service yaf restart +service snortd restart +service snort-producer restart +``` + http://git-wip-us.apache.org/repos/asf/metron/blob/0ba18026/metron-deployment/development/ubuntu14/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/development/ubuntu14/README.md b/metron-deployment/development/ubuntu14/README.md index 494937e..53bea67 100644 --- a/metron-deployment/development/ubuntu14/README.md +++ b/metron-deployment/development/ubuntu14/README.md @@ -98,8 +98,30 @@ In addition to re-running the entire provisioning play book, you may now re-run vagrant --ansible-tags="sensor-stubs" provision ``` -Tags are listed in the playbooks, some frequently used tags: +Tags are listed in the playbooks. Here are some frequently used tags: + `hdp-install` - Install HDP + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) -+ `sensors` - Deploy and start the sensors. ++ `sensors` - Deploy the sensors (see [Sensors](#sensors) for more details regarding this tag) + `sensor-stubs` - Deploy and start the sensor stubs. + +#### Sensors + +By default, the Metron development environment uses sensor stubs to mimic the behavior of the full sensors. This is done because the full sensors take a significant amount of time and CPU to build, install, and run. + +From time to time you may want to install the full sensors for testing (see the specifics of what that means [here](../../ansible/playbooks/sensor_install.yml)). This can be done by running the following command: + +``` +vagrant --ansible-skip-tags="sensor-stubs" up +``` + +This will skip only the `sensor-stubs` tag, allowing the ansible roles with the `sensors` tag to be run. This provisions the full sensors in a 'testing mode' so that they are more active, and thus more useful for testing (more details on that [here](../../ansible/roles/sensor-test-mode/)). **However**, when vagrant completes the sensors will NOT be running. In order to start the sensors and simulate traffic through them (which will create a fair amount of load on your test system), complete the below steps: + +``` +vagrant ssh +sudo su - +service pcap-replay restart +service yaf restart +service snortd restart +service snort-producer restart +``` +
