GitHub user JonZeolla opened a pull request: https://github.com/apache/metron/pull/837
METRON-1309: Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka ## Contributor Comments Based on the conversation on the [mailing list](https://lists.apache.org/thread.html/dd63e5bcf8a288dfaf3cdd551128e3df59580367080bda5f344e8538@%3Cdev.metron.apache.org%3E), this removes the bro kafka plugin from this repo and pulls it from our new apache/metron-bro-plugin-kafka repo. # Testing ## Please hold on testing, this needs to be validated 1. Create a working directory and pull in this PR ``` mkdir ~/metron-1309 git clone https://github.com/apache/metron ~/metron-1309/metron cd ~/metron-1309/metron git remote add jonzeolla https://github.com/jonzeolla/metron git pull jonzeolla METRON-1309 ``` 1. Modify [this](https://github.com/JonZeolla/metron/blob/METRON-1309/metron-deployment/vagrant/full-dev-platform/Vagrantfile#L20) to remove `sensors,` (to spin up the real sensors). ``` sed -i '' "s/ansibleSkipTags=.*/ansibleSkipTags=\'quick_dev\'/" metron-deployment/vagrant/full-dev-platform/Vagrantfile ``` 1. Spin up full-dev ``` cd metron-deployment/vagrant/full-dev-platform/ vagrant up ``` 1. Set up the environment in full-dev. ``` vagrant ssh sudo su - export PATH=$PATH:/usr/local/bro/bin service monit stop && service sensor-stubs stop bro && broctl stop ``` 1. Monitor the bro kafka topic ``` # Open a new terminal cd ~/metron-1309/metron/metron-deployment/vagrant/full-dev-platform vagrant ssh sudo su - export PATH=$PATH:/usr/local/bro/bin:/usr/hdp/current/kafka-broker/bin kafka-console-consumer.sh --zookeeper localhost:2181 --topic bro ``` 1. Run bro against some public pcaps. ``` mkdir -p ~/brotmp/nitroba ~/brotmp/example-traffic wget https://www.bro.org/static/traces/exercise-traffic.pcap -O ~/brotmp/example-traffic/exercise-traffic.pcap wget http://downloads.digitalcorpora.org/corpora/network-packet-dumps/2008-nitroba/nitroba.pcap -O ~/brotmp/nitroba/nitroba.pcap cd ~/brotmp/example-traffic bro -r exercise-traffic.pcap /usr/local/bro/share/bro/site/local.bro -C cd ~/brotmp/nitroba bro -r nitroba.pcap /usr/local/bro/share/bro/site/local.bro -C ``` 1. Verify that logs are properly getting to the bro kafka topic, via the terminal you set up two steps ago. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [X] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [X] Have you included steps or a guide to how the change may be verified and tested manually? - [X] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [X] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` #### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/JonZeolla/metron METRON-1309 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/837.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #837 ---- commit cbcd0e6ec1395275c99a6448d6cc26e511d3840d Author: Jon Zeolla <zeo...@gmail.com> Date: 2017-11-08T19:33:03Z METRON-1309: Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka ---- ---