METRON-979 markdown errors in site-book, part 2 (JonZeolla via mattf-horton) closes mattf-horton/metron#7 closes apache/metron#605
Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/d5abcf42 Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/d5abcf42 Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/d5abcf42 Branch: refs/heads/Metron_0.4.0 Commit: d5abcf429dad3c284a5f2469c0fb99713844be02 Parents: 1e9acc7 Author: JonZeolla <[email protected]> Authored: Mon Jun 5 14:17:46 2017 -0700 Committer: mattf <[email protected]> Committed: Mon Jun 5 14:17:46 2017 -0700 ---------------------------------------------------------------------- Upgrading.md | 18 ++++----- metron-deployment/Kerberos-manual-setup.md | 4 ++ metron-deployment/amazon-ec2/README.md | 53 ++++++++++++------------- metron-platform/metron-parsers/README.md | 2 - 4 files changed, 39 insertions(+), 38 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/d5abcf42/Upgrading.md ---------------------------------------------------------------------- diff --git a/Upgrading.md b/Upgrading.md index 3864ae3..6fb2486 100644 --- a/Upgrading.md +++ b/Upgrading.md @@ -2,6 +2,15 @@ This document constitutes a per-version listing of changes of configuration which are non-backwards compatible. +## 0.3.1 to 0.4.0 + +### [METRON-671: Refactor existing Ansible deployment to use Ambari MPack](https://issues.apache.org/jira/browse/METRON-671) + +#### Description +Since the Ansible Deployment uses the MPack, RPMs must be built prior to deployment. As a result, +[Docker](https://www.docker.com/) is required to perform a Quick-Dev, Full-Dev or Ansible deployment. +This effectively limits the build environment to Docker supported [platforms](https://docs.docker.com/engine/installation/#platform-support-matrix). + ## 0.3.0 to 0.3.1 ### [METRON-664: Make the index configuration per-writer with enabled/disabled](https://issues.apache.org/jira/browse/METRON-664) @@ -151,12 +160,3 @@ When using Ansible to deploy the latest Metron bits to an existing installation, #### Workaround Set the `topology.classpath` property for storm in Ambari to `/etc/hbase/conf:/etc/hadoop/conf` - -## 0.3.1 to PLACEHOLDER - -### [METRON-671: Refactor existing Ansible deployment to use Ambari MPack](https://issues.apache.org/jira/browse/METRON-671) - -#### Description -Since the Ansible Deployment uses the MPack, RPMs must be built prior to deployment. As a result, -[Docker](https://www.docker.com/) is required to perform a Quick-Dev, Full-Dev or Ansible deployment. -This effectively limits the build environment to Docker supported [platforms](https://docs.docker.com/engine/installation/#platform-support-matrix). http://git-wip-us.apache.org/repos/asf/metron/blob/d5abcf42/metron-deployment/Kerberos-manual-setup.md ---------------------------------------------------------------------- diff --git a/metron-deployment/Kerberos-manual-setup.md b/metron-deployment/Kerberos-manual-setup.md index 4efebc6..cdfbdfb 100644 --- a/metron-deployment/Kerberos-manual-setup.md +++ b/metron-deployment/Kerberos-manual-setup.md @@ -381,7 +381,11 @@ Push Data ``` cat sample-bro.txt | ${KAFKA_HOME}/kafka-broker/bin/kafka-console-producer.sh \ +<<<<<<< HEAD --broker-list ${BROKERLIST} \ +======= + --broker-list ${BROKERLIST} +>>>>>>> d550b9eff34d931aeae7151713c0e07f28719d4c --security-protocol SASL_PLAINTEXT \ --topic bro ``` http://git-wip-us.apache.org/repos/asf/metron/blob/d5abcf42/metron-deployment/amazon-ec2/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/amazon-ec2/README.md b/metron-deployment/amazon-ec2/README.md index b28efdd..7b8be93 100644 --- a/metron-deployment/amazon-ec2/README.md +++ b/metron-deployment/amazon-ec2/README.md @@ -158,33 +158,32 @@ The dashboard expects fields to be of a certain type. If the index templates ha If you see this error, please report your findings by creating a JIRA or dropping an email to the Metron Users mailing list. Follow these steps to work around the problem. -(1) Define which Elasticsearch host to interact with. Any Elasticsearch host should work. -``` -export ES_HOST="http://ec2-52-25-237-20.us-west-2.compute.amazonaws.com:9200"; -``` - -(2) Confirm the index templates are in fact missing. -``` -curl -s -XGET $ES_HOST/_template -``` - -(3) Manually load the index templates. -``` -cd metron-deployment -curl -s -XPOST $ES_HOST/_template/bro_index -d @roles/metron_elasticsearch_templates/files/es_templates/bro_index.template -curl -s -XPOST $ES_HOST/_template/snort_index -d @roles/metron_elasticsearch_templates/files/es_templates/snort_index.template -curl -s -XPOST $ES_HOST/_template/yaf_index -d @roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template -``` - -(4) Delete the existing indexes. Only a new index will use the templates defined in the previous step. - -``` -curl -s -XDELETE "$ES_HOST/yaf_index*" -curl -s -XDELETE "$ES_HOST/bro_index*" -curl -s -XDELETE "$ES_HOST/snort_index*" -``` - -(5) Open up Kibana and wait for the new indexes to be created. The dashboard should now work. +1. Define which Elasticsearch host to interact with. Any Elasticsearch host should work. + ``` + export ES_HOST="http://ec2-52-25-237-20.us-west-2.compute.amazonaws.com:9200"; + ``` + +1. Confirm the index templates are in fact missing. + ``` + curl -s -XGET $ES_HOST/_template + ``` + +1. Manually load the index templates. + ``` + cd metron-deployment + curl -s -XPOST $ES_HOST/_template/bro_index -d @roles/metron_elasticsearch_templates/files/es_templates/bro_index.template + curl -s -XPOST $ES_HOST/_template/snort_index -d @roles/metron_elasticsearch_templates/files/es_templates/snort_index.template + curl -s -XPOST $ES_HOST/_template/yaf_index -d @roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template + ``` + +1. Delete the existing indexes. Only a new index will use the templates defined in the previous step. + ``` + curl -s -XDELETE "$ES_HOST/yaf_index*" + curl -s -XDELETE "$ES_HOST/bro_index*" + curl -s -XDELETE "$ES_HOST/snort_index*" + ``` + +1. Open up Kibana and wait for the new indexes to be created. The dashboard should now work. ### Error: 'No handler was ready to authenticate...Check your credentials' http://git-wip-us.apache.org/repos/asf/metron/blob/d5abcf42/metron-platform/metron-parsers/README.md ---------------------------------------------------------------------- diff --git a/metron-platform/metron-parsers/README.md b/metron-platform/metron-parsers/README.md index 97e8e5f..e5d97e0 100644 --- a/metron-platform/metron-parsers/README.md +++ b/metron-platform/metron-parsers/README.md @@ -38,7 +38,6 @@ All Metron messages follow a specific format in order to ingest a message. If a ``` {"message" : message content} - ``` Where appropriate there is also a standardization around the 5-tuple JSON fields. This is done so the topology correlation engine further down stream can correlate messages from different topologies by these fields. We are currently working on expanding the message standardization beyond these fields, but this feature is not yet availabe. The standard field names are as follows: @@ -66,7 +65,6 @@ So putting it all together a typical Metron message with all 5-tuple fields pres "original_string": xxx, "additional-field 1": xxx, } - } ```
