Author: mattf
Date: Thu Apr 20 05:38:46 2017
New Revision: 19216
Log:
Adding book-site for Metron 0.4.0-RC1 (incubating)
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/Upgrading.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/apache-maven-fluido-1.3.0.min.css
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/print.css
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/site.css
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ApacheIncubating_Logo.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/accessories-text-editor.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/add.gif (with
props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site-properties.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/apache-maven-project-2.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/application-certificate.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/contact-new.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/custom-storm-site-final.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/document-properties.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/drive-harddisk.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-configure-kerberos.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-started.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enrichment_arch.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/fix.gif (with
props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_error_sml.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_help_sml.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_info_sml.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_success_sml.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_warning_sml.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/image-x-generic.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/indexing_arch.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/internet-web-browser.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-black.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-white.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/maven-feather.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/maas_arch.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/metron-logo.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/network-server.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/package-x-generic.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/parser_arch.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/pre-release.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/retired.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/sandbox.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/remove.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/rss.png (with
props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/update.gif
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/window-new.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings-white.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings.png
(with props)
dev/incubator/metron/0.4.0-RC1-incubating/book-site/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/js/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/js/apache-maven-fluido-1.3.0.min.js
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-maas-service/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-maas-service/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-profiler/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-profiler-client/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-profiler-client/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-profiler/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-statistics/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-statistics/HLLP.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-analytics/metron-statistics/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/amazon-ec2/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/amazon-ec2/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/docker/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/docker/ansible-docker/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/docker/ansible-docker/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/docker/rpm-docker/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/docker/rpm-docker/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/packer-build/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/packaging/packer-build/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/monit/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/monit/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/opentaxii/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/opentaxii/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/pcap_replay/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/pcap_replay/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/sensor-stubs/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/sensor-stubs/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/sensor-test-mode/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/roles/sensor-test-mode/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/Kerberos-setup.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/codelab-platform/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/codelab-platform/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/fastcapa-test-platform/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/fastcapa-test-platform/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/full-dev-platform/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/full-dev-platform/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/quick-dev-platform/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-deployment/vagrant/quick-dev-platform/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-docker/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-docker/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-interface/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-interface/metron-config/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-interface/metron-config/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-interface/metron-rest/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-interface/metron-rest/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-api/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-api/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-common/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-common/3rdPartyStellar.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-common/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-data-management/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-data-management/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-enrichment/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-enrichment/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-indexing/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-indexing/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-management/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-management/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-parsers/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-parsers/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-pcap-backend/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-pcap-backend/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-writer/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-platform/metron-writer/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/bro-plugin-kafka/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/bro-plugin-kafka/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/fastcapa/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/fastcapa/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/index.html
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/pycapa/
dev/incubator/metron/0.4.0-RC1-incubating/book-site/metron-sensors/pycapa/index.html
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/Upgrading.html
==============================================================================
--- dev/incubator/metron/0.4.0-RC1-incubating/book-site/Upgrading.html (added)
+++ dev/incubator/metron/0.4.0-RC1-incubating/book-site/Upgrading.html Thu Apr
20 05:38:46 2017
@@ -0,0 +1,314 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-04-19
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170419" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – Upgrading</title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript"
src="./js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $(
'.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a
href="http://metron.incubator.apache.org/"; id="bannerLeft">
+
<img src="images/metron-logo.png" alt="Apache Metron -
Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a
href="http://incubator.apache.org/"; id="bannerRight">
+
<img src="images/ApacheIncubating_Logo.png" alt="Apache
Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org"; class="externalLink"
title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/";
class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">Upgrading</li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published:
2017-04-19</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.4.0</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li>
+
+ <a href="index.html" title="Metron">
+ <i class="icon-chevron-down"></i>
+ Metron</a>
+ <ul class="nav nav-list">
+
+ <li class="active">
+
+ <a href="#"><i class="none"></i>Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="metron-analytics/index.html"
title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="metron-deployment/index.html"
title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="metron-interface/metron-config/index.html"
title="Config">
+ <i class="none"></i>
+ Config</a>
+ </li>
+
+ <li>
+
+ <a href="metron-interface/metron-rest/index.html"
title="Rest">
+ <i class="none"></i>
+ Rest</a>
+ </li>
+
+ <li>
+
+ <a href="metron-platform/index.html"
title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/"; title="Built
by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven"
src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <h1>Upgrading</h1>
+<p><a name="Upgrading"></a></p>
+<p>This document constitutes a per-version listing of changes of configuration
which are non-backwards compatible.</p>
+<div class="section">
+<h2><a name="a0.3.0_to_0.3.1"></a>0.3.0 to 0.3.1</h2>
+<div class="section">
+<h3><a
name="METRON-664:_Make_the_index_configuration_per-writer_with_enableddisabled"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-664";>METRON-664: Make the
index configuration per-writer with enabled/disabled</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0 the indexing configuration</p>
+
+<ul>
+
+<li>Is held in the enrichment configuration for a sensor</li>
+
+<li>Has properties which control every writers (i.e. HDFS, solr or
elasticsearch).</li>
+</ul>
+<p>In the 0.3.1 release, this configuration has been broken out and control
for individual writers are separated.</p>
+<p>Please see the description of the configurations in the indexing <a
class="externalLink"
href="https://github.com/apache/incubator-metron/tree/Metron_0.3.1/metron-platform/metron-indexing#sensor-indexing-configuration";>README</a></p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>Migrate the configurations from each sensor enrichment configuration and
create appropriate configurations for indexing.</p>
+<p>For instance, if a sensor enrichment config for sensor <tt>foo</tt> is in
<tt>$METRON_HOME/config/zookeeper/enrichments/foo.json</tt> and looks like</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "index" : "foo",
+ "batchSize" : 100
+}
+</pre></div></div>
+<p>You would create a file to configure each writer for sensor <tt>foo</tt>
called <tt>$METRON_HOME/config/zookeeper/indexing/foo.json</tt> with the
contents</p>
+
+<div class="source">
+<div class="source">
+<pre>{
+ "elasticsearch" : {
+ "index" : "foo",
+ "batchSize" : 100,
+ "enabled" : true
+ },
+ "hdfs" : {
+ "index" : "foo",
+ "batchSize" : 100,
+ "enabled" : true
+ }
+}
+</pre></div></div></div></div>
+<div class="section">
+<h3><a
name="METRON-675:_Make_Threat_Triage_rules_able_to_be_assigned_names_and_comments"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-675";>METRON-675: Make Threat
Triage rules able to be assigned names and comments</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0, threat triage rules were defined as a simple Map associating a
Stellar expression with a score. As of 0.3.1, due to the fact that there may be
many threat triage rules, we have made the rules more complex. To help organize
these, we have made the threat triage objects in their own right that contain
optional name and optional comment fields.</p>
+<p>This essentially makes the risk level rules slightly more complex. The
format goes from:</p>
+
+<div class="source">
+<div class="source">
+<pre>"riskLevelRules" : {
+ "stellar expression" : numeric score
+}
+</pre></div></div>
+<p>to:</p>
+
+<div class="source">
+<div class="source">
+<pre>"riskLevelRules" : [
+ {
+ "name" : "optional name",
+ "comment" : "optional comment",
+ "rule" : "stellar expression",
+ "score" : numeric score
+ }
+]
+</pre></div></div></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>For every sensor enrichment configuration, you will need to migrate the
<tt>riskLevelRules</tt> section to move from a map to a list of risk level rule
objects.</p></div></div>
+<div class="section">
+<h3><a name="METRON-283:_Migrate_Geo_Enrichment_outside_of_MySQL"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-283";>METRON-283: Migrate Geo
Enrichment outside of MySQL</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>As of 0.3.0, a MySQL database was used for storage and retrieval of GeoIP
information during enrichment. As of 0.3.1, the MySQL database is removed in
favor of using MaxMind’s binary GeoIP files and stored on HDFS</p>
+<p>After initial setup, this change is transparent and existing enrichment
definitions will run as-is.</p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>While new installs will not require any additional steps, in an existing
install a script must be run to retrieve and load the initial data.</p>
+<p>The shell script <tt>geo_enrichment_load.sh</tt> will retrieve MaxMind
GeoLite2 data and load data into HDFS, and update the configuration to point to
this data. In most cases the following usage will grab the data
appropriately:</p>
+
+<div class="source">
+<div class="source">
+<pre>$METRON_HOME/bin/geo_enrichment_load.sh -z
<zk_server>:<zk_port>
+</pre></div></div>
+<p>Additional options, including changing the source file location (which can
be a file:// location if the GeoIP data is already downloaded), are available
with the -h flag and are also detailed in the metron-data-management README.me
file.</p>
+<p>One caveat is that this script will NOT update on disk config files. It is
recommended to retrieve the configuration using</p>
+
+<div class="source">
+<div class="source">
+<pre>$METRON_HOME/bin/zk_load_configs.sh -z <zk_server>:<zk_port>
-m DUMP
+</pre></div></div>
+<p>The new config will be <tt>geo.hdfs.file</tt> in the global section of the
configuration. Append this key-value into the global.json in the config
directory. A PUSH is unnecessary</p></div></div>
+<div class="section">
+<h3><a
name="METRON-684:_Decouple_Timestamp_calculation_from_PROFILE_GET"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-684";>METRON-684: Decouple
Timestamp calculation from PROFILE_GET</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>During 0.3.1 we decoupled specifying durations for calls to the profiler
into a separate function. The consequence is that existing calls to
<tt>PROFILE_GET</tt> will need to migrate.</p></div>
+<div class="section">
+<h4><a name="Migration"></a>Migration</h4>
+<p>Existing calls to <tt>PROFILE_GET</tt> will need to change from
<tt>PROFILE_GET('profile', 'entity', duration, 'durationUnits')</tt> to
<tt>PROFILE_GET('profile', 'entity', PROFILE_FIXED(duration,
'durationUnits'))</tt></p></div></div></div>
+<div class="section">
+<h2><a name="a0.2.0BETA_to_0.3.0"></a>0.2.0BETA to 0.3.0</h2>
+<div class="section">
+<h3><a
name="METRON-447:_Monit_fails_to_reload_when_upgrading_from_0.2.0BETA_to_master"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-447";>METRON-447: Monit fails
to reload when upgrading from 0.2.0BETA to master</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p><tt>/etc/monit.d/enrichment-elasticsearch.monit</tt> was renamed to
<tt>/etc/monit.d/indexing-elasticsearch.monit</tt>, however the old file
isn’t removed via ansible, which causes the below error during an
upgrade: <tt>Starting monit: /etc/monit.d/enrichment-elasticsearch.monit:18:
Service
+name conflict, enrichment already defined
+'/usr/local/monit/status_enrichment_topology.sh'</tt></p></div></div>
+<div class="section">
+<h3><a
name="METRON-448:Upgrading_via_Ansible_deployment_does_not_add_topology.classpath"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-448";>METRON-448:Upgrading
via Ansible deployment does not add topology.classpath </a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>When using Ansible to deploy the latest Metron bits to an existing
installation, storm-site is not being updated with the new 0.2.1BETA parameter
<tt>topology.classpath</tt>. Topologies are unable to find the client configs
as a result.</p></div>
+<div class="section">
+<h4><a name="Workaround"></a>Workaround</h4>
+<p>Set the <tt>topology.classpath</tt> property for storm in Ambari to
<tt>/etc/hbase/conf:/etc/hadoop/conf</tt></p></div></div></div>
+<div class="section">
+<h2><a name="a0.3.1_to_PLACEHOLDER"></a>0.3.1 to PLACEHOLDER</h2>
+<div class="section">
+<h3><a
name="METRON-671:_Refactor_existing_Ansible_deployment_to_use_Ambari_MPack"></a><a
class="externalLink"
href="https://issues.apache.org/jira/browse/METRON-671";>METRON-671: Refactor
existing Ansible deployment to use Ambari MPack</a></h3>
+<div class="section">
+<h4><a name="Description"></a>Description</h4>
+<p>Since the Ansible Deployment uses the MPack, RPMs must be built prior to
deployment. As a result, <a class="externalLink"
href="https://www.docker.com/";>Docker</a> is required to perform a Quick-Dev,
Full-Dev or Ansible deployment. This effectively limits the build environment
to Docker supported <a class="externalLink"
href="https://docs.docker.com/engine/installation/#platform-support-matrix";>platforms</a>.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/apache-maven-fluido-1.3.0.min.css
==============================================================================
---
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/apache-maven-fluido-1.3.0.min.css
(added)
+++
dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/apache-maven-fluido-1.3.0.min.css
Thu Apr 20 05:38:46 2017
@@ -0,0 +1,9 @@
+/*!
+ * Bootstrap v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
[... 3 lines stripped ...]
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/print.css
==============================================================================
--- dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/print.css (added)
+++ dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/print.css Thu Apr
20 05:38:46 2017
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/* $Id: print.css 1201871 2011-11-14 20:18:24Z simonetripodi $ */
+
+#banner, #footer, #leftcol, #breadcrumbs, .docs #toc, .docs .courtesylinks,
#leftColumn, #navColumn {display: none !important;}
+#bodyColumn, body.docs div.docs {margin: 0 !important;border: none !important}
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/site.css
==============================================================================
--- dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/site.css (added)
+++ dev/incubator/metron/0.4.0-RC1-incubating/book-site/css/site.css Thu Apr 20
05:38:46 2017
@@ -0,0 +1 @@
+/* You can override this file with your own styles */
\ No newline at end of file
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ApacheIncubating_Logo.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ApacheIncubating_Logo.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/accessories-text-editor.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/accessories-text-editor.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/add.gif
==============================================================================
Binary file - no diff available.
Propchange: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/add.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site-properties.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site-properties.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/ambari-storm-site.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/apache-maven-project-2.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/apache-maven-project-2.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/application-certificate.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/application-certificate.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/contact-new.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/contact-new.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/custom-storm-site-final.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/custom-storm-site-final.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/document-properties.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/document-properties.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/drive-harddisk.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/drive-harddisk.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-configure-kerberos.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-configure-kerberos.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-started.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos-started.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enable-kerberos.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enrichment_arch.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/enrichment_arch.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/fix.gif
==============================================================================
Binary file - no diff available.
Propchange: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/fix.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_error_sml.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_error_sml.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_help_sml.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_help_sml.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_info_sml.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_info_sml.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_success_sml.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_success_sml.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_warning_sml.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/icon_warning_sml.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/image-x-generic.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/image-x-generic.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/indexing_arch.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/indexing_arch.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/internet-web-browser.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/internet-web-browser.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-black.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-black.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-white.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/build-by-maven-white.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/maven-feather.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/logos/maven-feather.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/maas_arch.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/maas_arch.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/metron-logo.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/metron-logo.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/network-server.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/network-server.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/package-x-generic.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/package-x-generic.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/parser_arch.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/parser_arch.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/pre-release.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/pre-release.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/retired.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/retired.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/sandbox.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/profiles/sandbox.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/remove.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/remove.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/rss.png
==============================================================================
Binary file - no diff available.
Propchange: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/rss.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/update.gif
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/update.gif
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/window-new.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/images/window-new.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings-white.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings-white.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings.png
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/metron/0.4.0-RC1-incubating/book-site/img/glyphicons-halflings.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: dev/incubator/metron/0.4.0-RC1-incubating/book-site/index.html
==============================================================================
--- dev/incubator/metron/0.4.0-RC1-incubating/book-site/index.html (added)
+++ dev/incubator/metron/0.4.0-RC1-incubating/book-site/index.html Thu Apr 20
05:38:46 2017
@@ -0,0 +1,273 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2017-04-19
+ | Rendered using Apache Maven Fluido Skin 1.3.0
+-->
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20170419" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Metron – </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript"
src="./js/apache-maven-fluido-1.3.0.min.js"></script>
+
+
+
+<script type="text/javascript">$( document ).ready( function() { $(
'.carousel' ).carousel( { interval: 3500 } ) } );</script>
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a
href="http://metron.incubator.apache.org/"; id="bannerLeft">
+
<img src="images/metron-logo.png" alt="Apache Metron -
Incubating" width="148px" height="48px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <a
href="http://incubator.apache.org/"; id="bannerRight">
+
<img src="images/ApacheIncubating_Logo.png" alt="Apache
Incubating" width="192px" height="48px"/>
+ </a>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="http://www.apache.org"; class="externalLink"
title="Apache">
+ Apache</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="http://metron.incubator.apache.org/";
class="externalLink" title="Metron-Incubating">
+ Metron-Incubating</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class="">
+ <a href="index.html" title="Documentation">
+ Documentation</a>
+ </li>
+ <li class="divider ">/</li>
+ <li class=""></li>
+
+
+
+ <li id="publishDate" class="pull-right">Last Published:
2017-04-19</li> <li class="divider pull-right">|</li>
+ <li id="projectVersion" class="pull-right">Version: 0.4.0</li>
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">User Documentation</li>
+
+ <li class="active">
+
+ <a href="#"><i class="icon-chevron-down"></i>Metron</a>
+ <ul class="nav nav-list">
+
+ <li>
+
+ <a href="Upgrading.html" title="Upgrading">
+ <i class="none"></i>
+ Upgrading</a>
+ </li>
+
+ <li>
+
+ <a href="metron-analytics/index.html"
title="Analytics">
+ <i class="icon-chevron-right"></i>
+ Analytics</a>
+ </li>
+
+ <li>
+
+ <a href="metron-deployment/index.html"
title="Deployment">
+ <i class="icon-chevron-right"></i>
+ Deployment</a>
+ </li>
+
+ <li>
+
+ <a href="metron-docker/index.html" title="Docker">
+ <i class="none"></i>
+ Docker</a>
+ </li>
+
+ <li>
+
+ <a href="metron-interface/metron-config/index.html"
title="Config">
+ <i class="none"></i>
+ Config</a>
+ </li>
+
+ <li>
+
+ <a href="metron-interface/metron-rest/index.html"
title="Rest">
+ <i class="none"></i>
+ Rest</a>
+ </li>
+
+ <li>
+
+ <a href="metron-platform/index.html"
title="Platform">
+ <i class="icon-chevron-right"></i>
+ Platform</a>
+ </li>
+
+ <li>
+
+ <a href="metron-sensors/index.html" title="Sensors">
+ <i class="icon-chevron-right"></i>
+ Sensors</a>
+ </li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+ <hr class="divider" />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/"; title="Built
by Maven" class="poweredBy">
+ <img class="builtBy" alt="Built by Maven"
src="./images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+ <p><a class="externalLink"
href="https://travis-ci.org/apache/incubator-metron";><img
src="https://travis-ci.org/apache/incubator-metron.svg?branch=master";
alt="Build Status" /></a></p>
+<h1>Apache Metron (Incubating)</h1>
+<p><a name="Apache_Metron_Incubating"></a></p>
+<p>Metron integrates a variety of open source big data technologies in order
to offer a centralized tool for security monitoring and analysis. Metron
provides capabilities for log aggregation, full packet capture indexing,
storage, advanced behavioral analytics and data enrichment, while applying the
most current threat intelligence information to security telemetry within a
single platform.</p>
+<p>For the latest information, please visit our website at <a
class="externalLink"
href="http://metron.incubator.apache.org/";>http://metron.incubator.apache.org/</a></p>
+<p>Metron can be divided into 4 areas:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p><b>A mechanism to capture, store, and normalize any type of security
telemetry at extremely high rates.</b> Because security telemetry is constantly
being generated, it requires a method for ingesting the data at high speeds and
pushing it to various processing units for advanced computation and
analytics.</p></li>
+
+<li>
+<p><b>Real time processing and application of enrichments</b> such as threat
intelligence, geolocation, and DNS information to telemetry being collected.
The immediate application of this information to incoming telemetry provides
the context and situational awareness, as well as the who and where information
critical for investigation</p></li>
+
+<li>
+<p><b>Efficient information storage</b> based on how the information will be
used:</p>
+
+<ul>
+
+<li>Logs and telemetry are stored such that they can be efficiently mined and
analyzed for concise security visibility</li>
+
+<li>The ability to extract and reconstruct full packets helps an analyst
answer questions such as who the true attacker was, what data was leaked, and
where that data was sent</li>
+
+<li>Long-term storage not only increases visibility over time, but also
enables advanced analytics such as machine learning techniques to be used to
create models on the information. Incoming data can then be scored against
these stored models for advanced anomaly detection.</li>
+ </ul></li>
+
+<li>
+<p><b>An interface that gives a security investigator a centralized view of
data and alerts passed through the system.</b> Metron’s interface
presents alert summaries with threat intelligence and enrichment data specific
to that alert on one single page. Furthermore, advanced search capabilities and
full packet extraction tools are presented to the analyst for investigation
without the need to pivot into additional tools. </p></li>
+</ol>
+<p>Big data is a natural fit for powerful security analytics. The Metron
framework integrates a number of elements from the Hadoop ecosystem to provide
a scalable platform for security analytics, incorporating such functionality as
full-packet capture, stream processing, batch processing, real-time search, and
telemetry aggregation. With Metron, our goal is to tie big data into security
analytics and drive towards an extensible centralized platform to effectively
enable rapid detection and rapid response for advanced security threats. </p>
+<p><a name="Obtaining_Metron"></a></p>
+<h1>Obtaining Metron</h1>
+<p>To obtain a release of Metron, please visit <a class="externalLink"
href="http://metron.incubator.apache.org/documentation/#releases";>http://metron.incubator.apache.org/documentation/#releases</a></p>
+<p>This repository is a collection of submodules for convenience which is
regularly updated to point to the latest versions. Github provides multiple
ways to obtain Metron’s code:</p>
+
+<ol style="list-style-type: decimal">
+
+<li>git clone –recursive <a class="externalLink"
href="https://github.com/apache/incubator-metron";>https://github.com/apache/incubator-metron</a></li>
+
+<li><a class="externalLink"
href="https://github.com/apache/incubator-metron/archive/master.zip";>Download
ZIP</a></li>
+
+<li>Clone or download each repository individually</li>
+</ol>
+<p>Option 3 is more likely to have the latest code.</p>
+<p><a name="Building_Metron"></a></p>
+<h1>Building Metron</h1>
+<p>Build the full project and run tests:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install
+</pre></div></div>
+<p>Build without tests:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install -DskipTests
+</pre></div></div>
+<p>Build with the HDP profile:</p>
+
+<div class="source">
+<div class="source">
+<pre>$ mvn clean install -PHDP-2.5.0.0
+</pre></div></div>
+<p>You can swap “install” for “package” in the
commands above if you don’t want to deploy the artifacts to your local
.m2 repo.</p>
+<p><a name="Navigating_the_Architecture"></a></p>
+<h1>Navigating the Architecture</h1>
+<p>Metron is at its core a Kappa architecture with Apache Storm as the
processing component and Apache Kafka as the unified data bus.</p>
+<p>Some high level links to the relevant subparts of the architecture, for
more information:</p>
+
+<ul>
+
+<li><a href="metron-platform/metron-parsers/index.html">Parsers</a> : Parsing
data from kafka into the Metron data model and passing it downstream to
Enrichment.</li>
+
+<li><a href="metron-platform/metron-enrichment/index.html">Enrichment</a> :
Enriching data post-parsing and providing the ability to tag a message as an
alert and assign a risk triage level via a custom rule language.</li>
+
+<li><a href="metron-platform/metron-indexing/index.html">Indexing</a> :
Indexing the data post-enrichment into HDFS, Elasticsearch or Solr.</li>
+</ul>
+<p>Some useful utilities that cross all of these parts of the architecture:</p>
+
+<ul>
+
+<li><a href="metron-platform/metron-common/index.html">Stellar</a> : A custom
data transformation language that is used throughout metron from simple field
transformation to expressing triage rules.</li>
+
+<li><a href="metron-analytics/metron-maas-service/index.html">Model as a
Service</a> : A Yarn application which can deploy machine learning and
statistical models onto the cluster along with the associated Stellar functions
to be able to call out to them in a scalable manner.</li>
+
+<li><a href="metron-platform/metron-data-management/index.html">Data
management</a> : A set of data management utilities aimed at getting data into
HBase in a format which will allow data flowing through metron to be enriched
with the results. Contains integrations with threat intelligence feeds exposed
via TAXII as well as simple flat file structures.</li>
+
+<li><a href="metron-analytics/metron-profiler/index.html">Profiler</a> : A
feature extraction mechanism that can generate a profile describing the
behavior of an entity. An entity might be a server, user, subnet or
application. Once a profile has been generated defining what normal behavior
looks-like, models can be built that identify anomalous behavior.</li>
+</ul>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row span12">Copyright © 2017.
+ All Rights Reserved.
+
+ </div>
+
+
+
+ </div>
+ </footer>
+ </body>
+</html>
![https://travis-ci.org/apache/incubator-metron.svg?branch=master"](https://travis-ci.org/apache/incubator-metron.svg?branch=master"){kind=link}