metron git commit: METRON-1038: Stellar should have a better collection of basic math operations closes apache/incubator-metron#650

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 27ab28923 -> b2375a1f1


METRON-1038: Stellar should have a better collection of basic math operations 
closes apache/incubator-metron#650


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/b2375a1f
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/b2375a1f
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/b2375a1f

Branch: refs/heads/master
Commit: b2375a1f1c66b19dc6a82cbcc8242741cdf65451
Parents: 27ab289
Author: cstella 
Authored: Thu Aug 3 10:17:59 2017 +0100
Committer: cstella 
Committed: Thu Aug 3 10:17:59 2017 +0100

--
 metron-stellar/stellar-common/README.md | 185 +--
 .../common/utils/math/MathOperation.java|  45 +
 .../common/utils/math/MathOperations.java   |  83 +
 .../common/utils/math/StellarMathFunction.java  |  64 +++
 .../stellar/dsl/functions/MathFunctions.java| 166 +++--
 .../dsl/functions/MathFunctionsTest.java| 162 +++-
 6 files changed, 631 insertions(+), 74 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/b2375a1f/metron-stellar/stellar-common/README.md
--
diff --git a/metron-stellar/stellar-common/README.md 
b/metron-stellar/stellar-common/README.md
index 2b5d4b6..f12c77d 100644
--- a/metron-stellar/stellar-common/README.md
+++ b/metron-stellar/stellar-common/README.md
@@ -101,58 +101,65 @@ In the core language functions, we support basic 
functional programming primitiv
 
 ## Stellar Core Functions
 
-|  
  |
-| --   
  |
-| [ `ABS`](../../metron-analytics/metron-statistics#abs)   
  |
-| [ `APPEND_IF_MISSING`](#append_if_missing)   
  |
-| [ `BIN`](../../metron-analytics/metron-statistics#bin)   
  |
-| [ `BLOOM_ADD`](#bloom_add)   
  |
-| [ `BLOOM_EXISTS`](#bloom_exists) 
  |
-| [ `BLOOM_INIT`](#bloom_init) 
  |
-| [ `BLOOM_MERGE`](#bloom_merge)   
  |
-| [ `CHOP`](#chop) 
  |
-| [ `CHOMP`](#chomp)   
  |
-| [ `COUNT_MATCHES`](#count_matches)   
  |
-| [ `DAY_OF_MONTH`](#day_of_month) 
  |
-| [ `DAY_OF_WEEK`](#day_of_week)   
  |
-| [ `DAY_OF_YEAR`](#day_of_year)   
  |
-| [ `DECODE`](#decode) 
  |
-| [ `DOMAIN_REMOVE_SUBDOMAINS`](#domain_remove_subdomains) 
  |
-| [ `DOMAIN_REMOVE_TLD`](#domain_remove_tld)   
  |
-| [ `DOMAIN_TO_TLD`](#domain_to_tld)   
  |
-| [ `ENCODE`](#encode) 
  |
-| [ `ENDS_WITH`](#ends_with)   
  |
-| [ `ENRICHMENT_EXISTS`](#enrichment_exists)   
  |
-| [ `ENRICHMENT_GET`](#enrichment_get) 
  |
-| [ `FILL_LEFT`](#fill_left)   
  |
-| [ `FILL_RIGHT`](#fill_right) 
  |
-| [ `FILTER`](#filter) 
  |
-| [ `FORMAT`](#format) 
  |
-| [ 
`HLLP_CARDINALITY`](../../metron-analytics/metron-statistics#hllp_cardinality)  
 |
-| [ `HLLP_INIT`](../../metron-analytics/metron-statistics#hllp_init)   
  |
-| [ `HLLP_MERGE`](../../metron-analytics/metron-statistics#hllp_merge) 
  |
-| [ 

metron git commit: METRON-1102: Add support for ingesting cybox URI observables from taxii feeds closes apache/incubator-metron#689

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master b3148a182 -> 3d95fdf90


METRON-1102: Add support for ingesting cybox URI observables from taxii feeds 
closes apache/incubator-metron#689


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/3d95fdf9
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/3d95fdf9
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/3d95fdf9

Branch: refs/heads/master
Commit: 3d95fdf906993c8d65b76a7e1311c7fb295e19b1
Parents: b3148a1
Author: cstella 
Authored: Fri Aug 11 13:29:07 2017 -0400
Committer: cstella 
Committed: Fri Aug 11 13:29:07 2017 -0400

--
 .../metron-data-management/README.md|  6 ++
 .../dataloads/extractor/stix/StixExtractor.java | 15 +++
 .../stix/types/ObjectTypeHandlers.java  |  1 +
 .../extractor/stix/types/URIHandler.java| 68 ++
 .../extractor/stix/URIHandlerTest.java  | 97 
 5 files changed, 187 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/3d95fdf9/metron-platform/metron-data-management/README.md
--
diff --git a/metron-platform/metron-data-management/README.md 
b/metron-platform/metron-data-management/README.md
index 014f3ef..c351f48 100644
--- a/metron-platform/metron-data-management/README.md
+++ b/metron-platform/metron-data-management/README.md
@@ -97,6 +97,11 @@ Consider the following config for importing STIX documents.  
This is a threat in
 format, so it is particularly relevant and attractive data to import for our 
purposes.  Because STIX is
 a standard format, there is no need to specify the schema or how to interpret 
the documents.
 
+We support the versions of Stix and Cybox supported by
+[java-stix](https://github.com/STIXProject/java-stix/tree/v1.2.0.2):
+* Stix - 
[1.2](https://github.com/STIXProject/schemas/blob/356cc4f6b06625465f0808388eb166807313b4e0/stix_core.xsd)
 and earlier
+* Cybox - 
[2.1](https://github.com/CybOXProject/schemas/blob/97beb32c376a9223e91b52cb3e4c8d2af6baf786/cybox_core.xsd)
 and earlier
+
 We support a subset of STIX messages for importation:
 
 | STIX Type | Specific Type | Enrichment Type Name |
@@ -107,6 +112,7 @@ We support a subset of STIX messages for importation:
 | Address   | MAC   | address:MAC  |
 | Domain| FQDN  | domain:FQDN  |
 | Hostname  |   | hostname |
+| URI   |   | uriobjecttype|
 
 
 NOTE: The enrichment type will be used as the type above.

http://git-wip-us.apache.org/repos/asf/metron/blob/3d95fdf9/metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
--
diff --git 
a/metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
 
b/metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
index 41b428e..f96bd2d 100644
--- 
a/metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
+++ 
b/metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
@@ -30,14 +30,18 @@ import org.mitre.cybox.cybox_2.Observables;
 import org.mitre.stix.common_1.IndicatorBaseType;
 import org.mitre.stix.indicator_2.Indicator;
 import org.mitre.stix.stix_1.STIXPackage;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 import java.io.File;
 import java.io.IOException;
+import java.lang.invoke.MethodHandles;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
 
 public class StixExtractor implements Extractor {
+private static final Logger LOG = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
 Map config;
 @Override
 public Iterable extract(String line) throws IOException {
@@ -50,11 +54,22 @@ public class StixExtractor implements Extractor {
 if(props != null) {
 ObjectTypeHandler handler = 
ObjectTypeHandlers.getHandlerByInstance(props);
 if (handler != null) {
+if(LOG.isDebugEnabled()) {
+LOG.debug("Found {} for properties {}"
+, handler.getTypeClass().getCanonicalName()
+, props.toXMLString());
+}
 Iterable extractions = 
handler.extract(props, config);
 for(LookupKV extraction : extractions) {
 

metron git commit: METRON-1094: MaaS will not start due to classpath error relating to Yarn closes apache/incubator-metron#688

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 3d95fdf90 -> 82bf89ba4


METRON-1094: MaaS will not start due to classpath error relating to Yarn closes 
apache/incubator-metron#688


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/82bf89ba
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/82bf89ba
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/82bf89ba

Branch: refs/heads/master
Commit: 82bf89ba4ada583ac68dbb44e7fd658ad45e8337
Parents: 3d95fdf
Author: cstella 
Authored: Fri Aug 11 13:30:04 2017 -0400
Committer: cstella 
Committed: Fri Aug 11 13:30:04 2017 -0400

--
 metron-analytics/metron-maas-service/pom.xml | 19 ---
 .../metron/maas/functions/MaaSFunctions.java |  2 +-
 .../apache/metron/maas/service/MockDGAModel.java |  2 +-
 3 files changed, 18 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/82bf89ba/metron-analytics/metron-maas-service/pom.xml
--
diff --git a/metron-analytics/metron-maas-service/pom.xml 
b/metron-analytics/metron-maas-service/pom.xml
index e76aec2..0ac9bac 100644
--- a/metron-analytics/metron-maas-service/pom.xml
+++ b/metron-analytics/metron-maas-service/pom.xml
@@ -33,19 +33,31 @@
   
   
 
+  org.apache.hadoop
+  hadoop-yarn-api
+  ${hadoop.version}
+  provided
+
+
   org.apache.metron
   metron-maas-common
   ${project.parent.version}
 
 
-  org.apache.metron
-  metron-common
-  ${project.parent.version}
+  de.javakaffee
+  kryo-serializers
+  ${global_kryo_serializers_version}
 
 
   org.apache.metron
   stellar-common
   ${project.parent.version}
+  
+
+  org.apache.hadoop
+  hadoop-auth
+
+  
 
 
   org.apache.hadoop
@@ -59,6 +71,7 @@
   ${hadoop.version}
   provided
 
+
 
   com.sun.jersey.contribs
   jersey-guice

http://git-wip-us.apache.org/repos/asf/metron/blob/82bf89ba/metron-analytics/metron-maas-service/src/main/java/org/apache/metron/maas/functions/MaaSFunctions.java
--
diff --git 
a/metron-analytics/metron-maas-service/src/main/java/org/apache/metron/maas/functions/MaaSFunctions.java
 
b/metron-analytics/metron-maas-service/src/main/java/org/apache/metron/maas/functions/MaaSFunctions.java
index e205518..eacb64d 100644
--- 
a/metron-analytics/metron-maas-service/src/main/java/org/apache/metron/maas/functions/MaaSFunctions.java
+++ 
b/metron-analytics/metron-maas-service/src/main/java/org/apache/metron/maas/functions/MaaSFunctions.java
@@ -29,13 +29,13 @@ import java.util.Map;
 import java.util.Optional;
 import java.util.concurrent.TimeUnit;
 import org.apache.curator.framework.CuratorFramework;
-import org.apache.metron.common.utils.JSONUtils;
 import org.apache.metron.maas.config.Endpoint;
 import org.apache.metron.maas.config.MaaSConfig;
 import org.apache.metron.maas.config.ModelEndpoint;
 import org.apache.metron.maas.discovery.ServiceDiscoverer;
 import org.apache.metron.maas.util.ConfigUtil;
 import org.apache.metron.maas.util.RESTUtil;
+import org.apache.metron.stellar.common.utils.JSONUtils;
 import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.ParseException;
 import org.apache.metron.stellar.dsl.Stellar;

http://git-wip-us.apache.org/repos/asf/metron/blob/82bf89ba/metron-analytics/metron-maas-service/src/test/java/org/apache/metron/maas/service/MockDGAModel.java
--
diff --git 
a/metron-analytics/metron-maas-service/src/test/java/org/apache/metron/maas/service/MockDGAModel.java
 
b/metron-analytics/metron-maas-service/src/test/java/org/apache/metron/maas/service/MockDGAModel.java
index 18604b4..66948aa 100644
--- 
a/metron-analytics/metron-maas-service/src/test/java/org/apache/metron/maas/service/MockDGAModel.java
+++ 
b/metron-analytics/metron-maas-service/src/test/java/org/apache/metron/maas/service/MockDGAModel.java
@@ -21,7 +21,7 @@ import com.fasterxml.jackson.core.JsonProcessingException;
 import com.google.common.collect.ImmutableMap;
 import com.sun.net.httpserver.HttpHandler;
 import com.sun.net.httpserver.HttpServer;
-import org.apache.metron.common.utils.JSONUtils;
+import org.apache.metron.stellar.common.utils.JSONUtils;
 
 import javax.ws.rs.*;
 import javax.ws.rs.core.Application;

metron git commit: METRON-1108: Metron configuration tabs are not showing up in Ambari this closes apache/incubator-metron#695

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master adb378f35 -> 02bc58782


METRON-1108: Metron configuration tabs are not showing up in Ambari this closes 
apache/incubator-metron#695


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/02bc5878
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/02bc5878
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/02bc5878

Branch: refs/heads/master
Commit: 02bc58782963379c7367db1d7f0254dba7c0df36
Parents: adb378f
Author: merrimanr 
Authored: Tue Aug 15 09:57:30 2017 -0400
Committer: cstella 
Committed: Tue Aug 15 09:57:30 2017 -0400

--
 .../METRON/CURRENT/themes/metron_theme.json | 12 
 1 file changed, 12 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/02bc5878/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json
index 0e5457d..ce352a4 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json
@@ -691,6 +691,18 @@
 }
   },
   {
+"config": "metron-indexing-env/update_table",
+"widget": {
+  "type": "text-field"
+}
+  },
+  {
+"config": "metron-indexing-env/update_cf",
+"widget": {
+  "type": "text-field"
+}
+  },
+  {
 "config": "metron-indexing-env/indexing_error_topic",
 "widget": {
   "type": "text-field"

svn commit: r20351 - /release/metron/0.3.1/

[cestella]

Author: cestella
Date: Thu Jul  6 19:06:39 2017
New Revision: 20351

Log:
Deleting 0.3.1 release of Metron

Removed:
release/metron/0.3.1/

[1/2] metron git commit: METRON-1022: Elasticsearch REST endpoint this closes apache/incubator-metron#636

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 0d1923f83 -> cf7043c59


http://git-wip-us.apache.org/repos/asf/metron/blob/cf7043c5/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SearchResult.java
--
diff --git 
a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SearchResult.java
 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SearchResult.java
new file mode 100644
index 000..ae4f9bd
--- /dev/null
+++ 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SearchResult.java
@@ -0,0 +1,51 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.indexing.dao.search;
+
+import java.util.Map;
+
+public class SearchResult {
+
+  private String id;
+  private Map source;
+  private float score;
+
+  public String getId() {
+return id;
+  }
+
+  public void setId(String id) {
+this.id = id;
+  }
+
+  public Map getSource() {
+return source;
+  }
+
+  public void setSource(Map source) {
+this.source = source;
+  }
+
+  public float getScore() {
+return score;
+  }
+
+  public void setScore(float score) {
+this.score = score;
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/cf7043c5/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortField.java
--
diff --git 
a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortField.java
 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortField.java
new file mode 100644
index 000..a3473fc
--- /dev/null
+++ 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortField.java
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.indexing.dao.search;
+
+public class SortField {
+  private String field;
+  private SortOrder sortOrder;
+
+  public String getField() {
+return field;
+  }
+
+  public void setField(String field) {
+this.field = field;
+  }
+
+  public SortOrder getSortOrder() {
+return sortOrder;
+  }
+
+  public void setSortOrder(String sortOrder) {
+this.sortOrder = SortOrder.fromString(sortOrder);
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/cf7043c5/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortOrder.java
--
diff --git 
a/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortOrder.java
 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortOrder.java
new file mode 100644
index 000..fde3279
--- /dev/null
+++ 
b/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SortOrder.java
@@ -0,0 +1,41 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use 

svn commit: r20299 [12/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.0/site-book/metron-platform/index.html
==
--- release/metron/0.4.0/site-book/metron-platform/index.html (added)
+++ release/metron/0.4.0/site-book/metron-platform/index.html Wed Jul  5 
06:56:42 2017
@@ -0,0 +1,268 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Current Build
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Current Build
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+Platform
+  
+  
+  
+
+  
+  
+Api
+
+
+  
+
+  
+  
+Common
+  
+  
+  
+
+  
+  
+Data-management
+
+  
+  
+
+  
+  
+Enrichment
+
+  
+  
+
+  
+  
+Indexing
+
+  
+  
+
+  
+  
+Management
+
+  
+  
+
+  
+  
+  

svn commit: r20299 [17/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.0/site-book/metron-sensors/bro-plugin-kafka/index.html
==
--- release/metron/0.4.0/site-book/metron-sensors/bro-plugin-kafka/index.html 
(added)
+++ release/metron/0.4.0/site-book/metron-sensors/bro-plugin-kafka/index.html 
Wed Jul  5 06:56:42 2017
@@ -0,0 +1,491 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Logging Bro Output to Kafka
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Logging Bro Output to Kafka
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+  
+   
   
+  
+
+  
+  
+Sensors
+
+  
+  
+
+Bro-plugin-kafka
+  
+  
+  
+
+  
+  
+Fastcapa
+
+  
+  
+
+  
+  
+Pycapa
+
+  
+
+  
+
+
+
+
+
+  
+
+   
+

svn commit: r20299 [7/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.0/site-book/metron-analytics/metron-statistics/index.html
==
--- 
release/metron/0.4.0/site-book/metron-analytics/metron-statistics/index.html 
(added)
+++ 
release/metron/0.4.0/site-book/metron-analytics/metron-statistics/index.html 
Wed Jul  5 06:56:42 2017
@@ -0,0 +1,943 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Statistics and Mathematical Functions
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Statistics and Mathematical Functions
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   

 
+  
+
+  
+  
+Analytics
+
+  
+  
+
+  
+  
+Maas-service
+
+  
+  
+
+  
+  
+Profiler
+
+  
+  
+
+  
+  
+Profiler-client
+
+
+  
+
+Statistics
+  
+  
+  
+
+  
+  
+HLLP
+
+  
+
+  
+
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+  
+   

svn commit: r20299 [18/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.0/site-book/metron-sensors/pycapa/index.html
==
--- release/metron/0.4.0/site-book/metron-sensors/pycapa/index.html (added)
+++ release/metron/0.4.0/site-book/metron-sensors/pycapa/index.html Wed Jul  5 
06:56:42 2017
@@ -0,0 +1,548 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Pycapa
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Pycapa
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+  
+   
   
+  
+
+  
+  
+Sensors
+
+  
+  
+
+  
+  
+Bro-plugin-kafka
+
+  
+  
+
+  
+  
+Fastcapa
+
+  
+  
+
+Pycapa
+  
+  
+
+  
+
+
+
+
+
+  
+
+   
+
+
+
+   

svn commit: r20299 [13/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.0/site-book/metron-platform/metron-common/index.html
==
--- release/metron/0.4.0/site-book/metron-platform/metron-common/index.html 
(added)
+++ release/metron/0.4.0/site-book/metron-platform/metron-common/index.html Wed 
Jul  5 06:56:42 2017
@@ -0,0 +1,2752 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Contents
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Contents
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


   
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Api
+
+
+  
+
+Common
+  
+  
+  
+
+  
+  
+3rdPartyStellar
+
+  
+
+  
+  
+
+  
+  
+Data-management
+
+  
+  
+
+  
+  
+Enrichment
+
+  
+  
+
+  
+  
+Indexing
+
+   

svn commit: r20299 [14/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.0/site-book/metron-platform/metron-data-management/index.html
==
--- 
release/metron/0.4.0/site-book/metron-platform/metron-data-management/index.html
 (added)
+++ 
release/metron/0.4.0/site-book/metron-platform/metron-data-management/index.html
 Wed Jul  5 06:56:42 2017
@@ -0,0 +1,1010 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Resource Data Management
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Resource Data Management
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


   
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Api
+
+
+  
+
+  
+  
+Common
+  
+  
+  
+
+Data-management
+  
+  
+  
+
+  
+  
+Enrichment
+
+  
+  
+
+  
+  
+Indexing
+
+  
+  
+
+  
+  
+

svn commit: r20299 [4/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: release/metron/0.4.0/site-book/js/apache-maven-fluido-1.3.0.min.js
==
--- release/metron/0.4.0/site-book/js/apache-maven-fluido-1.3.0.min.js (added)
+++ release/metron/0.4.0/site-book/js/apache-maven-fluido-1.3.0.min.js Wed Jul  
5 06:56:42 2017
@@ -0,0 +1,21 @@
+/*!
+ * jQuery JavaScript Library v1.8.0
+ * http://jquery.com/
+ *
+ * Includes Sizzle.js
+ * http://sizzlejs.com/
+ *
+ * Copyright 2012 jQuery Foundation and other contributors
+ * Released under the MIT license
+ * http://jquery.org/license
+ *
+ * Date: Thu Aug 09 2012 16:24:48 GMT-0400 (Eastern Daylight Time)
+ */
+(function(a2,aB){var 
x,af,o=a2.document,aI=a2.location,d=a2.navigator,bg=a2.jQuery,I=a2.$,am=Array.prototype.push,a4=Array.prototype.slice,aK=Array.prototype.indexOf,A=Object.prototype.toString,V=Object.prototype.hasOwnProperty,aN=String.prototype.trim,bG=function(e,bZ){return
 new 
bG.fn.init(e,bZ,x)},bx=/[\-+]?(?:\d*\.|)\d+(?:[eE][\-+]?\d+|)/.source,aa=/\S/,aV=/\s+/,D=aa.test("\xA0")?(/^[\s\xA0]+|[\s\xA0]+$/g):/^\s+|\s+$/g,bo=/^(?:[^#<]*(<[\w\W]+>)[^>]*$|#([\w\-]*)$)/,a=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,bf=/^[\],:{}\s]*$/,bi=/(?:^|:|,)(?:\s*\[)+/g,bD=/\\(?:["\\\/bfnrt]|u[\da-fA-F]{4})/g,a0=/"[^"\\\r\n]*"|true|false|null|-?(?:\d\d*\.|)\d+(?:[eE][\-+]?\d+|)/g,bP=/^-ms-/,aU=/-([\da-z])/gi,N=function(e,bZ){return(bZ+"").toUpperCase()},aF=function(){if(o.addEventListener){o.removeEventListener("DOMContentLoaded",aF,false);bG.ready()}else{if(o.readyState==="complete"){o.detachEvent("onreadystatechange",aF);bG.ready()}}},Z={};bG.fn=bG.prototype={constructor:bG,init:function(e,b2,b1){var
 b0,b3,b
 Z,b4;if(!e){return 
this}if(e.nodeType){this.context=this[0]=e;this.length=1;return this}if(typeof 
e==="string"){if(e.charAt(0)==="<"&(e.length-1)===">"&>=3){b0=[null,e,null]}else{b0=bo.exec(e)}if(b0&&(b0[1]||!b2)){if(b0[1]){b2=b2
 instanceof 
bG?b2[0]:b2;b4=(b2&?b2.ownerDocument||b2:o);e=bG.parseHTML(b0[1],b4,true);if(a.test(b0[1])&(b2)){this.attr.call(e,b2,true)}return
 
bG.merge(this,e)}else{b3=o.getElementById(b0[2]);if(b3&){if(b3.id!==b0[2]){return
 b1.find(e)}this.length=1;this[0]=b3}this.context=o;this.selector=e;return 
this}}else{if(!b2||b2.jquery){return(b2||b1).find(e)}else{return 
this.constructor(b2).find(e)}}}else{if(bG.isFunction(e)){return 
b1.ready(e)}}if(e.selector!==aB){this.selector=e.selector;this.context=e.context}return
 
bG.makeArray(e,this)},selector:"",jquery:"1.8.0",length:0,size:function(){return
 this.length},toArray:function(){return a4.call(this)},get:function(e){return 
e==null?this.toArray():(e<0?this[t
 his.length+e]:this[e])},pushStack:function(bZ,b1,e){var 
b0=bG.merge(this.constructor(),bZ);b0.prevObject=this;b0.context=this.context;if(b1==="find"){b0.selector=this.selector+(this.selector?"
 ":"")+e}else{if(b1){b0.selector=this.selector+"."+b1+"("+e+")"}}return 
b0},each:function(bZ,e){return 
bG.each(this,bZ,e)},ready:function(e){bG.ready.promise().done(e);return 
this},eq:function(e){e=+e;return 
e===-1?this.slice(e):this.slice(e,e+1)},first:function(){return 
this.eq(0)},last:function(){return this.eq(-1)},slice:function(){return 
this.pushStack(a4.apply(this,arguments),"slice",a4.call(arguments).join(","))},map:function(e){return
 this.pushStack(bG.map(this,function(b0,bZ){return 
e.call(b0,bZ,b0)}))},end:function(){return 
this.prevObject||this.constructor(null)},push:am,sort:[].sort,splice:[].splice};bG.fn.init.prototype=bG.fn;bG.extend=bG.fn.extend=function(){var
 
b7,b0,e,bZ,b4,b5,b3=arguments[0]||{},b2=1,b1=arguments.length,b6=false;if(typeof
 b3==="boolean"){b6=b3;b3=arguments[1]||{
 };b2=2}if(typeof 
b3!=="object"&&!bG.isFunction(b3)){b3={}}if(b1===b2){b3=this;--b2}for(;b20){return}af.resolveWith(o,[bG]);if(bG.fn.trigger){bG(o).trigger("ready").off("ready")}},isFunction:function(e){return
 bG.type(e)==="function"},isArray:Array.isArray||function(e){return 
bG.type(e)==="array"},isWindow:function(e){return 
e!=null&==e.window},isNumeric:fu
 nction(e){return !isNaN(parseFloat(e))&(e)},type:function(e){return 
e==null?String(e):Z[A.call(e)]||"object"},isPlainObject:function(b1){if(!b1||bG.type(b1)!=="object"||b1.nodeType||bG.isWindow(b1)){return
 
false}try{if(b1.constructor&&!V.call(b1,"constructor")&&!V.call(b1.constructor.prototype,"isPrototypeOf")){return
 false}}catch(b0){return false}var bZ;for(bZ in b1){}return 

svn commit: r20299 [6/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.0/site-book/metron-analytics/metron-statistics/HLLP.html
==
--- release/metron/0.4.0/site-book/metron-analytics/metron-statistics/HLLP.html 
(added)
+++ release/metron/0.4.0/site-book/metron-analytics/metron-statistics/HLLP.html 
Wed Jul  5 06:56:42 2017
@@ -0,0 +1,7599 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  HyperLogLogPlus
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+HyperLogLogPlus
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   

 
+  
+
+  
+  
+Analytics
+
+  
+  
+
+  
+  
+Maas-service
+
+  
+  
+
+  
+  
+Profiler
+
+  
+  
+
+  
+  
+Profiler-client
+
+   
   
+  
+
+  
+  
+Statistics
+
+  
+  
+
+HLLP
+  
+  
+
+  
+
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+  
+

svn commit: r20299 [11/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.0/site-book/metron-docker/index.html
==
--- release/metron/0.4.0/site-book/metron-docker/index.html (added)
+++ release/metron/0.4.0/site-book/metron-docker/index.html Wed Jul  5 06:56:42 
2017
@@ -0,0 +1,486 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Metron Docker
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Metron Docker
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+Docker
+  
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+  
+   
 
+  
+
+  
+  
+Sensors
+  
+  
+
+
+
+
+
+  
+
+   
+
+
+
+ http://maven.apache.org/; title="Built 
by Maven" class="poweredBy">
+
+  
+  
+  
+
+
+
+
+  
+Metron Docker
+
+Metron Docker is a https://docs.docker.com/compose/overview/;>Docker Compose application 
that is intended for development and 

svn commit: r20299 [8/18] - in /release/metron/0.4.0: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: release/metron/0.4.0/site-book/metron-deployment/amazon-ec2/index.html
==
--- release/metron/0.4.0/site-book/metron-deployment/amazon-ec2/index.html 
(added)
+++ release/metron/0.4.0/site-book/metron-deployment/amazon-ec2/index.html Wed 
Jul  5 06:56:42 2017
@@ -0,0 +1,561 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Apache Metron on Amazon EC2
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Apache Metron on Amazon EC2
+
+
+
+  Last Published: 
2017-06-27 |
+  Version: 0.4.0
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   










   
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+   



   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+Amazon-ec2
+  
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+Packer-build
+
+   
 
+  
+
+  
+  
+Roles
+  
+   
   
+  
+
+  
+  
+Vagrant
+  
+  
+
+  
+  
+
+  
+  
+Docker
+
+  
+  
+
+  
+  
+Config
+ 

[2/2] incubator-metron git commit: METRON-870: Add filtering by packet payload to the pcap query closes apache/incubator-metron#541

[cestella]

METRON-870: Add filtering by packet payload to the pcap query closes 
apache/incubator-metron#541


Project: http://git-wip-us.apache.org/repos/asf/incubator-metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-metron/commit/bf2528fd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-metron/tree/bf2528fd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-metron/diff/bf2528fd

Branch: refs/heads/master
Commit: bf2528fd3f4d474164ddf497459cd2421df3e4bb
Parents: 5dd8788
Author: cstella 
Authored: Thu Apr 27 12:20:45 2017 -0400
Committer: cstella 
Committed: Thu Apr 27 12:20:45 2017 -0400

--
 dependencies_with_url.csv   |   1 +
 .../pcapservice/PcapReceiverImplRestEasy.java   |  24 ++-
 .../PcapReceiverImplRestEasyTest.java   |  85 
 .../org/apache/metron/common/Constants.java |   7 +-
 .../stellar/GeoEnrichmentFunctions.java |   2 +-
 metron-platform/metron-pcap-backend/README.md   |  29 ++-
 .../metron/pcap/query/FixedCliConfig.java   |  12 +-
 .../metron/pcap/query/FixedCliParser.java   |  15 +-
 .../org/apache/metron/utils/PcapInspector.java  |  11 +-
 .../apache/metron/pcap/FixedPcapFilterTest.java | 211 ++-
 .../apache/metron/pcap/QueryPcapFilterTest.java | 121 +--
 .../PcapTopologyIntegrationTest.java|  42 +++-
 .../apache/metron/pcap/query/PcapCliTest.java   |  47 +++--
 metron-platform/metron-pcap/pom.xml |  19 ++
 .../java/org/apache/metron/pcap/PacketInfo.java |  25 ++-
 .../java/org/apache/metron/pcap/PcapHelper.java |  38 +++-
 .../metron/pcap/filter/PcapFieldResolver.java   |   8 +-
 .../apache/metron/pcap/filter/PcapFilter.java   |   2 +-
 .../pcap/filter/fixed/FixedPcapFilter.java  |  71 ++-
 .../pcap/filter/query/QueryPcapFilter.java  |   7 +-
 .../pcap/pattern/ByteArrayMatcherFunction.java  |  63 ++
 .../pcap/pattern/ByteArrayMatchingUtil.java |  64 ++
 .../pcap/filter/fixed/FixedPcapFilterTest.java  |  22 +-
 .../pcap/pattern/ByteArrayMatchingUtilTest.java | 133 
 24 files changed, 754 insertions(+), 305 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/bf2528fd/dependencies_with_url.csv
--
diff --git a/dependencies_with_url.csv b/dependencies_with_url.csv
index 053f11a..93a19b7 100644
--- a/dependencies_with_url.csv
+++ b/dependencies_with_url.csv
@@ -293,5 +293,6 @@ 
org.atteo.classindex:classindex:jar:3.3:compile,ASLv2,https://github.com/atteo/c
 
com.squareup.okhttp:okhttp:jar:2.4.0:compile,ASLv2,https://github.com/square/okhttp
 com.squareup.okio:okio:jar:1.4.0:compile,ASLv2,https://github.com/square/okhttp
 
org.htrace:htrace-core:jar:3.0.4:compile,ASLv2,http://htrace.incubator.apache.org/
+net.byteseek:byteseek:jar:2.0.3:compile,BSD,https://github.com/nishihatapalmer/byteseek
 
org.springframework.security.kerberos:spring-security-kerberos-client:jar:1.0.1.RELEASE:compile,ASLv2,https://github.com/spring-projects/spring-security-kerberos
 
org.springframework.security.kerberos:spring-security-kerberos-core:jar:1.0.1.RELEASE:compile,ASLv2,https://github.com/spring-projects/spring-security-kerberos

http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/bf2528fd/metron-platform/metron-api/src/main/java/org/apache/metron/pcapservice/PcapReceiverImplRestEasy.java
--
diff --git 
a/metron-platform/metron-api/src/main/java/org/apache/metron/pcapservice/PcapReceiverImplRestEasy.java
 
b/metron-platform/metron-api/src/main/java/org/apache/metron/pcapservice/PcapReceiverImplRestEasy.java
index 1f3c03e..9c58813 100644
--- 
a/metron-platform/metron-api/src/main/java/org/apache/metron/pcapservice/PcapReceiverImplRestEasy.java
+++ 
b/metron-platform/metron-api/src/main/java/org/apache/metron/pcapservice/PcapReceiverImplRestEasy.java
@@ -27,6 +27,7 @@ import org.apache.log4j.Logger;
 import org.apache.metron.common.Constants;
 import org.apache.metron.common.hadoop.SequenceFileIterable;
 import org.apache.metron.common.utils.timestamp.TimestampConverters;
+import org.apache.metron.pcap.PcapHelper;
 import org.apache.metron.pcap.filter.fixed.FixedPcapFilter;
 import org.apache.metron.pcap.filter.query.QueryPcapFilter;
 import org.apache.metron.pcap.mr.PcapJob;
@@ -37,10 +38,7 @@ import javax.ws.rs.core.Context;
 import javax.ws.rs.core.MediaType;
 import javax.ws.rs.core.Response;
 import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.EnumMap;
-import java.util.List;
+import java.util.*;
 
 @Path("/")
 public class PcapReceiverImplRestEasy {
@@ -197,6 +195,7 @@ public class PcapReceiverImplRestEasy {
   @DefaultValue("-1") @QueryParam 

[3/4] metron git commit: METRON-1051: Enable the ability to update indexed messages closes apache/incubator-metron#666

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/813adf2d/metron-platform/metron-common/src/main/java/org/apache/metron/common/utils/JSONUtils.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/utils/JSONUtils.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/utils/JSONUtils.java
index c61efc5..60d009f 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/utils/JSONUtils.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/utils/JSONUtils.java
@@ -27,6 +27,7 @@ import org.json.simple.parser.JSONParser;
 import org.json.simple.parser.ParseException;
 
 import java.io.*;
+import java.util.Map;
 
 public enum JSONUtils {
   INSTANCE;
@@ -37,6 +38,15 @@ public enum JSONUtils {
   private static ThreadLocal _mapper = 
ThreadLocal.withInitial(() ->
   new 
ObjectMapper().setSerializationInclusion(JsonInclude.Include.NON_NULL));
 
+  public  T convert(Object original, Class targetClass) {
+return _mapper.get().convertValue(original, targetClass);
+  }
+
+  public ObjectMapper getMapper() {
+return _mapper.get();
+  }
+
+
   public  T load(InputStream is, TypeReference ref) throws IOException {
 return _mapper.get().readValue(is, ref);
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/813adf2d/metron-platform/metron-data-management/pom.xml
--
diff --git a/metron-platform/metron-data-management/pom.xml 
b/metron-platform/metron-data-management/pom.xml
index fbfd71a..90c2c52 100644
--- a/metron-platform/metron-data-management/pom.xml
+++ b/metron-platform/metron-data-management/pom.xml
@@ -115,6 +115,13 @@
 ${project.parent.version}
 
 
+org.apache.metron
+metron-hbase
+${project.parent.version}
+test
+test-jar
+
+
 org.mitre.taxii
 taxii
 1.1.0.1

http://git-wip-us.apache.org/repos/asf/metron/blob/813adf2d/metron-platform/metron-data-management/src/test/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiIntegrationTest.java
--
diff --git 
a/metron-platform/metron-data-management/src/test/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiIntegrationTest.java
 
b/metron-platform/metron-data-management/src/test/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiIntegrationTest.java
index 0223514..f94a02a 100644
--- 
a/metron-platform/metron-data-management/src/test/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiIntegrationTest.java
+++ 
b/metron-platform/metron-data-management/src/test/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiIntegrationTest.java
@@ -31,8 +31,9 @@ import 
org.apache.metron.dataloads.extractor.stix.StixExtractor;
 import org.apache.metron.enrichment.converter.EnrichmentConverter;
 import org.apache.metron.enrichment.converter.EnrichmentKey;
 import org.apache.metron.enrichment.converter.EnrichmentValue;
-import org.apache.metron.test.mock.MockHTable;
 import org.apache.metron.enrichment.lookup.LookupKV;
+import org.apache.metron.hbase.mock.MockHTable;
+import org.apache.metron.hbase.mock.MockHBaseTableProvider;
 import org.junit.*;
 
 import java.io.IOException;
@@ -49,7 +50,7 @@ public class TaxiiIntegrationTest {
 @AfterClass
 public static void teardown() {
 MockTaxiiService.shutdown();
-MockHTable.Provider.clear();
+MockHBaseTableProvider.clear();
 }
 
 /**
@@ -91,7 +92,7 @@ public class TaxiiIntegrationTest {
 @Test
 public void testTaxii() throws Exception {
 
-final MockHTable.Provider provider = new MockHTable.Provider();
+final MockHBaseTableProvider provider = new MockHBaseTableProvider();
 final Configuration config = HBaseConfiguration.create();
 TaxiiHandler handler = new 
TaxiiHandler(TaxiiConnectionConfig.load(taxiiConnectionConfig), new 
StixExtractor(), config ) {
 @Override
@@ -115,7 +116,7 @@ public class TaxiiIntegrationTest {
 }
 Assert.assertTrue(maliciousAddresses.contains("94.102.53.142"));
 Assert.assertEquals(numStringsMatch(MockTaxiiService.pollMsg, 
"AddressObj:Address_Value condition=\"Equal\""), maliciousAddresses.size());
-MockHTable.Provider.clear();
+MockHBaseTableProvider.clear();
 
 // Ensure that the handler can be run multiple times without 
connection issues.
 handler.run();

http://git-wip-us.apache.org/repos/asf/metron/blob/813adf2d/metron-platform/metron-elasticsearch/pom.xml
--
diff --git a/metron-platform/metron-elasticsearch/pom.xml 
b/metron-platform/metron-elasticsearch/pom.xml
index 6cef1e9..40989c6 100644
--- 

[4/4] metron git commit: METRON-1051: Enable the ability to update indexed messages closes apache/incubator-metron#666

[cestella]

METRON-1051: Enable the ability to update indexed messages closes 
apache/incubator-metron#666


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/813adf2d
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/813adf2d
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/813adf2d

Branch: refs/heads/master
Commit: 813adf2dac1b4c07c0e4ae134366c61388ff5b67
Parents: 3d47513
Author: cstella 
Authored: Mon Aug 7 14:14:44 2017 -0400
Committer: cstella 
Committed: Mon Aug 7 14:14:44 2017 -0400

--
 metron-analytics/metron-profiler-client/pom.xml |  13 +
 .../metron/profiler/client/GetProfileTest.java  |  23 +-
 .../client/HBaseProfilerClientTest.java |   4 +-
 metron-analytics/metron-profiler/pom.xml|  13 +
 .../integration/ProfilerIntegrationTest.java|  30 +-
 .../configuration/metron-indexing-env.xml   |  12 +
 .../package/scripts/indexing_commands.py|  64 ++
 .../CURRENT/package/scripts/indexing_master.py  |   5 +
 .../package/scripts/params/params_linux.py  |   5 +
 .../package/scripts/params/status_params.py |   4 +
 .../CURRENT/package/templates/global.json.j2|   4 +-
 .../METRON/CURRENT/themes/metron_theme.json |  27 +-
 metron-interface/metron-rest/README.md  |  70 ++
 metron-interface/metron-rest/pom.xml|  16 +
 .../apache/metron/rest/MetronRestConstants.java |   1 +
 .../apache/metron/rest/config/IndexConfig.java  |  40 +-
 .../rest/controller/SearchController.java   |  22 +
 .../rest/controller/UpdateController.java   |  70 ++
 .../metron/rest/service/SearchService.java  |   5 +
 .../metron/rest/service/UpdateService.java  |  29 +
 .../service/impl/IndexDaoSearchServiceImpl.java |  75 --
 .../rest/service/impl/SearchServiceImpl.java|  87 +++
 .../rest/service/impl/UpdateServiceImpl.java|  58 ++
 .../src/main/resources/application-test.yml |   7 +-
 .../src/main/resources/application.yml  |   3 +-
 .../metron-rest/src/main/scripts/metron-rest|   3 +-
 .../apache/metron/rest/config/TestConfig.java   |  19 +-
 .../rest/controller/DaoControllerTest.java  |  58 ++
 .../SearchControllerIntegrationTest.java|  51 +-
 .../UpdateControllerIntegrationTest.java| 192 +
 .../src/test/resources/zookeeper/global.json|   4 +
 .../apache/metron/common/utils/JSONUtils.java   |  10 +
 metron-platform/metron-data-management/pom.xml  |   7 +
 .../nonbulk/taxii/TaxiiIntegrationTest.java |   9 +-
 metron-platform/metron-elasticsearch/pom.xml|  17 +
 .../elasticsearch/dao/ElasticsearchDao.java | 114 ++-
 .../elasticsearch/utils/ElasticsearchUtils.java |  29 +-
 .../writer/ElasticsearchWriter.java |  12 +
 .../ElasticsearchDaoIntegrationTest.java| 147 
 .../ElasticsearchSearchIntegrationTest.java | 147 
 .../ElasticsearchUpdateIntegrationTest.java | 219 ++
 .../components/ElasticSearchComponent.java  |  36 +
 metron-platform/metron-enrichment/pom.xml   |   7 +
 .../simplehbase/SimpleHBaseAdapterTest.java |   7 +-
 .../threatintel/ThreatIntelAdapterTest.java |   7 +-
 .../integration/EnrichmentIntegrationTest.java  |  24 +-
 .../components/ConfigUploadComponent.java   |  12 +-
 .../integration/mock/MockTableProvider.java |  45 --
 .../SimpleHBaseEnrichmentFunctionsTest.java |  17 +-
 metron-platform/metron-hbase-client/pom.xml | 100 +++
 metron-platform/metron-hbase/pom.xml|  16 +
 .../org/apache/metron/hbase/TableProvider.java  |  11 +-
 .../hbase/mock/MockHBaseTableProvider.java  |  49 ++
 .../apache/metron/hbase/mock/MockHTable.java| 701 ++
 metron-platform/metron-indexing/README.md   |  24 +
 metron-platform/metron-indexing/pom.xml |  46 ++
 .../metron/indexing/dao/AccessConfig.java   |  39 +
 .../apache/metron/indexing/dao/HBaseDao.java| 135 
 .../apache/metron/indexing/dao/IndexDao.java| 113 ++-
 .../metron/indexing/dao/IndexDaoFactory.java|  38 +-
 .../indexing/dao/IndexUpdateCallback.java   |  27 +
 .../metron/indexing/dao/MultiIndexDao.java  | 159 
 .../metron/indexing/dao/search/GetRequest.java  |  47 ++
 .../indexing/dao/search/SearchRequest.java  |  20 +
 .../indexing/dao/search/SearchResponse.java |   8 +
 .../indexing/dao/search/SearchResult.java   |  25 +
 .../metron/indexing/dao/update/Document.java|  88 +++
 .../dao/update/OriginalNotFoundException.java   |  28 +
 .../indexing/dao/update/PatchRequest.java   | 101 +++
 .../indexing/dao/update/ReplaceRequest.java |  77 ++
 .../apache/metron/indexing/dao/InMemoryDao.java |  34 +-
 .../dao/IndexingDaoIntegrationTest.java | 475 
 .../indexing/dao/SearchIntegrationTest.java | 477 
 .../src/main/config/zookeeper/global.json   |  

[2/4] metron git commit: METRON-1051: Enable the ability to update indexed messages closes apache/incubator-metron#666

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/813adf2d/metron-platform/metron-hbase/src/test/java/org/apache/metron/hbase/mock/MockHTable.java
--
diff --git 
a/metron-platform/metron-hbase/src/test/java/org/apache/metron/hbase/mock/MockHTable.java
 
b/metron-platform/metron-hbase/src/test/java/org/apache/metron/hbase/mock/MockHTable.java
new file mode 100644
index 000..e75b533
--- /dev/null
+++ 
b/metron-platform/metron-hbase/src/test/java/org/apache/metron/hbase/mock/MockHTable.java
@@ -0,0 +1,701 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.hbase.mock;
+
+
+import com.google.common.collect.ImmutableList;
+import com.google.protobuf.Descriptors;
+import com.google.protobuf.Message;
+import com.google.protobuf.Service;
+import com.google.protobuf.ServiceException;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.client.RowMutations;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.metron.hbase.TableProvider;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.NavigableSet;
+import java.util.NoSuchElementException;
+import java.util.TreeMap;
+
+/**
+ * MockHTable.
+ *
+ * This implementation is a selected excerpt from 
https://gist.github.com/agaoglu/613217
+ */
+public class MockHTable implements HTableInterface {
+
+
+  private final String tableName;
+  private final List columnFamilies = new ArrayList<>();
+  private HColumnDescriptor[] descriptors;
+  private final List putLog;
+  private NavigableMap(Bytes.BYTES_COMPARATOR);
+
+  private static List toKeyValue(byte[] row, NavigableMap>> rowdata, int maxVersions) {
+return toKeyValue(row, rowdata, 0, Long.MAX_VALUE, maxVersions);
+  }
+
+  private static List toKeyValue(byte[] row, NavigableMap>> rowdata, long timestampStart, 
long timestampEnd, int maxVersions) {
+List ret = new ArrayList();
+for (byte[] family : rowdata.keySet())
+  for (byte[] qualifier : rowdata.get(family).keySet()) {
+int versionsAdded = 0;
+for (Map.Entry tsToVal : 
rowdata.get(family).get(qualifier).descendingMap().entrySet()) {
+  if (versionsAdded++ == maxVersions)
+break;
+  Long timestamp = tsToVal.getKey();
+  if (timestamp < timestampStart)
+continue;
+  if (timestamp > timestampEnd)
+continue;
+  byte[] value = tsToVal.getValue();
+  ret.add(new KeyValue(row, family, qualifier, timestamp, value));
+}
+  }
+return ret;
+  }
+  public MockHTable(String tableName) {
+this.tableName = tableName;
+this.putLog = new ArrayList<>();
+  }
+
+  public MockHTable(String tableName, String... columnFamilies) {
+this(tableName);
+

metron git commit: METRON-934: Component and task id are missing in the indexing topology Hdfs file names. closes apache/incubator-metron#574

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 4b3ccac36 -> 1277b6c32


METRON-934: Component and task id are missing in the indexing topology Hdfs 
file names. closes apache/incubator-metron#574


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1277b6c3
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1277b6c3
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1277b6c3

Branch: refs/heads/master
Commit: 1277b6c3229ee4b7cb066902ac56e03145c18157
Parents: 4b3ccac
Author: dlyle65535 
Authored: Tue May 16 10:25:09 2017 -0400
Committer: cstella 
Committed: Tue May 16 10:25:09 2017 -0400

--
 .../metron/common/writer/BulkMessageWriter.java |  3 +-
 .../writer/ElasticsearchWriter.java |  3 +-
 .../writer/SimpleHbaseEnrichmentWriter.java |  3 +-
 .../bolt/BulkMessageWriterBoltTest.java |  5 ++-
 .../apache/metron/parsers/bolt/ParserBolt.java  |  2 +-
 .../apache/metron/parsers/bolt/WriterBolt.java  |  2 +-
 .../metron/parsers/bolt/WriterHandler.java  |  5 ++-
 .../metron/parsers/bolt/ParserBoltTest.java | 15 ---
 .../metron/parsers/bolt/WriterBoltTest.java |  8 ++--
 .../apache/metron/solr/writer/SolrWriter.java   |  3 +-
 .../metron/solr/writer/SolrWriterTest.java  |  6 +--
 .../org/apache/metron/writer/NoopWriter.java|  3 +-
 .../metron/writer/WriterToBulkWriter.java   |  3 +-
 .../writer/bolt/BulkMessageWriterBolt.java  |  6 ++-
 .../apache/metron/writer/hdfs/HdfsWriter.java   |  4 +-
 .../metron/writer/hdfs/HdfsWriterTest.java  | 47 +---
 16 files changed, 74 insertions(+), 44 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/1277b6c3/metron-platform/metron-common/src/main/java/org/apache/metron/common/writer/BulkMessageWriter.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/writer/BulkMessageWriter.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/writer/BulkMessageWriter.java
index 5f427f4..64ba0cc 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/writer/BulkMessageWriter.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/writer/BulkMessageWriter.java
@@ -17,6 +17,7 @@
  */
 package org.apache.metron.common.writer;
 
+import org.apache.storm.task.TopologyContext;
 import org.apache.storm.tuple.Tuple;
 import org.apache.metron.common.configuration.Configurations;
 import org.apache.metron.common.configuration.EnrichmentConfigurations;
@@ -28,7 +29,7 @@ import java.util.Map;
 
 public interface BulkMessageWriter extends AutoCloseable, 
Serializable {
 
-  void init(Map stormConf, WriterConfiguration config) throws Exception;
+  void init(Map stormConf, TopologyContext topologyContext, 
WriterConfiguration config) throws Exception;
 
   /**
   * Writes the messages to a particular output (e.g. Elasticsearch). 
Exceptions trigger failure of the entire batch.

http://git-wip-us.apache.org/repos/asf/metron/blob/1277b6c3/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java
index c90bc8c..a643b44 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java
@@ -17,6 +17,7 @@
  */
 package org.apache.metron.elasticsearch.writer;
 
+import org.apache.storm.task.TopologyContext;
 import org.apache.storm.tuple.Tuple;
 import com.google.common.base.Splitter;
 import com.google.common.collect.ImmutableList;
@@ -57,7 +58,7 @@ public class ElasticsearchWriter implements 
BulkMessageWriter, Seria
   }
 
   @Override
-  public void init(Map stormConf, WriterConfiguration configurations) {
+  public void init(Map stormConf, TopologyContext topologyContext, 
WriterConfiguration configurations) {
 Map globalConfiguration = configurations.getGlobalConfig();
 
 Settings.Builder settingsBuilder = Settings.settingsBuilder();

http://git-wip-us.apache.org/repos/asf/metron/blob/1277b6c3/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
--
diff --git 

metron git commit: METRON-936: Fixes to pcap for performance and testing (mmiklavc via cestella) closes apache/incubator-metron#585

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master be0307659 -> c0b082523


METRON-936: Fixes to pcap for performance and testing (mmiklavc via cestella) 
closes apache/incubator-metron#585


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/c0b08252
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/c0b08252
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/c0b08252

Branch: refs/heads/master
Commit: c0b0825235ce5c26649282c43f3c57f092864f94
Parents: be03076
Author: mmiklavc <michael.miklav...@gmail.com>
Authored: Wed May 17 10:32:46 2017 -0400
Committer: cstella <ceste...@gmail.com>
Committed: Wed May 17 10:32:46 2017 -0400

--
 metron-platform/metron-pcap-backend/README.md   | 198 +++
 .../src/main/config/pcap.properties |  10 +-
 .../src/main/flux/pcap/remote.yaml  |  14 +-
 .../metron/spout/pcap/HDFSWriterCallback.java   |  76 ---
 .../metron/spout/pcap/HDFSWriterConfig.java |  47 +
 .../metron/spout/pcap/PartitionHDFSWriter.java  |  61 --
 .../pcap/deserializer/FromKeyDeserializer.java  |  22 +--
 .../deserializer/FromPacketDeserializer.java|  16 +-
 .../pcap/deserializer/KeyValueDeserializer.java |  15 +-
 .../org/apache/metron/utils/PcapInspector.java  |  29 +--
 .../PcapTopologyIntegrationTest.java|   9 +-
 .../deserializer/FromKeyDeserializerTest.java   |  39 
 .../java/org/apache/metron/pcap/PcapHelper.java |  53 -
 .../java/org/apache/metron/pcap/mr/PcapJob.java |  19 +-
 14 files changed, 505 insertions(+), 103 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/c0b08252/metron-platform/metron-pcap-backend/README.md
--
diff --git a/metron-platform/metron-pcap-backend/README.md 
b/metron-platform/metron-pcap-backend/README.md
index 5b554b5..da19611 100644
--- a/metron-platform/metron-pcap-backend/README.md
+++ b/metron-platform/metron-pcap-backend/README.md
@@ -4,6 +4,16 @@ The purpose of the Metron PCAP backend is to create a storm 
topology
 capable of ingesting rapidly raw packet capture data directly into HDFS
 from Kafka.
 
+* [Sensors](#the-sensors-feeding-kafka)
+* [PCAP Topology](#the-pcap-topology)
+* [HDFS Files](#the-files-on-hdfs)
+* [Configuration](#configuration)
+* [Starting the Topology](#starting-the-topology)
+* [Utilities](#utilities)
+  * [Inspector Utility](#inspector-utility)
+  * [Query Filter Utility](#query-filter-utility)
+* [Performance Tuning](#performance-tuning)
+
 ## The Sensors Feeding Kafka
 
 This component must be fed by fast packet capture components upstream
@@ -150,3 +160,191 @@ The packet data will be exposed via the`packet` variable 
in Stellar.
 
 The format of this regular expression is described 
[here](https://github.com/nishihatapalmer/byteseek/blob/master/sequencesyntax.md).
 
+## Performance Tuning
+The PCAP topology is extremely lightweight and functions as a Spout-only 
topology. In order to tune the topology, users currently must specify a 
combination of
+properties in pcap.properties as well as configuration in the pcap remote.yaml 
flux file itself. Tuning the number of partitions in your Kafka topic
+will have a dramatic impact on performance as well. We ran data into Kafka at 
1.1 Gbps and our tests resulted in configuring 128 partitions for our kakfa 
topic
+along with the following settings in pcap.properties and remote.yaml 
(unrelated properties for performance have been removed):
+
+### pcap.properties file
+```
+spout.kafka.topic.pcap=pcap
+storm.topology.workers=16
+kafka.spout.parallelism=128
+kafka.pcap.numPackets=10
+kafka.pcap.maxTimeMS=0
+hdfs.replication=1
+hdfs.sync.every=1
+```
+You'll notice that the number of kakfa partitions equals the spout 
parallelism, and this is no coincidence. The ordering guarantees for a 
partition in Kafka enforces that you may have no more
+consumers than 1 per topic. Any additional parallelism will leave you with 
dormant threads consuming resources but performing no additional work. For our 
cluster with 4 Storm Supervisors, we found 16 workers to
+provide optimal throughput as well. We were largely IO bound rather than CPU 
bound with the incoming PCAP data.
+
+### remote.yaml
+In the flux file, we introduced the following configuration:
+
+```
+name: "pcap"
+config:
+topology.workers: ${storm.topology.workers}
+topology.worker.childopts: ${topology.worker.childopts}
+topology.auto-credentials: ${storm.auto.credentials}
+topology.ackers.executors: 0
+components:
+
+  # Any kafka props for the producer go here.
+  - id: "kafkaProps"
+className: "java.util.HashMap"
+configMethods:
+  -   name: "put&qu

metron git commit: METRON-995: Temporary variables in stellar enrichments which are maps do not function as expected closes apache/metron#615

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master f523c1795 -> de2c871ab


METRON-995: Temporary variables in stellar enrichments which are maps do not 
function as expected closes apache/metron#615


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/de2c871a
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/de2c871a
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/de2c871a

Branch: refs/heads/master
Commit: de2c871abf3e259311281743952eb465362cf90c
Parents: f523c17
Author: cstella 
Authored: Mon Jun 19 10:37:50 2017 -0400
Committer: cstella 
Committed: Mon Jun 19 10:37:50 2017 -0400

--
 .../enrichment/handler/StellarConfig.java   | 13 ++---
 .../metron/common/stellar/StellarCompiler.java  |  2 +
 .../StellarEnrichmentConfigTest.java| 37 +
 .../configuration/StellarEnrichmentTest.java| 44 +++-
 .../metron/common/stellar/StellarTest.java  |  5 ++
 .../adapters/stellar/StellarAdapter.java| 35 ++---
 .../adapters/stellar/StellarAdapterTest.java| 55 
 .../integration/EnrichmentIntegrationTest.java  |  5 ++
 .../main/config/zookeeper/enrichments/test.json |  4 +-
 9 files changed, 185 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/de2c871a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
index 7d49fbd..a888b81 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
@@ -17,18 +17,18 @@
  */
 package org.apache.metron.common.configuration.enrichment.handler;
 
-import com.google.common.base.Splitter;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.Iterables;
+import com.google.common.base.Joiner;
 import org.apache.metron.common.stellar.StellarAssignment;
 import org.apache.metron.common.stellar.StellarProcessor;
 import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 import java.util.*;
 import java.util.function.Function;
 
 public class StellarConfig implements Config {
-
+  protected static final Logger _LOG = 
LoggerFactory.getLogger(StellarConfig.class);
   @Override
   public List getSubgroups(Iterable> config) 
{
 boolean includeEmpty = false;
@@ -81,10 +81,10 @@ public class StellarConfig implements Config {
   {
 StellarProcessor processor = new StellarProcessor();
 List messages = new ArrayList<>();
-Map defaultStellarStatementGroup = new HashMap<>();
+List defaultStellarStatementGroup = new ArrayList<>();
 for(Map.Entry kv : config) {
   if(kv.getValue() instanceof String) {
-defaultStellarStatementGroup.put(kv.getKey(), (String)kv.getValue());
+defaultStellarStatementGroup.add((String)kv.getValue());
   }
   else if(kv.getValue() instanceof Map) {
 JSONObject ret = new JSONObject();
@@ -103,6 +103,7 @@ public class StellarConfig implements Config {
   ret.put("", getMessage(getFields(processor, 
defaultStellarStatementGroup), message));
   messages.add(ret);
 }
+_LOG.debug("Stellar enrichment split: {}", messages );
 return messages;
   }
 

http://git-wip-us.apache.org/repos/asf/metron/blob/de2c871a/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
index 136728d..f4e101b 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java
@@ -17,6 +17,7 @@
  */
 package org.apache.metron.common.stellar;
 
+import org.antlr.v4.runtime.ParserRuleContext;
 import org.apache.commons.lang3.StringEscapeUtils;
 import org.apache.metron.common.dsl.Context;
 import org.apache.metron.common.dsl.Token;
@@ -347,6 +348,7 @@ public class 

metron git commit: METRON-955: Make the default sync policy for HDFS Writer be based on the batch size closes apache/incubator-metron#589

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master c1c212117 -> be0307659


METRON-955: Make the default sync policy for HDFS Writer be based on the batch 
size closes apache/incubator-metron#589


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/be030765
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/be030765
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/be030765

Branch: refs/heads/master
Commit: be03076599544f2baedb1b3010ad50625b2f32ae
Parents: c1c2121
Author: cstella 
Authored: Tue May 16 16:50:04 2017 -0400
Committer: cstella 
Committed: Tue May 16 16:50:04 2017 -0400

--
 .../apache/metron/writer/hdfs/HdfsWriter.java   | 22 -
 .../metron/writer/hdfs/SourceHandler.java   | 14 +--
 .../metron/writer/hdfs/SyncPolicyCreator.java   | 25 
 .../metron/writer/hdfs/HdfsWriterTest.java  |  4 ++--
 4 files changed, 50 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/be030765/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
--
diff --git 
a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
 
b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
index a86dfbc..e0ab502 100644
--- 
a/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
+++ 
b/metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
@@ -40,16 +40,18 @@ import org.json.simple.JSONObject;
 import java.io.IOException;
 import java.io.Serializable;
 import java.util.*;
+import java.util.function.Function;
 
 public class HdfsWriter implements BulkMessageWriter, Serializable 
{
   List rotationActions = new ArrayList<>();
   FileRotationPolicy rotationPolicy = new NoRotationPolicy();
-  SyncPolicy syncPolicy = new CountSyncPolicy(1); //sync every time, duh.
+  SyncPolicy syncPolicy;
   FileNameFormat fileNameFormat;
   Map sourceHandlerMap = new HashMap<>();
   int maxOpenFiles = 500;
   transient StellarProcessor stellarProcessor;
   transient Map stormConfig;
+  transient SyncPolicyCreator syncPolicyCreator;
 
 
   public HdfsWriter withFileNameFormat(FileNameFormat fileNameFormat){
@@ -81,6 +83,14 @@ public class HdfsWriter implements 
BulkMessageWriter, Serializable {
 this.stormConfig = stormConfig;
 this.stellarProcessor = new StellarProcessor();
 this.fileNameFormat.prepare(stormConfig,topologyContext);
+if(syncPolicy != null) {
+  //if the user has specified the sync policy, we don't want to override 
their wishes.
+  syncPolicyCreator = (source,config) -> syncPolicy;
+}
+else {
+  //if the user has not, then we want to have the sync policy depend on 
the batch size.
+  syncPolicyCreator = (source, config) -> new CountSyncPolicy(config == 
null?1:config.getBatchSize(source));
+}
   }
 
 
@@ -92,18 +102,18 @@ public class HdfsWriter implements 
BulkMessageWriter, Serializable {
) throws Exception
   {
 BulkWriterResponse response = new BulkWriterResponse();
+
 // Currently treating all the messages in a group for pass/failure.
 try {
   // Messages can all result in different HDFS paths, because of Stellar 
Expressions, so we'll need to iterate through
   for(JSONObject message : messages) {
-Map val = configurations.getSensorConfig(sourceType);
 String path = getHdfsPathExtension(
 sourceType,
 
(String)configurations.getSensorConfig(sourceType).getOrDefault(IndexingConfigurations.OUTPUT_PATH_FUNCTION_CONF,
 ""),
 message
 );
-SourceHandler handler = getSourceHandler(sourceType, path);
-handler.handle(message);
+SourceHandler handler = getSourceHandler(sourceType, path, 
configurations);
+handler.handle(message, sourceType, configurations, syncPolicyCreator);
   }
 } catch (Exception e) {
   response.addAllErrors(e, tuples);
@@ -142,7 +152,7 @@ public class HdfsWriter implements 
BulkMessageWriter, Serializable {
 sourceHandlerMap.clear();
   }
 
-  synchronized SourceHandler getSourceHandler(String sourceType, String 
stellarResult) throws IOException {
+  synchronized SourceHandler getSourceHandler(String sourceType, String 
stellarResult, WriterConfiguration config) throws IOException {
 SourceHandlerKey key = new SourceHandlerKey(sourceType, stellarResult);
 SourceHandler ret = sourceHandlerMap.get(key);
 if(ret == null) {
@@ -151,7 +161,7 @@ public class HdfsWriter 

[3/3] metron git commit: METRON-980: Short circuit operations for Stellar closes apache/metron#606

[cestella]

METRON-980: Short circuit operations for Stellar closes apache/metron#606


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/30d0e2a6
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/30d0e2a6
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/30d0e2a6

Branch: refs/heads/master
Commit: 30d0e2a6faa1c1205522758e5fd15f35ebc9fcbb
Parents: 7d6121b
Author: cstella 
Authored: Fri Jun 9 11:48:27 2017 -0400
Committer: cstella 
Committed: Fri Jun 9 11:48:27 2017 -0400

--
 .../profiler/client/window/WindowProcessor.java |   2 +-
 metron-platform/metron-common/README.md |   1 +
 .../metron/common/stellar/generated/Stellar.g4  |  16 +-
 .../org/apache/metron/common/dsl/Token.java |  14 +
 .../common/stellar/BaseStellarProcessor.java|   1 +
 .../metron/common/stellar/FrameContext.java |  45 ++
 .../metron/common/stellar/StellarCompiler.java  | 287 +--
 .../stellar/evaluators/ArithmeticEvaluator.java |  42 +-
 ...mparisonExpressionWithOperatorEvaluator.java |   9 +-
 .../evaluators/DoubleLiteralEvaluator.java  |   5 +-
 .../evaluators/FloatLiteralEvaluator.java   |   5 +-
 .../stellar/evaluators/IntLiteralEvaluator.java |   5 +-
 .../evaluators/LongLiteralEvaluator.java|   5 +-
 .../stellar/evaluators/NumberEvaluator.java |   3 +-
 .../evaluators/NumberLiteralEvaluator.java  |   8 +-
 .../stellar/generated/StellarBaseListener.java  |  36 +
 .../stellar/generated/StellarListener.java  |  30 +
 .../common/stellar/generated/StellarParser.java | 774 +++
 .../dsl/functions/FunctionalFunctionsTest.java  |  17 +
 .../common/stellar/StellarInterpreterTest.java  |  24 +-
 .../metron/common/stellar/StellarTest.java  | 101 ++-
 .../evaluators/ArithmeticEvaluatorTest.java |  58 +-
 ...isonExpressionWithOperatorEvaluatorTest.java |  10 +-
 .../evaluators/DoubleLiteralEvaluatorTest.java  |   8 +-
 .../evaluators/FloatLiteralEvaluatorTest.java   |   8 +-
 .../evaluators/IntLiteralEvaluatorTest.java |   8 +-
 .../evaluators/LongLiteralEvaluatorTest.java|   8 +-
 .../evaluators/NumberLiteralEvaluatorTest.java  |  18 +-
 28 files changed, 1029 insertions(+), 519 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/30d0e2a6/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
--
diff --git 
a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
 
b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
index 16a7513..1ae4aaf 100644
--- 
a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
+++ 
b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
@@ -89,7 +89,7 @@ public class WindowProcessor extends WindowBaseListener {
 if(checkForException(ctx)) {
   return;
 }
-stack.push(new Token<>(ctx.getText().substring(1), String.class));
+stack.push(new Token<>(ctx.getText().substring(1), String.class ));
   }
 
   /**

http://git-wip-us.apache.org/repos/asf/metron/blob/30d0e2a6/metron-platform/metron-common/README.md
--
diff --git a/metron-platform/metron-common/README.md 
b/metron-platform/metron-common/README.md
index 9e1de7a..95f59ba 100644
--- a/metron-platform/metron-common/README.md
+++ b/metron-platform/metron-common/README.md
@@ -26,6 +26,7 @@ The query language supports the following:
   * The literal `"\"foo\""` would represent `"foo"`
   * The literal `'foo \\ bar'` would represent `foo \ bar`
 * Simple boolean operations: `and`, `not`, `or`
+  * Boolean expressions are short-circuited (e.g. `true or FUNC()` would never 
execute `FUNC`)
 * Simple arithmetic operations: `*`, `/`, `+`, `-` on real numbers or integers
 * Simple comparison operations `<`, `>`, `<=`, `>=`
 * Simple equality comparison operations `==`, `!=`

http://git-wip-us.apache.org/repos/asf/metron/blob/30d0e2a6/metron-platform/metron-common/src/main/antlr4/org/apache/metron/common/stellar/generated/Stellar.g4
--
diff --git 
a/metron-platform/metron-common/src/main/antlr4/org/apache/metron/common/stellar/generated/Stellar.g4
 
b/metron-platform/metron-common/src/main/antlr4/org/apache/metron/common/stellar/generated/Stellar.g4
index 8602fde..3577377 100644
--- 
a/metron-platform/metron-common/src/main/antlr4/org/apache/metron/common/stellar/generated/Stellar.g4
+++ 

metron git commit: METRON-958: PCAP Query job throws exception when no files returned by time range query closes apache/metron#593

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 356881ad2 -> e2197316d


METRON-958: PCAP Query job throws exception when no files returned by time 
range query closes apache/metron#593


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/e2197316
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/e2197316
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/e2197316

Branch: refs/heads/master
Commit: e2197316df5754aba601134ecce19b12092b6f37
Parents: 356881a
Author: mmiklavc 
Authored: Sun May 21 21:08:45 2017 -0400
Committer: cstella 
Committed: Sun May 21 21:08:45 2017 -0400

--
 .../org/apache/metron/pcap/PcapJobTest.java | 110 +-
 .../PcapTopologyIntegrationTest.java| 123 +++
 .../apache/metron/pcap/PcapFilenameHelper.java  |  77 ++
 .../java/org/apache/metron/pcap/PcapHelper.java |  35 ++---
 .../java/org/apache/metron/pcap/mr/PcapJob.java | 105 +
 .../metron/pcap/utils/FileFilterUtil.java   | 138 +
 .../metron/pcap/PcapFilenameHelperTest.java |  75 ++
 .../metron/pcap/mr/FileFilterUtilTest.java  | 150 +++
 8 files changed, 568 insertions(+), 245 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/e2197316/metron-platform/metron-pcap-backend/src/test/java/org/apache/metron/pcap/PcapJobTest.java
--
diff --git 
a/metron-platform/metron-pcap-backend/src/test/java/org/apache/metron/pcap/PcapJobTest.java
 
b/metron-platform/metron-pcap-backend/src/test/java/org/apache/metron/pcap/PcapJobTest.java
index 81725d8..3536a7e 100644
--- 
a/metron-platform/metron-pcap-backend/src/test/java/org/apache/metron/pcap/PcapJobTest.java
+++ 
b/metron-platform/metron-pcap-backend/src/test/java/org/apache/metron/pcap/PcapJobTest.java
@@ -7,7 +7,7 @@
  * "License"); you may not use this file except in compliance
  * with the License.  You may obtain a copy of the License at
  *
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
@@ -18,10 +18,10 @@
 
 package org.apache.metron.pcap;
 
-import com.google.common.collect.Iterables;
+import static java.lang.Long.toUnsignedString;
+import static org.hamcrest.CoreMatchers.equalTo;
+
 import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.fs.FileSystem;
-import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.io.BytesWritable;
 import org.apache.hadoop.io.LongWritable;
 import org.apache.metron.common.utils.timestamp.TimestampConverters;
@@ -29,106 +29,9 @@ import org.apache.metron.pcap.mr.PcapJob;
 import org.junit.Assert;
 import org.junit.Test;
 
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
-
-import static java.lang.Long.toUnsignedString;
-import static org.hamcrest.CoreMatchers.equalTo;
-
 public class PcapJobTest {
 
   @Test
-  public void test_getPaths_NoFiles() throws Exception {
-PcapJob job;
-{
-  final List inputFiles = new ArrayList() {{
-  }};
-  job = new PcapJob() {
-@Override
-protected Iterable listFiles(FileSystem fs, Path basePath) 
throws IOException {
-  return inputFiles;
-}
-  };
-  Iterable paths = job.getPaths(null, null, 0, 1000);
-  Assert.assertTrue(Iterables.isEmpty(paths));
-}
-  }
-
-  @Test
-  public void test_getPaths_leftEdge() throws Exception {
-PcapJob job;
-{
-  final List inputFiles = new ArrayList() {{
-add(new 
Path("/apps/metron/pcap/pcap_pcap_1461589332993573000_0_73686171-64a1-46e5-9e67-66cf603fb094"));
-add(new 
Path("/apps/metron/pcap/pcap_pcap_1561589332993573000_0_73686171-64a1-46e5-9e67-66cf603fb094"));
-  }};
-  job = new PcapJob() {
-@Override
-protected Iterable listFiles(FileSystem fs, Path basePath) 
throws IOException {
-  return inputFiles;
-}
-  };
-  Iterable paths = job.getPaths(null, null, 0, 
TimestampConverters.MILLISECONDS.toNanoseconds(System.currentTimeMillis()));
-  Assert.assertEquals(1, Iterables.size(paths));
-}
-  }
-
-  @Test
-  public void test_getPaths_rightEdge() throws Exception {
-PcapJob job;
-{
-  final List inputFiles = new ArrayList() {{
-add(new 
Path("/apps/metron/pcap/pcap0_pcap_1461589332993573000_0_73686171-64a1-46e5-9e67-66cf603fb094"));
-add(new 
Path("/apps/metron/pcap/pcap1_pcap_1461589333993573000_0_73686171-64a1-46e5-9e67-66cf603fb094"));
-  }};
-  job = new PcapJob() {
-@Override
-

svn commit: r21714 - /release/metron/0.4.0/

[cestella]

Author: cestella
Date: Tue Sep 19 23:11:07 2017
New Revision: 21714

Log:
Removing Apache Metron 0.4.0 after 0.4.1 release

Removed:
release/metron/0.4.0/

metron git commit: METRON-1180: Make Stellar Shell accept zookeeper quorum as a CSV list and not require a port closes apache/incubator-metron#751

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 76947ad52 -> 6017c5974


METRON-1180: Make Stellar Shell accept zookeeper quorum as a CSV list and not 
require a port closes apache/incubator-metron#751


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/6017c597
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/6017c597
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/6017c597

Branch: refs/heads/master
Commit: 6017c5974a868f94ef86b9493ee0d244ffd2df65
Parents: 76947ad
Author: cstella 
Authored: Wed Sep 13 13:09:51 2017 -0700
Committer: cstella 
Committed: Wed Sep 13 13:09:51 2017 -0700

--
 .../shell/StellarShellOptionsValidator.java | 50 +++-
 .../shell/StellarShellOptionsValidatorTest.java | 30 +---
 2 files changed, 38 insertions(+), 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/6017c597/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
--
diff --git 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
index 97f5b70..ab92401 100644
--- 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
+++ 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
@@ -26,6 +26,8 @@ import java.net.UnknownHostException;
 import java.util.function.Predicate;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
+
+import com.google.common.base.Splitter;
 import org.apache.commons.cli.CommandLine;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.validator.routines.InetAddressValidator;
@@ -73,30 +75,32 @@ public class StellarShellOptionsValidator {
   /**
* Zookeeper argument should be in the form [HOST|IP]:PORT.
*
-   * @param z the zookeeper url fragment
+   * @param zMulti the zookeeper url fragment
*/
-  private static void validateZookeeperOption(String z) throws 
IllegalArgumentException {
-
-Matcher matcher = validPortPattern.matcher(z);
-if (!matcher.matches()) {
-  throw new IllegalArgumentException(String.format("Zookeeper option must 
have port: %s", z));
-}
-
-if (matcher.groupCount() != 2) {
-  throw new IllegalArgumentException(
-  String.format("Zookeeper Option must be in the form of 
[HOST|IP]:PORT  %s", z));
-}
-String name = matcher.group(1);
-Integer port = Integer.parseInt(matcher.group(2));
-
-if (!hostnameValidator.test(name) && !inetAddressValidator.isValid(name)) {
-  throw new IllegalArgumentException(
-  String.format("Zookeeper Option %s is not a valid host name or ip 
address  %s", name, z));
-}
-
-if(port == 0 || port > 65535){
-  throw new IllegalArgumentException(
-  String.format("Zookeeper Option %s port is not valid",z));
+  private static void validateZookeeperOption(String zMulti) throws 
IllegalArgumentException {
+for(String z : Splitter.on(",").split(zMulti)) {
+  Matcher matcher = validPortPattern.matcher(z);
+  boolean hasPort = z.contains(":");
+  if (hasPort && !matcher.matches()) {
+throw new IllegalArgumentException(String.format("Zookeeper option 
must have valid port: %s", z));
+  }
+
+  if (hasPort && matcher.groupCount() != 2) {
+throw new IllegalArgumentException(
+String.format("Zookeeper Option must be in the form of 
[HOST|IP]:PORT  %s", z));
+  }
+  String name = hasPort?matcher.group(1):z;
+  Integer port = hasPort?Integer.parseInt(matcher.group(2)):null;
+
+  if (!hostnameValidator.test(name) && 
!inetAddressValidator.isValid(name)) {
+throw new IllegalArgumentException(
+String.format("Zookeeper Option %s is not a valid host name or 
ip address  %s", name, z));
+  }
+
+  if (hasPort && (port == 0 || port > 65535)) {
+throw new IllegalArgumentException(
+String.format("Zookeeper Option %s port is not valid", z));
+  }
 }
   }
 

http://git-wip-us.apache.org/repos/asf/metron/blob/6017c597/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
--
diff --git 
a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
 

svn commit: r21632 [1/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Author: cestella
Date: Fri Sep 15 23:37:46 2017
New Revision: 21632

Log:
Adding artifacts for Metron 0.4.1

Added:
release/metron/0.4.1/
release/metron/0.4.1/CHANGES
release/metron/0.4.1/KEYS
release/metron/0.4.1/LICENSE
release/metron/0.4.1/NOTICE
release/metron/0.4.1/RELEASE_NOTES
release/metron/0.4.1/apache-metron-0.4.1.tar.gz   (with props)
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.asc
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.md5
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.sha
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.sha1
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.sha256
release/metron/0.4.1/apache-metron-0.4.1.tar.gz.sha512
release/metron/0.4.1/site-book/
release/metron/0.4.1/site-book/Upgrading.html
release/metron/0.4.1/site-book/css/
release/metron/0.4.1/site-book/css/apache-maven-fluido-1.3.0.min.css
release/metron/0.4.1/site-book/css/print.css
release/metron/0.4.1/site-book/css/site.css
release/metron/0.4.1/site-book/images/
release/metron/0.4.1/site-book/images/ApacheIncubating_Logo.png   (with 
props)
release/metron/0.4.1/site-book/images/accessories-text-editor.png   (with 
props)
release/metron/0.4.1/site-book/images/add.gif   (with props)
release/metron/0.4.1/site-book/images/ambari-storm-site-properties.png   
(with props)
release/metron/0.4.1/site-book/images/ambari-storm-site.png   (with props)
release/metron/0.4.1/site-book/images/apache-maven-project-2.png   (with 
props)
release/metron/0.4.1/site-book/images/application-certificate.png   (with 
props)
release/metron/0.4.1/site-book/images/contact-new.png   (with props)
release/metron/0.4.1/site-book/images/custom-storm-site-final.png   (with 
props)
release/metron/0.4.1/site-book/images/document-properties.png   (with props)
release/metron/0.4.1/site-book/images/drive-harddisk.png   (with props)

release/metron/0.4.1/site-book/images/enable-kerberos-configure-kerberos.png   
(with props)
release/metron/0.4.1/site-book/images/enable-kerberos-started.png   (with 
props)
release/metron/0.4.1/site-book/images/enable-kerberos.png   (with props)
release/metron/0.4.1/site-book/images/enrichment_arch.png   (with props)
release/metron/0.4.1/site-book/images/fix.gif   (with props)
release/metron/0.4.1/site-book/images/icon_error_sml.gif   (with props)
release/metron/0.4.1/site-book/images/icon_help_sml.gif   (with props)
release/metron/0.4.1/site-book/images/icon_info_sml.gif   (with props)
release/metron/0.4.1/site-book/images/icon_success_sml.gif   (with props)
release/metron/0.4.1/site-book/images/icon_warning_sml.gif   (with props)
release/metron/0.4.1/site-book/images/image-x-generic.png   (with props)
release/metron/0.4.1/site-book/images/indexing_arch.png   (with props)
release/metron/0.4.1/site-book/images/internet-web-browser.png   (with 
props)
release/metron/0.4.1/site-book/images/logos/
release/metron/0.4.1/site-book/images/logos/build-by-maven-black.png   
(with props)
release/metron/0.4.1/site-book/images/logos/build-by-maven-white.png   
(with props)
release/metron/0.4.1/site-book/images/logos/maven-feather.png   (with props)
release/metron/0.4.1/site-book/images/maas_arch.png   (with props)
release/metron/0.4.1/site-book/images/metron-logo.png   (with props)
release/metron/0.4.1/site-book/images/network-server.png   (with props)
release/metron/0.4.1/site-book/images/package-x-generic.png   (with props)
release/metron/0.4.1/site-book/images/parser_arch.png   (with props)
release/metron/0.4.1/site-book/images/profiles/
release/metron/0.4.1/site-book/images/profiles/pre-release.png   (with 
props)
release/metron/0.4.1/site-book/images/profiles/retired.png   (with props)
release/metron/0.4.1/site-book/images/profiles/sandbox.png   (with props)
release/metron/0.4.1/site-book/images/remove.gif   (with props)
release/metron/0.4.1/site-book/images/rss.png   (with props)
release/metron/0.4.1/site-book/images/update.gif   (with props)
release/metron/0.4.1/site-book/images/window-new.png   (with props)
release/metron/0.4.1/site-book/img/
release/metron/0.4.1/site-book/img/glyphicons-halflings-white.png   (with 
props)
release/metron/0.4.1/site-book/img/glyphicons-halflings.png   (with props)
release/metron/0.4.1/site-book/index.html
release/metron/0.4.1/site-book/js/
release/metron/0.4.1/site-book/js/apache-maven-fluido-1.3.0.min.js
release/metron/0.4.1/site-book/metron-analytics/
release/metron/0.4.1/site-book/metron-analytics/index.html
release/metron/0.4.1/site-book/metron-analytics/metron-maas-service/

release/metron/0.4.1/site-book/metron-analytics/metron-maas-service/index.html
release/metron/0.4.1/site-book/metron-analytics/metron-profiler/
release/metron/0.4.1/site-book/metron-analytics/metron-profiler-client

svn commit: r21632 [6/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-analytics/metron-statistics/HLLP.html
==
--- release/metron/0.4.1/site-book/metron-analytics/metron-statistics/HLLP.html 
(added)
+++ release/metron/0.4.1/site-book/metron-analytics/metron-statistics/HLLP.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,7620 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  HyperLogLogPlus
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+HyperLogLogPlus
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   

 
+  
+
+  
+  
+Analytics
+
+  
+  
+
+  
+  
+Maas-service
+
+  
+  
+
+  
+  
+Profiler
+
+  
+  
+
+  
+  
+Profiler-client
+
+   
   
+  
+
+  
+  
+Statistics
+
+  
+  
+
+HLLP
+  
+  
+
+  
+
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+ 

svn commit: r21632 [8/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-deployment/Kerberos-manual-setup.html
==
--- release/metron/0.4.1/site-book/metron-deployment/Kerberos-manual-setup.html 
(added)
+++ release/metron/0.4.1/site-book/metron-deployment/Kerberos-manual-setup.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,860 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Kerberos Setup
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Kerberos Setup
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+Kerberos-manual-setup
+  
+  
+  
+
+  
+  
+Amazon-ec2
+
+
+  
+
+  
+  
+Other-examples
+  
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+Packer-build
+
+   

svn commit: r21632 [2/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: release/metron/0.4.1/LICENSE
==
--- release/metron/0.4.1/LICENSE (added)
+++ release/metron/0.4.1/LICENSE Fri Sep 15 23:37:46 2017
@@ -0,0 +1,953 @@
+Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other modifications
+  represent, as a whole, an original work of authorship. For the purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  (except as stated in this section) patent license to make, have made,
+  use, offer to sell, sell, import, and otherwise transfer the Work,
+  where such license applies only to those patent claims licensable
+  by such Contributor that are necessarily infringed by their
+  Contribution(s) alone or by combination of their Contribution(s)
+  with the Work to which such Contribution(s) was submitted. If You
+  institute patent litigation against any entity (including a
+  cross-claim or counterclaim in a lawsuit) alleging that the Work
+  or a 

svn commit: r21632 [5/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-analytics/metron-profiler/index.html
==
--- release/metron/0.4.1/site-book/metron-analytics/metron-profiler/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-analytics/metron-profiler/index.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,1012 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Metron Profiler
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Metron Profiler
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   

 
+  
+
+  
+  
+Analytics
+
+  
+  
+
+  
+  
+Maas-service
+
+  
+  
+
+Profiler
+  
+  
+  
+
+  
+  
+Profiler-client
+
+
+  
+
+  
+  
+Statistics
+  
+  
+
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   

svn commit: r21632 [21/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-sensors/index.html
==
--- release/metron/0.4.1/site-book/metron-sensors/index.html (added)
+++ release/metron/0.4.1/site-book/metron-sensors/index.html Fri Sep 15 
23:37:46 2017
@@ -0,0 +1,238 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Metron Sensors
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Metron Sensors
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


   
+  
+
+  
+  
+Platform
+  
+   
 
+  
+
+Sensors
+  
+  
+  
+
+  
+  
+Bro-plugin-kafka
+
+  
+  
+
+  
+  
+Fastcapa
+
+ 

svn commit: r21632 [23/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html
==
--- 
release/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html 
(added)
+++ 
release/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,532 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Problem Statement
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Problem Statement
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


   
+  
+
+  
+  
+Platform
+  
+   
 
+  
+
+  
+  
+Sensors
+  
+
+  
+
+  
+

svn commit: r21632 [7/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-analytics/metron-statistics/index.html
==
--- 
release/metron/0.4.1/site-book/metron-analytics/metron-statistics/index.html 
(added)
+++ 
release/metron/0.4.1/site-book/metron-analytics/metron-statistics/index.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,964 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Statistics and Mathematical Functions
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Statistics and Mathematical Functions
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   

 
+  
+
+  
+  
+Analytics
+
+  
+  
+
+  
+  
+Maas-service
+
+  
+  
+
+  
+  
+Profiler
+
+  
+  
+
+  
+  
+Profiler-client
+
+
+  
+
+Statistics
+  
+  
+  
+
+  
+  
+HLLP
+
+  
+
+  
+
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+  

svn commit: r21632 [20/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-platform/metron-writer/index.html
==
--- release/metron/0.4.1/site-book/metron-platform/metron-writer/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-platform/metron-writer/index.html Fri 
Sep 15 23:37:46 2017
@@ -0,0 +1,349 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Writer
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Writer
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Performance-tuning-guide
+
+  
+  
+
+  
+  
+Api
+
+  
+  
+
+  
+  
+Common
+
+  

svn commit: r21632 [9/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: release/metron/0.4.1/site-book/metron-deployment/index.html
==
--- release/metron/0.4.1/site-book/metron-deployment/index.html (added)
+++ release/metron/0.4.1/site-book/metron-deployment/index.html Fri Sep 15 
23:37:46 2017
@@ -0,0 +1,536 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Overview
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Overview
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+Deployment
+  
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+  
+  
+Amazon-ec2
+
+
+  
+
+  
+  
+Other-examples
+  
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+Packer-build
+
+   
   

svn commit: r21632 [12/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-deployment/roles/index.html
==
--- release/metron/0.4.1/site-book/metron-deployment/roles/index.html (added)
+++ release/metron/0.4.1/site-book/metron-deployment/roles/index.html Fri Sep 
15 23:37:46 2017
@@ -0,0 +1,325 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Ansible Roles
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Ansible Roles
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+  
+  
+Amazon-ec2
+
+
+  
+
+  
+  
+Other-examples
+  
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+Packer-build
+
+   

svn commit: r21632 [17/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-platform/metron-data-management/index.html
==
--- 
release/metron/0.4.1/site-book/metron-platform/metron-data-management/index.html
 (added)
+++ 
release/metron/0.4.1/site-book/metron-platform/metron-data-management/index.html
 Fri Sep 15 23:37:46 2017
@@ -0,0 +1,1055 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Resource Data Management
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Resource Data Management
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Performance-tuning-guide
+
+  
+  
+
+  
+  
+Api
+
+  
+  
+
+ 

svn commit: r21632 [15/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-interface/metron-rest/index.html
==
--- release/metron/0.4.1/site-book/metron-interface/metron-rest/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-interface/metron-rest/index.html Fri 
Sep 15 23:37:46 2017
@@ -0,0 +1,2301 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Metron REST
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Metron REST
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+Rest
+  
+   


   
+  
+
+  
+  
+Platform
+  
+   
 
+  
+
+  
+  
+Sensors
+  
+
+  
+
+  
+  
+Stellar-common
+  
+ 

svn commit: r21632 [22/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-stellar/stellar-common/index.html
==
--- release/metron/0.4.1/site-book/metron-stellar/stellar-common/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-stellar/stellar-common/index.html Fri 
Sep 15 23:37:46 2017
@@ -0,0 +1,3410 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Contents
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Contents
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


   
+  
+
+  
+  
+Platform
+  
+   
 
+  
+
+  
+  
+Sensors
+  
+
+  
+
+Stellar-common
+  
+  

svn commit: r21632 [3/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-an

[cestella]

Added: release/metron/0.4.1/site-book/images/maas_arch.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/maas_arch.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/metron-logo.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/metron-logo.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/network-server.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/network-server.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/package-x-generic.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/package-x-generic.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/parser_arch.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/parser_arch.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/profiles/pre-release.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/profiles/pre-release.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/profiles/retired.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/profiles/retired.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/profiles/sandbox.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/profiles/sandbox.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/remove.gif
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/remove.gif
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/rss.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/rss.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/update.gif
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/update.gif
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/images/window-new.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/images/window-new.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/img/glyphicons-halflings-white.png
==
Binary file - no diff available.

Propchange: release/metron/0.4.1/site-book/img/glyphicons-halflings-white.png
--
svn:mime-type = application/octet-stream

Added: release/metron/0.4.1/site-book/img/glyphicons-halflings.png
==
Binary file - no diff available.

Propchange: 

svn commit: r21632 [18/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-platform/metron-management/index.html
==
--- release/metron/0.4.1/site-book/metron-platform/metron-management/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-platform/metron-management/index.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,1667 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Stellar REPL Management Utilities
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Stellar REPL Management Utilities
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Performance-tuning-guide
+
+  
+  
+
+  
+  
+Api
+
+  
+  
+
+

svn commit: r21632 [14/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-deployment/vagrant/quick-dev-platform/index.html
==
--- 
release/metron/0.4.1/site-book/metron-deployment/vagrant/quick-dev-platform/index.html
 (added)
+++ 
release/metron/0.4.1/site-book/metron-deployment/vagrant/quick-dev-platform/index.html
 Fri Sep 15 23:37:46 2017
@@ -0,0 +1,399 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Quick Development Platform
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Quick Development Platform
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+  
+  
+Amazon-ec2
+
+
+  
+
+  
+  
+Other-examples
+  
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+  

svn commit: r21632 [16/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-platform/Performance-tuning-guide.html
==
--- 
release/metron/0.4.1/site-book/metron-platform/Performance-tuning-guide.html 
(added)
+++ 
release/metron/0.4.1/site-book/metron-platform/Performance-tuning-guide.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,677 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Metron Performance Tuning Guide
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Metron Performance Tuning Guide
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+
+  
+  
+
+Performance-tuning-guide
+  
+  
+  
+
+  
+  
+Api
+
+  
+  
+
+  
+  
+Common
+  

svn commit: r21632 [19/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: release/metron/0.4.1/site-book/metron-platform/metron-parsers/index.html
==
--- release/metron/0.4.1/site-book/metron-platform/metron-parsers/index.html 
(added)
+++ release/metron/0.4.1/site-book/metron-platform/metron-parsers/index.html 
Fri Sep 15 23:37:46 2017
@@ -0,0 +1,871 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Parsers
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Parsers
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




 
+  
+
+  
+  
+Deployment
+  
+  
+  
+
+  
+  
+Alerts
+
+  
+  
+
+  
+  
+Config
+
+  
+  
+
+  
+  
+Rest
+
+   


 
+  
+
+  
+  
+Platform
+
+  
+  
+
+  
+  
+Performance-tuning-guide
+
+  
+  
+
+  
+  
+Api
+
+  
+  
+
+  
+  
+Common
+
+ 

svn commit: r21632 [10/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
==
--- 
release/metron/0.4.1/site-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
 (added)
+++ 
release/metron/0.4.1/site-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
 Fri Sep 15 23:37:46 2017
@@ -0,0 +1,1648 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+  
+  
+Amazon-ec2
+
+   
   
+  
+
+  
+  
+Other-examples
+
+  
+  
+
+Manual_Install_CentOS6
+  
+  
+
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+

svn commit: r21632 [13/23] - in /release/metron/0.4.1: ./ site-book/ site-book/css/ site-book/images/ site-book/images/logos/ site-book/images/profiles/ site-book/img/ site-book/js/ site-book/metron-a

[cestella]

Added: 
release/metron/0.4.1/site-book/metron-deployment/roles/sensor-test-mode/index.html
==
--- 
release/metron/0.4.1/site-book/metron-deployment/roles/sensor-test-mode/index.html
 (added)
+++ 
release/metron/0.4.1/site-book/metron-deployment/roles/sensor-test-mode/index.html
 Fri Sep 15 23:37:46 2017
@@ -0,0 +1,338 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Sensor Test Mode
+
+
+
+
+  
+
+
+  
+
+$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
+  
+
+
+  
+
+
+
+
+  
+
+http://metron.apache.org/; 
id="bannerLeft">
+   
 
+
+  
+  
+
+  
+
+  
+
+
+
+  
+http://www.apache.org; class="externalLink" 
title="Apache">
+Apache
+
+  /
+
+http://metron.apache.org/; class="externalLink" 
title="Metron">
+Metron
+
+  /
+
+
+Documentation
+
+  /
+Sensor Test Mode
+
+
+
+  Last Published: 
2017-09-08 |
+  Version: 0.4.1
+
+
+  
+
+
+  
+
+  
+
+
+
+User Documentation
+   











  
  
+  
+
+  
+  
+Metron
+
+  
+  
+
+  
+  
+Upgrading
+
+   
   
+  
+
+  
+  
+Analytics
+  
+  
+  
+
+  
+  
+Docker
+
+   




   
+  
+
+  
+  
+Deployment
+
+  
+  
+
+  
+  
+Kerberos-ambari-setup
+
+  
+  
+
+  
+  
+Kerberos-manual-setup
+
+  
+  
+
+  
+  
+Amazon-ec2
+
+
+  
+
+  
+  
+Other-examples
+  
+  
+  
+
+  
+  
+Ambari
+
+  
+  
+
+  
+  
+Ansible-docker
+
+  
+  
+
+  
+  
+Rpm-docker
+
+  
+  
+
+  
+  
+Packer-build
+

metron git commit: METRON-1179: Make STATS_ADD to take a list closes apache/incubator-metron#750

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 6017c5974 -> 2c56a13e5


METRON-1179: Make STATS_ADD to take a list closes apache/incubator-metron#750


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/2c56a13e
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/2c56a13e
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/2c56a13e

Branch: refs/heads/master
Commit: 2c56a13e53065954a853cc33a19b569b686cb70d
Parents: 6017c59
Author: cstella 
Authored: Wed Sep 13 16:53:54 2017 -0700
Committer: cstella 
Committed: Wed Sep 13 16:53:54 2017 -0700

--
 metron-analytics/metron-statistics/README.md|  2 +-
 .../statistics/StellarStatisticsFunctions.java  | 19 +---
 .../StellarStatisticsFunctionsTest.java | 24 
 3 files changed, 41 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/2c56a13e/metron-analytics/metron-statistics/README.md
--
diff --git a/metron-analytics/metron-statistics/README.md 
b/metron-analytics/metron-statistics/README.md
index f822845..982132a 100644
--- a/metron-analytics/metron-statistics/README.md
+++ b/metron-analytics/metron-statistics/README.md
@@ -60,7 +60,7 @@ functions can be used from everywhere where Stellar is used.
   * Description: Adds one or more input values to those that are used to 
calculate the summary statistics.
   * Input:
 * stats - The Stellar statistics object.  If null, then a new one is 
initialized.
-* value+ - One or more numbers to add
+* value+ - One or more items to add. Each item may be a number or a list 
of numbers. If an item is a list, each number in the list will be added.
   * Returns: A Stellar statistics object
 
  `STATS_BIN`

http://git-wip-us.apache.org/repos/asf/metron/blob/2c56a13e/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/StellarStatisticsFunctions.java
--
diff --git 
a/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/StellarStatisticsFunctions.java
 
b/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/StellarStatisticsFunctions.java
index d02734e..a6cf605 100644
--- 
a/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/StellarStatisticsFunctions.java
+++ 
b/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/StellarStatisticsFunctions.java
@@ -125,7 +125,7 @@ public class StellarStatisticsFunctions {
   , description = "Adds one or more input values to those that are 
used to calculate the summary statistics."
   , params = {
   "stats - The Stellar statistics object.  If null, then a 
new one is initialized."
- , "value+ - One or more numbers to add"
+ , "value+ - One or more items to add. Each item may be a 
number or a list of numbers. If an item is a list, each number in the list will 
be added."
  }
   , returns = "A Stellar statistics object"
   )
@@ -141,8 +141,21 @@ public class StellarStatisticsFunctions {
 
   // add each of the numeric values
   for(int i=1; i

metron git commit: METRON-1186: Profiler Functions use classutils from shaded storm closes apache/incubator-metron#758

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 2ae1f5adf -> f03c1c36e


METRON-1186: Profiler Functions use classutils from shaded storm closes 
apache/incubator-metron#758


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/f03c1c36
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/f03c1c36
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/f03c1c36

Branch: refs/heads/master
Commit: f03c1c36ed1bce84cf5e1f8746e1cd6799f516c4
Parents: 2ae1f5a
Author: cstella 
Authored: Thu Sep 14 14:59:11 2017 -0700
Committer: cstella 
Committed: Thu Sep 14 14:59:11 2017 -0700

--
 .../apache/metron/profiler/client/stellar/ProfilerFunctions.java  | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/f03c1c36/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
--
diff --git 
a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
 
b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
index d6f2c6a..64c1e2e 100644
--- 
a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
+++ 
b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/ProfilerFunctions.java
@@ -20,7 +20,7 @@
 
 package org.apache.metron.profiler.client.stellar;
 
-import org.apache.commons.collections4.ListUtils;
+import org.apache.commons.lang3.ClassUtils;
 import org.apache.metron.common.configuration.profiler.ProfilerConfig;
 import org.apache.metron.common.utils.JSONUtils;
 import org.apache.metron.profiler.ProfileMeasurement;
@@ -29,7 +29,6 @@ import org.apache.metron.stellar.dsl.Context;
 import org.apache.metron.stellar.dsl.ParseException;
 import org.apache.metron.stellar.dsl.Stellar;
 import org.apache.metron.stellar.dsl.StellarFunction;
-import org.apache.storm.shade.org.apache.commons.lang.ClassUtils;
 import org.json.simple.JSONArray;
 import org.json.simple.JSONObject;
 import org.json.simple.parser.JSONParser;

metron git commit: METRON-1122: Add support for the profiler in the management pack this closes apache/incubator-metron#709

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 7f9f86c00 -> 8560d2dfc


METRON-1122: Add support for the profiler in the management pack this closes 
apache/incubator-metron#709


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/8560d2df
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/8560d2df
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/8560d2df

Branch: refs/heads/master
Commit: 8560d2dfc96dd0215ca50acc3148300e7ef2ad75
Parents: 7f9f86c
Author: cstella 
Authored: Tue Aug 29 14:41:10 2017 -0400
Committer: cstella 
Committed: Tue Aug 29 14:41:10 2017 -0400

--
 .../METRON/CURRENT/role_command_order.json  |   2 +
 .../configuration/metron-indexing-env.xml   |   2 +-
 .../configuration/metron-profiler-env.xml   | 155 +++
 .../common-services/METRON/CURRENT/metainfo.xml |  43 
 .../package/scripts/params/params_linux.py  |  25 +++
 .../package/scripts/params/status_params.py |  11 ++
 .../package/scripts/profiler_commands.py| 194 ++
 .../CURRENT/package/scripts/profiler_master.py  |  94 +
 .../package/templates/profiler.properties.j2|  47 +
 .../METRON/CURRENT/service_advisor.py   |   5 +
 .../METRON/CURRENT/themes/metron_theme.json | 198 ++-
 .../roles/ambari_config/vars/single_node_vm.yml |   7 +-
 .../roles/ambari_config/vars/small_cluster.yml  |   5 +-
 13 files changed, 776 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/8560d2df/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
index 820179a..015f026 100755
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
@@ -5,11 +5,13 @@
 "_comment" : "dependencies for all cases",
 "METRON_INDEXING-INSTALL" : ["METRON_PARSERS-INSTALL"],
 "METRON_ENRICHMENT-INSTALL": ["METRON_INDEXING-INSTALL"],
+"METRON_PROFILER-INSTALL": ["METRON_ENRICHMENT-INSTALL"],
 "METRON_REST-INSTALL": ["METRON_PARSERS-INSTALL"],
 "METRON_PARSERS-START" : ["NAMENODE-START", "ZOOKEEPER_SERVER-START", 
"KAFKA_BROKER-START", "STORM_REST_API-START","METRON_ENRICHMENT_MASTER-START"],
 "METRON_ENRICHMENT_MASTER-START" : ["NAMENODE-START", 
"ZOOKEEPER_SERVER-START", "KAFKA_BROKER-START", "STORM_REST_API-START", 
"HBASE_MASTER-START", "HBASE_REGIONSERVER-START"],
 "METRON_ENRICHMENT_SERVICE_CHECK-SERVICE_CHECK" : 
["METRON_ENRICHMENT_MASTER-START"],
 "METRON_INDEXING-START" : ["NAMENODE-START", "ZOOKEEPER_SERVER-START", 
"KAFKA_BROKER-START", "STORM_REST_API-START","METRON_PARSERS-START"],
+"METRON_PROFILER-START" : ["NAMENODE-START", "ZOOKEEPER_SERVER-START", 
"KAFKA_BROKER-START", "HBASE_MASTER-START", "HBASE_REGIONSERVER-START", 
"METRON_ENRICHMENT-INSTALL"],
 "METRON_REST-START": 
["KAFKA_BROKER-START","STORM_REST_API-START","ZOOKEEPER_SERVER-START","NAMENODE-START","METRON_PARSERS-INSTALL","METRON_INDEXING-INSTALL","METRON_ENRICHMENT-INSTALL"],
 "METRON_MANAGEMENT_UI-START": ["METRON_REST-START"],
 "STORM_REST_API-STOP" : 
["METRON_ENRICHMENT_MASTER-STOP","METRON_PARSERS-STOP","METRON_INDEXING-STOP","METRON_REST-STOP","METRON_MANAGEMENT_UI-STOP"],

http://git-wip-us.apache.org/repos/asf/metron/blob/8560d2df/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-indexing-env.xml
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-indexing-env.xml
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-indexing-env.xml
index e36730a..6abbe77 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-indexing-env.xml
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-indexing-env.xml
@@ -59,7 +59,7 @@
 Indexing Writer Class Name
 

metron git commit: METRON-632: Added validation of "shew.enrichmentType" and "shew.keyColumns" closes apache/incubator-metron#732

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master ac0e05f01 -> 353bc8b3b


METRON-632: Added validation of "shew.enrichmentType" and "shew.keyColumns" 
closes apache/incubator-metron#732


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/353bc8b3
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/353bc8b3
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/353bc8b3

Branch: refs/heads/master
Commit: 353bc8b3be70c5b3da10ae98445521c31ae16568
Parents: ac0e05f
Author: zezutom 
Authored: Mon Oct 2 09:50:20 2017 -0400
Committer: cstella 
Committed: Mon Oct 2 09:50:20 2017 -0400

--
 .../writer/SimpleHbaseEnrichmentWriter.java |  44 ++-
 .../SimpleHBaseEnrichmentWriterTest.java| 118 +++
 2 files changed, 161 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/353bc8b3/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
--
diff --git 
a/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
 
b/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
index ad3b6d7..d0e9f67 100644
--- 
a/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
+++ 
b/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java
@@ -124,6 +124,8 @@ public class SimpleHbaseEnrichmentWriter extends 
AbstractWriter implements BulkM
 
   @Override
   public void configure(String sensorName, WriterConfiguration configuration) {
+validateEnrichmentType(sensorName, configuration);
+validateKeyColumns(sensorName, configuration);
 String hbaseProviderImpl = 
Configurations.HBASE_PROVIDER.getAndConvert(configuration.getSensorConfig(sensorName),String.class);
 if(hbaseProviderImpl != null) {
   provider = ReflectionUtils.createInstance(hbaseProviderImpl);
@@ -134,6 +136,39 @@ public class SimpleHbaseEnrichmentWriter extends 
AbstractWriter implements BulkM
 LOG.debug("Sensor: '{}': {Provider: '{}', Converter: '{}'}", sensorName, 
getClassName(provider), getClassName(converter));
   }
 
+
+  private void validateEnrichmentType(String sensorName, WriterConfiguration 
configuration) {
+Map sensorConfig = 
configuration.getSensorConfig(sensorName);
+Object enrichmentTypeObj = 
Configurations.ENRICHMENT_TYPE.get(sensorConfig);
+if (enrichmentTypeObj == null) {
+  throw new IllegalArgumentException(String.format("%s must be provided", 
Configurations.ENRICHMENT_TYPE.getKey()));
+}
+
+if (!(enrichmentTypeObj instanceof String)) {
+  throw new IllegalArgumentException(String.format("%s must be a string", 
Configurations.ENRICHMENT_TYPE.getKey()));
+}
+
+String enrichmentType = enrichmentTypeObj.toString();
+if (enrichmentType.trim().isEmpty()) {
+  throw new IllegalArgumentException(String.format("%s must not be an 
empty string",
+  Configurations.ENRICHMENT_TYPE.getKey()));
+}
+  }
+
+  private void validateKeyColumns(String sensorName, WriterConfiguration 
configuration) {
+Map sensorConfig = 
configuration.getSensorConfig(sensorName);
+Object keyColumnsObj = Configurations.KEY_COLUMNS.get(sensorConfig);
+
+try {
+  List keyColumns = getColumns(keyColumnsObj, true);
+  if (keyColumns == null || keyColumns.isEmpty()) {
+throw new IllegalArgumentException(String.format("%s must be 
provided", Configurations.KEY_COLUMNS.getKey()));
+  }
+} catch (RuntimeException ex) {
+  throw new IllegalArgumentException(ex.getMessage(), ex);
+}
+  }
+
   private String getClassName(Object object) {
 return object == null ? "" : object.getClass().getName();
   }
@@ -196,7 +231,14 @@ public class SimpleHbaseEnrichmentWriter extends 
AbstractWriter implements BulkM
 else if (o instanceof List) {
   List keyCols = new ArrayList<>();
   for(Object key : (List)o) {
-keyCols.add(key.toString());
+if (key == null) {
+  throw new IllegalArgumentException("Column name must not be null");
+}
+String columnName = key.toString();
+if (columnName.trim().isEmpty()) {
+  throw new IllegalArgumentException("Column name must not be empty");
+}
+keyCols.add(columnName);
   }
   LOG.debug("Key columns: '{}'", String.join(",", keyCols));
   return keyCols;

metron git commit: METRON-1209: Make stellar repl take logging properties, like other CLI apps in metron closes apache/incubator-metron#772

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 0cc0e2ee0 -> 60299


METRON-1209: Make stellar repl take logging properties, like other CLI apps in 
metron closes apache/incubator-metron#772


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/6029
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/6029
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/6029

Branch: refs/heads/master
Commit: 60299e75664dc36bbd014668a019cd1d00f2
Parents: 0cc0e2e
Author: cstella 
Authored: Tue Oct 3 10:06:12 2017 -0400
Committer: cstella 
Committed: Tue Oct 3 10:06:12 2017 -0400

--
 metron-stellar/pom.xml  |   1 +
 .../stellar-3rd-party-example/README.md |   6 +++
 .../stellar-3rd-party-example/pom.xml   |  39 
 .../metron/stellar/external/TimeFunction.java   |  44 +++
 .../stellar/common/shell/StellarShell.java  |  19 +---
 .../resolver/ClasspathFunctionResolver.java |  10 -
 .../classpath-resources/custom-1.0-SNAPSHOT.jar | Bin 3570 -> 3968 bytes
 .../resolver/ClasspathFunctionResolverTest.java |  15 ---
 8 files changed, 110 insertions(+), 24 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/6029/metron-stellar/pom.xml
--
diff --git a/metron-stellar/pom.xml b/metron-stellar/pom.xml
index fd62786..075ceb5 100644
--- a/metron-stellar/pom.xml
+++ b/metron-stellar/pom.xml
@@ -41,6 +41,7 @@
 
 
 
+stellar-3rd-party-example
 stellar-common
 
 

http://git-wip-us.apache.org/repos/asf/metron/blob/6029/metron-stellar/stellar-3rd-party-example/README.md
--
diff --git a/metron-stellar/stellar-3rd-party-example/README.md 
b/metron-stellar/stellar-3rd-party-example/README.md
new file mode 100644
index 000..14c489e
--- /dev/null
+++ b/metron-stellar/stellar-3rd-party-example/README.md
@@ -0,0 +1,6 @@
+# Introduction
+
+This is just a sample 3rd party stellar function (which is used in our
+unit testing).  If Stellar changes its API (e.g. the annotation changes
+classes), then this will need to be built and the output jar of this (in
+`target`) will need to be placed in 
`../stellar-common/src/test/classpath-resources/`.

http://git-wip-us.apache.org/repos/asf/metron/blob/6029/metron-stellar/stellar-3rd-party-example/pom.xml
--
diff --git a/metron-stellar/stellar-3rd-party-example/pom.xml 
b/metron-stellar/stellar-3rd-party-example/pom.xml
new file mode 100644
index 000..119892c
--- /dev/null
+++ b/metron-stellar/stellar-3rd-party-example/pom.xml
@@ -0,0 +1,39 @@
+
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+  metron-stellar
+  org.apache.metron
+  0.4.1
+
+stellar-3rd-party-example
+jar
+
+3rd party Functions (just for tests)
+
+  UTF-8
+
+
+
+  
+org.apache.metron
+stellar-common
+${project.parent.version}
+provided
+  
+
+

http://git-wip-us.apache.org/repos/asf/metron/blob/6029/metron-stellar/stellar-3rd-party-example/src/main/java/org/apache/metron/stellar/external/TimeFunction.java
--
diff --git 
a/metron-stellar/stellar-3rd-party-example/src/main/java/org/apache/metron/stellar/external/TimeFunction.java
 
b/metron-stellar/stellar-3rd-party-example/src/main/java/org/apache/metron/stellar/external/TimeFunction.java
new file mode 100644
index 000..887531e
--- /dev/null
+++ 
b/metron-stellar/stellar-3rd-party-example/src/main/java/org/apache/metron/stellar/external/TimeFunction.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language 

metron git commit: METRON-1131: The Stellar REPL rejects valid hostnames for zookeeper in its CLI options this closes apache/incubator-metron#715

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master f34affdb2 -> 174964832


METRON-1131: The Stellar REPL rejects valid hostnames for zookeeper in its CLI 
options this closes apache/incubator-metron#715


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/17496483
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/17496483
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/17496483

Branch: refs/heads/master
Commit: 174964832eafba927ae4e7aa407d2a1100a8b776
Parents: f34affd
Author: cstella 
Authored: Fri Aug 25 09:17:48 2017 -0400
Committer: cstella 
Committed: Fri Aug 25 09:17:48 2017 -0400

--
 .../shell/StellarShellOptionsValidator.java | 24 ++--
 .../shell/StellarShellOptionsValidatorTest.java |  9 
 2 files changed, 22 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/17496483/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
--
diff --git 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
index 411c288..97f5b70 100644
--- 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
+++ 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidator.java
@@ -21,21 +21,31 @@
 package org.apache.metron.stellar.common.shell;
 
 import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.function.Predicate;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 import org.apache.commons.cli.CommandLine;
+import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.validator.routines.InetAddressValidator;
 
 public class StellarShellOptionsValidator {
 
   private static final Pattern validPortPattern = 
Pattern.compile("(^.*)[:](\\d+)$");
-  private static final Pattern validHostNamePattern;
+  private static final Predicate hostnameValidator = hostname -> {
+if(StringUtils.isEmpty(hostname)) {
+  return false;
+}
+try {
+  InetAddress add = InetAddress.getByName(hostname);
+  return true;
+} catch (UnknownHostException e) {
+  return false;
+}
+  };
+
 
-  static {
-validHostNamePattern = Pattern.compile(
-"^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]).)"
-+ "*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9-]*[A-Za-z0-9])$");
-  }
 
   private static final InetAddressValidator inetAddressValidator = 
InetAddressValidator
   .getInstance();
@@ -79,7 +89,7 @@ public class StellarShellOptionsValidator {
 String name = matcher.group(1);
 Integer port = Integer.parseInt(matcher.group(2));
 
-if (!validHostNamePattern.matcher(name).matches() && 
!inetAddressValidator.isValid(name)) {
+if (!hostnameValidator.test(name) && !inetAddressValidator.isValid(name)) {
   throw new IllegalArgumentException(
   String.format("Zookeeper Option %s is not a valid host name or ip 
address  %s", name, z));
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/17496483/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
--
diff --git 
a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
 
b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
index 4f74308..52fef96 100644
--- 
a/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
+++ 
b/metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/common/shell/StellarShellOptionsValidatorTest.java
@@ -28,6 +28,7 @@ import org.junit.Assert;
 import org.junit.Test;
 
 import java.io.File;
+import java.util.regex.Pattern;
 
 import static org.junit.Assert.*;
 
@@ -35,15 +36,15 @@ public class StellarShellOptionsValidatorTest {
 
   @Test
   public void validateOptions() throws Exception {
-String[] validZHostArg = new String[]{"-z", "host1:"};
+String[] validZHostArg = new String[]{"-z", "localhost:"};
 String[] validZIPArg = new String[]{"-z", "10.10.10.3:"};
-String[] invalidZNoPortArg = new String[]{"-z", "host1"};
+String[] invalidZNoPortArg = new String[]{"-z", "youtube.com"};
  

metron git commit: METRON-1128: MAAS_GET_ENDPOINT - Unable to resolve function error seen for squid example this closes apache/incubator-metron#714

[cestella]

rySet()) {
+ret.put("endpoint:" + kv.getKey(), kv.getValue());
+  }
+  return ret;
+}
+
+@Override
+public synchronized void initialize(Context context) {
+  try {
+Optional clientOptional = 
context.getCapability(Context.Capabilities.ZOOKEEPER_CLIENT);
+CuratorFramework client = null;
+if (clientOptional.isPresent() && clientOptional.get() instanceof 
CuratorFramework) {
+  client = (CuratorFramework) clientOptional.get();
+} else {
+  throw new IllegalStateException("Unable to initialize function: 
Cannot find zookeeper client.");
+}
+try {
+  discoverer = createDiscoverer(client);
+  context.addCapability(Context.Capabilities.SERVICE_DISCOVERER, () -> 
discoverer);
+  isValidState = true;
+} catch (Exception e) {
+  LOG.error(e.getMessage(), e);
+  throw new IllegalStateException("Unable to initialize 
MAAS_GET_ENDPOINT", e);
+}
+  }
+  finally {
+isInitialized = true;
+  }
+}
+
+@Override
+public boolean isInitialized() {
+  return isInitialized;
+}
+  }
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/7f9f86c0/metron-analytics/metron-maas-service/README.md
--
diff --git a/metron-analytics/metron-maas-service/README.md 
b/metron-analytics/metron-maas-service/README.md
index 5ed108b..d234130 100644
--- a/metron-analytics/metron-maas-service/README.md
+++ b/metron-analytics/metron-maas-service/README.md
@@ -126,16 +126,29 @@ Now let's install some prerequisites:
 Start Squid via `service squid start`
 
 Now that we have flask and jinja, we can create a mock DGA service to deploy 
with MaaS:
-* Download the files in 
[this](https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a) gist 
into the `/root/mock_dga` directory
-* Make `rest.sh` executable via `chmod +x /root/mock_dga/rest.sh`
+* Download the files in 
[this](https://gist.github.com/cestella/cba10aff0f970078a4c2c8cade3a4d1a) gist 
into the `$HOME/mock_dga` directory
+* Make `rest.sh` executable via `chmod +x $HOME/mock_dga/rest.sh`
 
 This service will treat `yahoo.com` and `amazon.com` as legit and everything 
else as malicious.  The contract is that the REST service exposes an endpoint 
`/apply` and returns back JSON maps with a single key `is_malicious` which can 
be `malicious` or `legit`.
 
 ## Deploy Mock DGA Service via MaaS
 
+The following presumes that you are a logged in as a user who has a
+home directory in HDFS under `/user/$USER`.  If you do not, please create one
+and ensure the permissions are set appropriate:
+```
+su - hdfs -c "hadoop fs -mkdir /user/$USER"
+su - hdfs -c "hadoop fs -chown $USER:$USER /user/$USER"
+```
+Or, in the common case for the `metron` user:
+```
+su - hdfs -c "hadoop fs -mkdir /user/metron"
+su - hdfs -c "hadoop fs -chown metron:metron /user/metron"
+```
+
 Now let's start MaaS and deploy the Mock DGA Service:
 * Start MaaS via `$METRON_HOME/bin/maas_service.sh -zq node1:2181`
-* Start one instance of the mock DGA model with 512M of memory via 
`$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -lmp /root/mock_dga -hmp 
/user/root/models -mo ADD -m 512 -n dga -v 1.0 -ni 1`
+* Start one instance of the mock DGA model with 512M of memory via 
`$METRON_HOME/bin/maas_deploy.sh -zq node1:2181 -lmp $HOME/mock_dga -hmp 
/user/$USER/models -mo ADD -m 512 -n dga -v 1.0 -ni 1`
 * As a sanity check:
   * Ensure that the model is running via `$METRON_HOME/bin/maas_deploy.sh -zq 
node1:2181 -mo LIST`.  You should see `Model dga @ 1.0` be displayed and under 
that a url such as (but not exactly) `http://node1:36161`
   * Try to hit the model via curl: `curl 
'http://localhost:36161/apply?host=caseystella.com'` and ensure that it returns 
a JSON map indicating the domain is malicious.
@@ -170,8 +183,6 @@ Now that we have a deployed model, let's adjust the 
configurations for the Squid
 * Edit the squid enrichment configuration at 
`$METRON_HOME/config/zookeeper/enrichments/squid.json` (this file will not 
exist, so create a new one) to make the threat triage adjust the level of risk 
based on the model output:
 ```
 {
-  "index": "squid",
-  "batchSize": 1,
   "enrichment" : {
 "fieldMap": {}
   },

http://git-wip-us.apache.org/repos/asf/metron/blob/7f9f86c0/metron-analytics/metron-maas-service/pom.xml
--
diff --git a/metron-analytics/metron-maas-service/pom.xml 
b/metron-analytics/metron-maas-service/pom.xml
index 0ac9bac..4eeceae 100644
--- a/metron-analytics/metron-maas-service/pom.xml
+++ b/metron-analytics/metron-maas-service/pom.xml
@@ -49,17 +49,6 @@
   ${global_kryo_serializers_version}
 
 
-  org.apache.metron
-  stellar-c

metron git commit: METRON-1134: Allow parser command line options to be specified in the zookeeper parser config. closes apache/incubator-metron#717

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 8ef18d31a -> 38eebd59c


METRON-1134: Allow parser command line options to be specified in the zookeeper 
parser config. closes apache/incubator-metron#717


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/38eebd59
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/38eebd59
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/38eebd59

Branch: refs/heads/master
Commit: 38eebd59ce2884ead7f3797ff98f7308409e05dd
Parents: 8ef18d3
Author: cstella 
Authored: Thu Aug 31 15:46:29 2017 -0400
Committer: cstella 
Committed: Thu Aug 31 15:46:29 2017 -0400

--
 ...orParserConfigControllerIntegrationTest.java |  18 +-
 .../configuration/SensorParserConfig.java   | 208 -
 .../management/ConfigurationFunctionsTest.java  |  10 +-
 metron-platform/metron-parsers/README.md|  11 +
 .../parsers/topology/ParserTopologyBuilder.java |  87 ++--
 .../parsers/topology/ParserTopologyCLI.java | 218 +++---
 .../parsers/topology/config/ValueSupplier.java  |  30 ++
 .../components/ParserTopologyComponent.java |  30 +-
 .../parsers/topology/ParserTopologyCLITest.java | 425 ++-
 9 files changed, 934 insertions(+), 103 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/38eebd59/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/SensorParserConfigControllerIntegrationTest.java
--
diff --git 
a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/SensorParserConfigControllerIntegrationTest.java
 
b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/SensorParserConfigControllerIntegrationTest.java
index f5ea23d..66771eb 100644
--- 
a/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/SensorParserConfigControllerIntegrationTest.java
+++ 
b/metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/SensorParserConfigControllerIntegrationTest.java
@@ -19,6 +19,7 @@ package org.apache.metron.rest.controller;
 
 import org.adrianwalker.multilinestring.Multiline;
 import org.apache.commons.io.FileUtils;
+import org.apache.metron.common.configuration.SensorParserConfig;
 import org.apache.metron.rest.MetronRestConstants;
 import org.apache.metron.rest.service.GrokService;
 import org.apache.metron.rest.service.SensorParserConfigService;
@@ -37,6 +38,7 @@ import org.springframework.web.context.WebApplicationContext;
 
 import java.io.File;
 import java.io.IOException;
+import java.lang.reflect.Method;
 
 import static org.apache.metron.rest.MetronRestConstants.TEST_PROFILE;
 import static org.hamcrest.Matchers.hasSize;
@@ -196,11 +198,17 @@ public class SensorParserConfigControllerIntegrationTest {
 cleanFileSystem();
 this.sensorParserConfigService.delete("broTest");
 this.sensorParserConfigService.delete("squidTest");
-
+Method[] method = SensorParserConfig.class.getMethods();
+int numFields = 0;
+for(Method m : method) {
+  if(m.getName().startsWith("set")) {
+numFields++;
+  }
+}
 this.mockMvc.perform(post(sensorParserConfigUrl).with(httpBasic(user, 
password)).with(csrf()).contentType(MediaType.parseMediaType("application/json;charset=UTF-8")).content(squidJson))
 .andExpect(status().isCreated())
 
.andExpect(content().contentType(MediaType.parseMediaType("application/json;charset=UTF-8")))
-.andExpect(jsonPath("$.*", hasSize(10)))
+.andExpect(jsonPath("$.*", hasSize(numFields)))
 
.andExpect(jsonPath("$.parserClassName").value("org.apache.metron.parsers.GrokParser"))
 .andExpect(jsonPath("$.sensorTopic").value("squidTest"))
 
.andExpect(jsonPath("$.parserConfig.grokPath").value("target/patterns/squidTest"))
@@ -215,7 +223,7 @@ public class SensorParserConfigControllerIntegrationTest {
 this.mockMvc.perform(get(sensorParserConfigUrl + 
"/squidTest").with(httpBasic(user,password)))
 .andExpect(status().isOk())
 
.andExpect(content().contentType(MediaType.parseMediaType("application/json;charset=UTF-8")))
-.andExpect(jsonPath("$.*", hasSize(10)))
+.andExpect(jsonPath("$.*", hasSize(numFields)))
 
.andExpect(jsonPath("$.parserClassName").value("org.apache.metron.parsers.GrokParser"))
 .andExpect(jsonPath("$.sensorTopic").value("squidTest"))
 
.andExpect(jsonPath("$.parserConfig.grokPath").value("target/patterns/squidTest"))
@@ -244,7 +252,7 @@ public class SensorParserConfigControllerIntegrationTest {
 

metron git commit: METRON-1145: Profiler mpack does not create kafka acl and hbase acls correctly in kerberized cluster closes apache/incubator-metron#726

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 38eebd59c -> 31d99f122


METRON-1145: Profiler mpack does not create kafka acl and hbase acls correctly 
in kerberized cluster closes apache/incubator-metron#726


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/31d99f12
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/31d99f12
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/31d99f12

Branch: refs/heads/master
Commit: 31d99f122e89d81590df84f66c2dc9fb336a03b8
Parents: 38eebd5
Author: cstella 
Authored: Thu Aug 31 15:48:01 2017 -0400
Committer: cstella 
Committed: Thu Aug 31 15:48:01 2017 -0400

--
 .../CURRENT/package/scripts/profiler_commands.py|  4 
 .../CURRENT/package/scripts/profiler_master.py  | 16 
 2 files changed, 20 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/31d99f12/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_commands.py
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_commands.py
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_commands.py
index ddd66cb..4dfa371 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_commands.py
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_commands.py
@@ -98,6 +98,10 @@ class ProfilerCommands:
 Logger.info("Done creating HBase Tables")
 self.set_hbase_configured()
 
+def init_kafka_acls(self):
+Logger.info('Creating Kafka ACls for profiler')
+metron_service.init_kafka_acls(self.__params, [self.__profiler_topic], 
['profiler'])
+
 def set_hbase_acls(self):
 Logger.info("Setting HBase ACLs")
 if self.__params.security_enabled:

http://git-wip-us.apache.org/repos/asf/metron/blob/31d99f12/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
index 4946ab0..77c32f0 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
@@ -27,6 +27,7 @@ from resource_management.libraries.script import Script
 
 from metron_security import storm_security_setup
 import metron_service
+import metron_security
 from profiler_commands import ProfilerCommands
 
 
@@ -59,6 +60,9 @@ class Profiler(Script):
 commands.create_hbase_tables()
 if params.security_enabled and not commands.is_hbase_acl_configured():
 commands.set_hbase_acls()
+if params.security_enabled and not commands.is_acl_configured():
+commands.init_kafka_acls()
+commands.set_acl_configured()
 
 Logger.info("Calling security setup")
 storm_security_setup(params)
@@ -68,6 +72,18 @@ class Profiler(Script):
 env.set_params(params)
 self.configure(env)
 commands = ProfilerCommands(params)
+if params.security_enabled:
+metron_security.kinit(params.kinit_path_local,
+  params.metron_keytab_path,
+  params.metron_principal_name,
+  execute_user=params.metron_user)
+
+if params.security_enabled and not commands.is_hbase_acl_configured():
+commands.set_hbase_acls()
+if params.security_enabled and not commands.is_acl_configured():
+commands.init_kafka_acls()
+commands.set_acl_configured()
+
 commands.start_profiler_topology(env)
 
 def stop(self, env, upgrade_type=None):

[2/2] metron git commit: METRON-1277 Add match statement to Stellar language closes apache/incubator-metron#814

[cestella]

METRON-1277 Add match statement to Stellar language closes 
apache/incubator-metron#814


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/c749b517
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/c749b517
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/c749b517

Branch: refs/heads/master
Commit: c749b51730f7fd24a900da31c3fe2072a54086bc
Parents: f084404
Author: Otto Fowler 
Authored: Thu Nov 30 15:16:18 2017 -0500
Committer: cstella 
Committed: Thu Nov 30 15:16:18 2017 -0500

--
 Upgrading.md|   13 +
 metron-stellar/stellar-common/README.md |   29 +-
 .../metron/stellar/common/generated/Stellar.g4  |   27 +
 .../stellar-common/src/main/java/Stellar.tokens |   66 +-
 .../src/main/java/StellarLexer.tokens   |   66 +-
 .../metron/stellar/common/StellarCompiler.java  |  122 ++-
 .../common/generated/StellarBaseListener.java   |   84 ++
 .../stellar/common/generated/StellarLexer.java  |  379 +++
 .../common/generated/StellarListener.java   |  180 +++-
 .../stellar/common/generated/StellarParser.java | 1007 --
 .../metron/stellar/dsl/functions/MatchTest.java |  377 +++
 11 files changed, 1736 insertions(+), 614 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/c749b517/Upgrading.md
--
diff --git a/Upgrading.md b/Upgrading.md
index dd68de0..c3892ce 100644
--- a/Upgrading.md
+++ b/Upgrading.md
@@ -4,6 +4,19 @@ configuration which are non-backwards compatible.
 
 ## 0.4.1 to 0.4.2
 
+### [METRON-1277: STELLAR Add Match functionality to 
language](https://issues.apache.org/jira/browse/METRON-1277)
+As we continue to evolve the Stellar language, it is possible that new keywords
+will be added to the language.  This may cause compatablity issues where these
+reserved words and symbols are used in existing scripts.
+
+Adding `match` to the Stellar lanaguage has introduced the following new
+reserved keywords and symbols:
+
+`match`, `default`, `{`, `}`, '=>'
+
+Any stellar expressions which use these keywords not in quotes will need to be
+modified.
+
 ### [METRON-1158: Build backend for grouping alerts into meta 
alerts](https://issues.apache.org/jira/browse/METRON-1158)
 In order to allow for meta alerts to be queries alongside regular alerts in 
Elasticsearch 2.x,
 it is necessary to add an additional field to the templates and mapping for 
existing sensors.

http://git-wip-us.apache.org/repos/asf/metron/blob/c749b517/metron-stellar/stellar-common/README.md
--
diff --git a/metron-stellar/stellar-common/README.md 
b/metron-stellar/stellar-common/README.md
index 5fe9ae7..e6bfc7c 100644
--- a/metron-stellar/stellar-common/README.md
+++ b/metron-stellar/stellar-common/README.md
@@ -30,6 +30,7 @@ The Stellar language supports the following:
 * Simple comparison operations `<`, `>`, `<=`, `>=`
 * Simple equality comparison operations `==`, `!=`
 * if/then/else comparisons (i.e. `if var1 < 10 then 'less than 10' else '10 or 
more'`)
+* Simple match evaluations (i.e. `match{ var1 < 10 => 'warn', var1 >= 10 => 
'critical', default => 'info'}`
 * Determining whether a field exists (via `exists`)
 * An `in` operator that works like the `in` in Python
 * The ability to have parenthesis to make order of operations explicit
@@ -41,10 +42,11 @@ The following keywords need to be single quote escaped in 
order to be used in St
 |   |   | | | |
 | :---: | :---: | :-: | :-: | :-: |
 | not   | else  | exists  | if  | then|
-| and   | or| in  | NaN | ==  |
-| !=| \<=   | \>  | \>= | \+  |
-| \-| \<| ?   | \*  | /   |
-| , |   | | | |
+| and   | or| in  | NaN | match   |
+| default   | ==| !=  | \<= | \>  | 
+| \>=   | \+| \-  | \<  | ?   | 
+| \*| / | ,   | \{  | \}  |
+| \=>   |   | | | |
 
 Using parens such as: "foo" : "\" requires escaping; "foo": "\'\\'"
 
@@ -100,6 +102,25 @@ In the core language functions, we support basic 
functional programming primitiv
 * `FILTER` - Filters a list by a predicate in the form of a lambda expression. 
 For instance `FILTER([ 'foo', 

[1/2] metron git commit: METRON-1277 Add match statement to Stellar language closes apache/incubator-metron#814

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master f08440481 -> c749b5173


http://git-wip-us.apache.org/repos/asf/metron/blob/c749b517/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java
--
diff --git 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java
 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java
index 3ee2603..3bd4ad1 100644
--- 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java
+++ 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarParser.java
@@ -40,10 +40,10 @@ public class StellarParser extends Parser {
IN=1, LAMBDA_OP=2, DOUBLE_QUOTE=3, SINGLE_QUOTE=4, COMMA=5, 
PERIOD=6, 
AND=7, OR=8, NOT=9, TRUE=10, FALSE=11, EQ=12, NEQ=13, LT=14, 
LTE=15, GT=16, 
GTE=17, QUESTION=18, COLON=19, IF=20, THEN=21, ELSE=22, 
NULL=23, NAN=24, 
-   MINUS=25, PLUS=26, DIV=27, MUL=28, LBRACE=29, RBRACE=30, 
LBRACKET=31, 
-   RBRACKET=32, LPAREN=33, RPAREN=34, NIN=35, EXISTS=36, 
EXPONENT=37, INT_LITERAL=38, 
-   DOUBLE_LITERAL=39, FLOAT_LITERAL=40, LONG_LITERAL=41, 
IDENTIFIER=42, STRING_LITERAL=43, 
-   COMMENT=44, WS=45;
+   MATCH=25, DEFAULT=26, MATCH_ACTION=27, MINUS=28, PLUS=29, 
DIV=30, MUL=31, 
+   LBRACE=32, RBRACE=33, LBRACKET=34, RBRACKET=35, LPAREN=36, 
RPAREN=37, 
+   NIN=38, EXISTS=39, EXPONENT=40, INT_LITERAL=41, 
DOUBLE_LITERAL=42, FLOAT_LITERAL=43, 
+   LONG_LITERAL=44, IDENTIFIER=45, STRING_LITERAL=46, COMMENT=47, 
WS=48;
public static final int
RULE_transformation = 0, RULE_transformation_expr = 1, 
RULE_if_expr = 2, 
RULE_then_expr = 3, RULE_else_expr = 4, RULE_conditional_expr = 
5, RULE_logical_expr = 6, 
@@ -51,30 +51,35 @@ public class StellarParser extends Parser {
RULE_comp_operator = 11, RULE_func_args = 12, RULE_op_list = 
13, RULE_list_entity = 14, 
RULE_kv_list = 15, RULE_map_entity = 16, RULE_arithmetic_expr = 
17, RULE_arithmetic_expr_mul = 18, 
RULE_functions = 19, RULE_arithmetic_operands = 20, 
RULE_identifier_operand = 21, 
-   RULE_lambda_without_args = 22, RULE_lambda_with_args = 23, 
RULE_lambda_variables = 24, 
-   RULE_single_lambda_variable = 25, RULE_lambda_variable = 26;
+   RULE_default_operand = 22, RULE_lambda_without_args = 23, 
RULE_lambda_with_args = 24, 
+   RULE_lambda_variables = 25, RULE_single_lambda_variable = 26, 
RULE_lambda_variable = 27, 
+   RULE_match_expr = 28, RULE_match_clauses = 29, 
RULE_match_clause = 30, 
+   RULE_match_clause_action = 31, RULE_match_clause_check = 32;
public static final String[] ruleNames = {
"transformation", "transformation_expr", "if_expr", 
"then_expr", "else_expr", 
"conditional_expr", "logical_expr", "b_expr", "in_expr", 
"comparison_expr", 
"transformation_entity", "comp_operator", "func_args", 
"op_list", "list_entity", 
"kv_list", "map_entity", "arithmetic_expr", 
"arithmetic_expr_mul", "functions", 
-   "arithmetic_operands", "identifier_operand", 
"lambda_without_args", "lambda_with_args", 
-   "lambda_variables", "single_lambda_variable", "lambda_variable"
+   "arithmetic_operands", "identifier_operand", "default_operand", 
"lambda_without_args", 
+   "lambda_with_args", "lambda_variables", 
"single_lambda_variable", "lambda_variable", 
+   "match_expr", "match_clauses", "match_clause", 
"match_clause_action", 
+   "match_clause_check"
};
 
private static final String[] _LITERAL_NAMES = {
null, null, "'->'", "'\"'", "'''", "','", "'.'", null, null, 
null, null, 
null, "'=='", "'!='", "'<'", "'<='", "'>'", "'>='", "'?'", 
"':'", null, 
-   null, null, null, "'NaN'", "'-'", "'+'", "'/'", "'*'", "'{'", 
"'}'", "'['", 
-   "']'", "'('", "')'"
+   null, null, null, "'NaN'", null, null, "'=>'", "'-'", "'+'", 
"'/'", "'*'", 
+   "'{'", "'}'", "'['", "']'", "'('", "')'"
};
private static final String[] _SYMBOLIC_NAMES = {
null, "IN", "LAMBDA_OP", "DOUBLE_QUOTE", "SINGLE_QUOTE", 
"COMMA", "PERIOD", 
"AND", "OR", "NOT", "TRUE", "FALSE", "EQ", "NEQ", "LT", "LTE", 
"GT", "GTE", 
-   "QUESTION", "COLON", "IF", "THEN", "ELSE", "NULL", "NAN", 
"MINUS", "PLUS", 
-   "DIV", "MUL", "LBRACE", "RBRACE", "LBRACKET", "RBRACKET", 
"LPAREN", "RPAREN", 
-   "NIN", "EXISTS", "EXPONENT", 

metron git commit: METRON-1306: When index template install fails, we should fail the install closes apache/incubator-metron#834

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 610540ef0 -> 37587689e


METRON-1306: When index template install fails, we should fail the install 
closes apache/incubator-metron#834


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/37587689
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/37587689
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/37587689

Branch: refs/heads/master
Commit: 37587689e9feadeb54465aa7319f114a4e11255b
Parents: 610540e
Author: cstella 
Authored: Tue Dec 12 15:24:07 2017 -0500
Committer: cstella 
Committed: Tue Dec 12 15:24:07 2017 -0500

--
 .../METRON/CURRENT/package/scripts/indexing_master.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/37587689/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
index 8992950..c79dfd5 100755
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
@@ -90,8 +90,9 @@ class Indexing(Script):
 
 except Exception as e:
 msg = "WARNING: Elasticsearch index templates could not be 
installed.  " \
-  "Is Elasticsearch running?  Will reattempt install on next 
start.  error={0}"
+  "Is Elasticsearch running?  error={0}"
 Logger.warning(msg.format(e))
+raise
 
 commands.start_indexing_topology(env)
 

metron git commit: METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via cestella) closes apache/incubator-metron#829

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 358e40bd1 -> 17d1ac348


METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via cestella) 
closes apache/incubator-metron#829


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/17d1ac34
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/17d1ac34
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/17d1ac34

Branch: refs/heads/master
Commit: 17d1ac348138fe029e4a341c54201fc6b4b50448
Parents: 358e40b
Author: nickwallen <n...@nickallen.org>
Authored: Tue Nov 7 09:56:08 2017 -0500
Committer: cstella <ceste...@gmail.com>
Committed: Tue Nov 7 09:56:08 2017 -0500

--
 .../METRON/CURRENT/package/scripts/indexing_master.py| 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/17d1ac34/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
index e88b949..ed3e4ff 100755
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
@@ -80,14 +80,19 @@ class Indexing(Script):
 from params import params
 env.set_params(params)
 self.configure(env)
+commands = IndexingCommands(params)
+
 # Install elasticsearch templates
 try:
 if not commands.is_elasticsearch_template_installed():
 self.elasticsearch_template_install(env)
 commands.set_elasticsearch_template_installed()
-except:
-Logger.warning("WARNING: Elasticsearch templates could not be 
installed. The Elasticsearch service is probably down.")
-commands = IndexingCommands(params)
+
+except Exception as e:
+msg = "WARNING: Elasticsearch index templates could not be 
installed.  " \
+  "Is Elasticsearch running?  Will reattempt install on next 
start.  error={0}"
+Logger.warning(msg.format(e))
+
 commands.start_indexing_topology(env)
 
 def stop(self, env, upgrade_type=None):

metron git commit: METRON-1254: Conditionals as map keys do not function in Stellar closes apache/incubator-metron#801

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 5c41bf7f3 -> 191d5d84a


METRON-1254: Conditionals as map keys do not function in Stellar closes 
apache/incubator-metron#801


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/191d5d84
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/191d5d84
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/191d5d84

Branch: refs/heads/master
Commit: 191d5d84ae5ff9f7c464555c6a1285d9c5a702bf
Parents: 5c41bf7
Author: cstella 
Authored: Mon Oct 30 11:44:51 2017 -0400
Committer: cstella 
Committed: Mon Oct 30 11:44:51 2017 -0400

--
 .../metron/stellar/common/generated/Stellar.g4  |   5 +
 .../metron/stellar/common/StellarCompiler.java  |   4 +-
 .../common/generated/StellarBaseListener.java   |  12 +
 .../common/generated/StellarListener.java   |  12 +
 .../stellar/common/generated/StellarParser.java | 538 +++
 .../stellar/dsl/functions/BasicStellarTest.java |  19 +
 6 files changed, 366 insertions(+), 224 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/191d5d84/metron-stellar/stellar-common/src/main/antlr4/org/apache/metron/stellar/common/generated/Stellar.g4
--
diff --git 
a/metron-stellar/stellar-common/src/main/antlr4/org/apache/metron/stellar/common/generated/Stellar.g4
 
b/metron-stellar/stellar-common/src/main/antlr4/org/apache/metron/stellar/common/generated/Stellar.g4
index edd9632..dd92e3d 100644
--- 
a/metron-stellar/stellar-common/src/main/antlr4/org/apache/metron/stellar/common/generated/Stellar.g4
+++ 
b/metron-stellar/stellar-common/src/main/antlr4/org/apache/metron/stellar/common/generated/Stellar.g4
@@ -194,6 +194,8 @@ op_list :
   | op_list COMMA identifier_operand
   | conditional_expr
   | op_list COMMA conditional_expr
+  | comparison_expr
+  | op_list COMMA comparison_expr
   ;
 
 list_entity :
@@ -203,7 +205,9 @@ list_entity :
 
 kv_list :
   identifier_operand COLON transformation_expr
+  | comparison_expr COLON transformation_expr
   | kv_list COMMA identifier_operand COLON transformation_expr
+  | kv_list COMMA comparison_expr COLON transformation_expr
   ;
 
 map_entity :
@@ -249,6 +253,7 @@ identifier_operand :
   | NULL #NullConst
   | EXISTS LPAREN IDENTIFIER RPAREN #ExistsFunc
   | LPAREN conditional_expr RPAREN #condExpr_paren
+  | functions #func
   ;
 
 

http://git-wip-us.apache.org/repos/asf/metron/blob/191d5d84/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/StellarCompiler.java
--
diff --git 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/StellarCompiler.java
 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/StellarCompiler.java
index fbb5bc0..69564df 100644
--- 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/StellarCompiler.java
+++ 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/StellarCompiler.java
@@ -662,7 +662,7 @@ public class StellarCompiler extends StellarBaseListener {
   public void exitMap_entity(StellarParser.Map_entityContext ctx) {
 final FrameContext.Context context = getArgContext();
 expression.tokenDeque.push(new Token<>( (tokenDeque, state) -> {
-  HashMap args = new HashMap<>();
+  HashMap args = new HashMap<>();
   Object value = null;
   for (int i = 0; true; i++) {
 Token token = popDeque(tokenDeque);
@@ -672,7 +672,7 @@ public class StellarCompiler extends StellarBaseListener {
   if (i % 2 == 0) {
 value = token.getValue();
   } else {
-args.put(token.getValue() + "", value);
+args.put(token.getValue(), value);
   }
 }
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/191d5d84/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarBaseListener.java
--
diff --git 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarBaseListener.java
 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarBaseListener.java
index 6e0ae8c..d2806b9 100644
--- 
a/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarBaseListener.java
+++ 
b/metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/generated/StellarBaseListener.java
@@ -709,6 +709,18 @@ public class StellarBaseListener implements 
StellarListener {
 *
 * The default 

metron git commit: METRON-1243: Add a REST endpoint which allows us to get a list of all indice closes apache/incubator-metron#797

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master a25de0fc1 -> 1c12e22df


METRON-1243: Add a REST endpoint which allows us to get a list of all indice 
closes apache/incubator-metron#797


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1c12e22d
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1c12e22d
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1c12e22d

Branch: refs/heads/master
Commit: 1c12e22df03b53981ca8cef426ab26f263f289c0
Parents: a25de0f
Author: cstella 
Authored: Wed Oct 25 14:48:17 2017 -0400
Committer: cstella 
Committed: Wed Oct 25 14:48:17 2017 -0400

--
 .../SensorIndexingConfigController.java |  8 ++
 .../service/SensorIndexingConfigService.java|  2 +
 .../impl/SensorIndexingConfigServiceImpl.java   | 34 +++-
 ...IndexingConfigControllerIntegrationTest.java | 15 
 .../SensorIndexingConfigServiceImplTest.java| 83 +++-
 5 files changed, 140 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/1c12e22d/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorIndexingConfigController.java
--
diff --git 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorIndexingConfigController.java
 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorIndexingConfigController.java
index ffd80fb..ff1d6bd 100644
--- 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorIndexingConfigController.java
+++ 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/SensorIndexingConfigController.java
@@ -32,6 +32,7 @@ import org.springframework.web.bind.annotation.RequestMapping;
 import org.springframework.web.bind.annotation.RequestMethod;
 import org.springframework.web.bind.annotation.RestController;
 
+import java.util.List;
 import java.util.Map;
 
 @RestController
@@ -74,6 +75,13 @@ public class SensorIndexingConfigController {
 return new ResponseEntity<>(sensorIndexingConfigService.getAll(), 
HttpStatus.OK);
   }
 
+  @ApiOperation(value = "Retrieves all indices")
+  @ApiResponse(message = "Returns all the indices in use", code = 200)
+  @RequestMapping(value = "/list/indices/{writerName}", method = 
RequestMethod.GET)
+  ResponseEntity getAllIndices(@ApiParam(name="writerName", 
value="Writer name.  One of solr, elasticsearch or hdfs", 
required=true)@PathVariable String writerName) throws Exception {
+return new 
ResponseEntity<>(sensorIndexingConfigService.getAllIndices(writerName), 
HttpStatus.OK);
+  }
+
   @ApiOperation(value = "Deletes a SensorIndexingConfig from Zookeeper")
   @ApiResponses(value = { @ApiResponse(message = "SensorIndexingConfig was 
deleted", code = 200),
   @ApiResponse(message = "SensorIndexingConfig is missing", code = 
404) })

http://git-wip-us.apache.org/repos/asf/metron/blob/1c12e22d/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorIndexingConfigService.java
--
diff --git 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorIndexingConfigService.java
 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorIndexingConfigService.java
index 0699431..0f32e53 100644
--- 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorIndexingConfigService.java
+++ 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorIndexingConfigService.java
@@ -32,6 +32,8 @@ public interface SensorIndexingConfigService {
 
   List getAllTypes() throws RestException;
 
+  Iterable getAllIndices(String writerName) throws RestException;
+
   boolean delete(String name) throws RestException;
 
 }

http://git-wip-us.apache.org/repos/asf/metron/blob/1c12e22d/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SensorIndexingConfigServiceImpl.java
--
diff --git 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SensorIndexingConfigServiceImpl.java
 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SensorIndexingConfigServiceImpl.java
index 5c73b26..a90d123 100644
--- 
a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SensorIndexingConfigServiceImpl.java
+++ 
b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/SensorIndexingConfigServiceImpl.java
@@ -18,14 +18,16 @@
 package org.apache.metron.rest.service.impl;

metron git commit: METRON-1251: Typo and formatting fixes for metron-rest README closes apache/incubator-metron#800

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master cc111ec98 -> ad9851aff


METRON-1251: Typo and formatting fixes for metron-rest README closes 
apache/incubator-metron#800


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/ad9851af
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/ad9851af
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/ad9851af

Branch: refs/heads/master
Commit: ad9851afff3f4b351d26571df651c06b811db1da
Parents: cc111ec
Author: JonZeolla 
Authored: Fri Oct 20 18:34:38 2017 -0400
Committer: cstella 
Committed: Fri Oct 20 18:34:38 2017 -0400

--
 metron-interface/metron-rest/README.md | 100 ++--
 1 file changed, 50 insertions(+), 50 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/ad9851af/metron-interface/metron-rest/README.md
--
diff --git a/metron-interface/metron-rest/README.md 
b/metron-interface/metron-rest/README.md
index 3824917..b79b44d 100644
--- a/metron-interface/metron-rest/README.md
+++ b/metron-interface/metron-rest/README.md
@@ -14,19 +14,19 @@ This module provides a RESTful API for interacting with 
Metron.
 ### From Source
 
 1. Package the application with Maven:
-  ```
-mvn clean package
-  ```
+```
+mvn clean package
+```
 
 1. Untar the archive in the $METRON_HOME directory.  The directory structure 
will look like:
-  ```
-config
-  rest_application.yml
-bin
-  metron-rest
-lib
-  metron-rest-$METRON_VERSION.jar
-  ```
+```
+config
+  rest_application.yml
+bin
+  metron-rest
+lib
+  metron-rest-$METRON_VERSION.jar
+```
 
 1. Copy the `$METRON_HOME/bin/metron-rest` script to `/etc/init.d/metron-rest`
 
@@ -35,9 +35,9 @@ lib
 1. Deploy the RPM at 
`/metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-rest-$METRON_VERSION-*.noarch.rpm`
 
 1. Install the RPM with:
-  ```
-rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
-  ```
+```
+rpm -ih metron-rest-$METRON_VERSION-*.noarch.rpm
+```
 
 ## Configuration
 
@@ -76,7 +76,7 @@ No optional parameter has a default.
 | METRON_PRINCIPAL_NAME | Kerberos principal for the metron 
user| Optional
 | METRON_SERVICE_KEYTAB | Path to the Kerberos keytab for the 
metron user   | Optional
 
-These are set in the `/etc/sysconfig/metron` file.
+These are set in the `/etc/default/metron` file.
 
 ## Database setup
 
@@ -87,7 +87,7 @@ Spring uses Hibernate as the default ORM framework but 
another framework is need
 
 The REST application comes with [embedded database 
support](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html#boot-features-embedded-database-support)
 for development purposes.
 
-For example, edit these variables in `/etc/sysconfig/metron` before starting 
the application to configure H2:
+For example, edit these variables in `/etc/default/metron` before starting the 
application to configure H2:
 ```
 METRON_JDBC_DRIVER="org.h2.Driver"
 METRON_JDBC_URL="jdbc:h2:file:~/metrondb"
@@ -112,42 +112,42 @@ The following configures the application for MySQL:
 1. Install MySQL if not already available (this example uses version 5.7, 
installation instructions can be found 
[here](https://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html))
 
 1. Create a metron user and REST database and permission the user for that 
database:
-  ```
-CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
-CREATE DATABASE IF NOT EXISTS metronrest;
-GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'node1';
-  ```
+```
+CREATE USER 'metron'@'node1' IDENTIFIED BY 'Myp@ssw0rd';
+CREATE DATABASE IF NOT EXISTS metronrest;
+GRANT ALL PRIVILEGES ON metronrest.* TO 'metron'@'node1';
+```
 
 1. Install the MySQL JDBC client onto the REST application host and 
configurate the METRON_JDBC_CLIENT_PATH variable:
-  ```
-cd $METRON_HOME/lib
-wget 
https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.41.tar.gz
-tar xf mysql-connector-java-5.1.41.tar.gz
-  ```
-
-1. Edit these variables in `/etc/sysconfig/metron` to configure the REST 
application for MySQL:
-  ```
-METRON_JDBC_DRIVER="com.mysql.jdbc.Driver"
-METRON_JDBC_URL="jdbc:mysql://mysql_host:3306/metronrest"
-METRON_JDBC_USERNAME="metron"
-METRON_JDBC_PLATFORM="mysql"
-METRON_JDBC_CLIENT_PATH=$METRON_HOME/lib/mysql-connector-java-5.1.41/mysql-connector-java-5.1.41-bin.jar
-  ```
+```
+cd $METRON_HOME/lib
+wget 
https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.41.tar.gz
+tar xf mysql-connector-java-5.1.41.tar.gz
+```
+
+1. Edit these variables 

metron git commit: METRON-1350: Add reservoir sampling functions to Stellar closes apache/incubator-metron#867

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master adb024070 -> 3f0b1b7b4


METRON-1350: Add reservoir sampling functions to Stellar closes 
apache/incubator-metron#867


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/3f0b1b7b
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/3f0b1b7b
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/3f0b1b7b

Branch: refs/heads/master
Commit: 3f0b1b7b4a002d3f364bd2aee7b5921c0435c4a4
Parents: adb0240
Author: cstella 
Authored: Wed Dec 20 09:30:03 2017 -0500
Committer: cstella 
Committed: Wed Dec 20 09:30:03 2017 -0500

--
 metron-analytics/metron-statistics/README.md|  26 +++
 .../metron/statistics/sampling/Sampler.java |  41 +
 .../metron/statistics/sampling/SamplerUtil.java |  39 
 .../sampling/SamplingInitFunctions.java |  84 +
 .../sampling/SamplingOpsFunctions.java  | 178 +++
 .../statistics/sampling/UniformSampler.java |  95 ++
 .../sampling/SamplerFunctionsTest.java  | 130 ++
 .../statistics/sampling/UniformSamplerTest.java | 118 
 metron-stellar/stellar-common/README.md |   4 +
 9 files changed, 715 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/3f0b1b7b/metron-analytics/metron-statistics/README.md
--
diff --git a/metron-analytics/metron-statistics/README.md 
b/metron-analytics/metron-statistics/README.md
index 982132a..508b612 100644
--- a/metron-analytics/metron-statistics/README.md
+++ b/metron-analytics/metron-statistics/README.md
@@ -53,6 +53,32 @@ functions can be used from everywhere where Stellar is used.
   * bounds - A list of value bounds (excluding min and max) in sorted order.
 * Returns: Which bin N the value falls in such that bound(N-1) < value <= 
bound(N).  No min and max bounds are provided, so values smaller than the 0'th 
bound go in the 0'th bin, and values greater than the last bound go in the M'th 
bin.
 
+### Sampling Functions
+
+ `SAMPLE_ADD`
+* Description: Add a value or collection of values to a sampler.
+* Input:
+  * sampler - Sampler to use.  If null, then a default Uniform sampler is 
created.
+  * o - The value to add.  If o is an Iterable, then each item is added.
+* Returns: The sampler.
+
+ `SAMPLE_GET`
+* Description: Return the sample.
+* Input:
+  * sampler - Sampler to use.
+* Returns: The resulting sample.
+
+ `SAMPLE_INIT`
+* Description: Create a [reservoir 
sampler](https://en.wikipedia.org/wiki/Reservoir_sampling) of a specific size 
or, if unspecified, size 1024.  Elements sampled by the reservoir sampler will 
be included in the final sample with equal probability.
+* Input:
+  * size? - The size of the reservoir sampler.  If unspecified, the size is 
1024.
+* Returns: The sampler object.
+
+ `SAMPLE_MERGE`
+* Description: Merge and resample a collection of samples.
+* Input:
+  * samplers - A list of samplers to merge.
+* Returns: A sampler which represents the resampled merger of the samplers.
 
 ### Distributional Statistics
 

http://git-wip-us.apache.org/repos/asf/metron/blob/3f0b1b7b/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/sampling/Sampler.java
--
diff --git 
a/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/sampling/Sampler.java
 
b/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/sampling/Sampler.java
new file mode 100644
index 000..93104fb
--- /dev/null
+++ 
b/metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/sampling/Sampler.java
@@ -0,0 +1,41 @@
+/*
+ *
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ *
+ */
+package org.apache.metron.statistics.sampling;
+
+import java.util.function.Supplier;
+
+public interface Sampler extends Supplier {
+  int 

[15/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-management/index.html
--
diff --git a/site/current-book/metron-platform/metron-management/index.html 
b/site/current-book/metron-platform/metron-management/index.html
index cca14f1..f9ea1ce 100644
--- a/site/current-book/metron-platform/metron-management/index.html
+++ b/site/current-book/metron-platform/metron-management/index.html
@@ -1,1040 +1,644 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Stellar REPL Management Utilities
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Stellar REPL Management Utilities
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Stellar REPL Management Utilities
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-  

[31/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/amazon-ec2/index.html
--
diff --git a/site/current-book/metron-deployment/amazon-ec2/index.html 
b/site/current-book/metron-deployment/amazon-ec2/index.html
index 4e7e771..8bf69a8 100644
--- a/site/current-book/metron-deployment/amazon-ec2/index.html
+++ b/site/current-book/metron-deployment/amazon-ec2/index.html
@@ -1,536 +1,428 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Apache Metron on Amazon EC2
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Apache Metron on Amazon EC2
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Apache Metron on Amazon EC2
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   




   
-  
-
-  
-  
-Deployment
-
-  
-  
-
-  
-  
-Kerberos-ambari-setup
-
-  
-  
-
-  
-  
-Kerberos-manual-setup
-
-  
-  
-
-Amazon-ec2
-  
-   

metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Repository: metron
Updated Branches:
  refs/heads/feature/METRON-1416-upgrade-solr 417088763 -> 0717cfc25


Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/0717cfc2
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/0717cfc2
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/0717cfc2

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 0717cfc25acc80b80121b424bed93db7014a1376
Parents: 4170887
Author: cstella 
Authored: Mon Jun 11 21:41:06 2018 -0400
Committer: cstella 
Committed: Mon Jun 11 21:41:06 2018 -0400

--
 .../org/apache/metron/common/Constants.java |  1 +
 .../elasticsearch/dao/ElasticsearchDao.java |  4 ++
 .../dao/ElasticsearchMetaAlertDao.java  | 30 +-
 .../dao/ElasticsearchMetaAlertUpdateDao.java|  4 +-
 .../indexing/dao/metaalert/MetaAlertConfig.java | 43 
 .../AbstractLuceneMetaAlertUpdateDao.java   |  3 ++
 .../dao/metaalert/MetaAlertIntegrationTest.java |  2 +-
 .../AbstractLuceneMetaAlertUpdateDaoTest.java   | 26 +---
 .../metron/solr/dao/SolrMetaAlertDao.java   | 29 +
 .../metron/solr/dao/SolrMetaAlertSearchDao.java | 15 ---
 .../metron/solr/dao/SolrMetaAlertUpdateDao.java |  2 +-
 .../SolrMetaAlertIntegrationTest.java   | 28 +
 12 files changed, 128 insertions(+), 59 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/0717cfc2/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
index f74660c..4a8bea2 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
@@ -28,6 +28,7 @@ public class Constants {
   public static final long DEFAULT_CONFIGURED_BOLT_TIMEOUT = 5000;
   public static final String SENSOR_TYPE = "source.type";
   public static final String SENSOR_TYPE_FIELD_PROPERTY = "source.type.field";
+  public static final String THREAT_SCORE_FIELD_PROPERTY = 
"threat.triage.score.field";
   public static final String ENRICHMENT_TOPIC = "enrichments";
   public static final String INDEXING_TOPIC = "indexing";
   public static final String ERROR_STREAM = "error";

http://git-wip-us.apache.org/repos/asf/metron/blob/0717cfc2/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
index eae0a39..3eb86ce 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
@@ -85,6 +85,10 @@ public class ElasticsearchDao implements IndexDao {
 //uninitialized.
   }
 
+  public AccessConfig getAccessConfig() {
+return accessConfig;
+  }
+
   @Override
   public synchronized void init(AccessConfig config) {
 if (this.client == null) {

http://git-wip-us.apache.org/repos/asf/metron/blob/0717cfc2/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
index f73a640..649077e 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
@@ -68,6 +68,7 @@ import java.util.Map.Entry;
 import java.util.Optional;
 import java.util.Set;
 import java.util.UUID;
+import java.util.function.Supplier;
 import java.util.stream.Collectors;
 
 import static org.apache.metron.common.Constants.GUID;
@@ -83,10 +84,8 @@ public class ElasticsearchMetaAlertDao implements 
MetaAlertDao {
   public static final String THREAT_TRIAGE_FIELD = 
MetaAlertConstants.THREAT_FIELD_DEFAULT
   

[22/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-interface/metron-rest/index.html
--
diff --git a/site/current-book/metron-interface/metron-rest/index.html 
b/site/current-book/metron-interface/metron-rest/index.html
index f673ca6..eb378b3 100644
--- a/site/current-book/metron-interface/metron-rest/index.html
+++ b/site/current-book/metron-interface/metron-rest/index.html
@@ -1,253 +1,157 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Metron REST
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Metron REST
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Metron REST
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-Rest
-  
-   

[19/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-data-management/index.html
--
diff --git 
a/site/current-book/metron-platform/metron-data-management/index.html 
b/site/current-book/metron-platform/metron-data-management/index.html
index df34389..dea600c 100644
--- a/site/current-book/metron-platform/metron-data-management/index.html
+++ b/site/current-book/metron-platform/metron-data-management/index.html
@@ -1,359 +1,190 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Resource Data Management
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Resource Data Management
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Resource Data Management
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-

metron git commit: METRON-1601: Rename metaalert alert nested field to metron_alert to avoid collision closes apache/incubator-metron#1049

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 40796c06a -> edec7b18b


METRON-1601: Rename metaalert alert nested field to metron_alert to avoid 
collision closes apache/incubator-metron#1049


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/edec7b18
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/edec7b18
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/edec7b18

Branch: refs/heads/master
Commit: edec7b18bc40e40f35916489ba2fde2ca4916dc5
Parents: 40796c0
Author: cstella 
Authored: Fri Jun 8 09:49:54 2018 -0400
Committer: cstella 
Committed: Fri Jun 8 09:49:54 2018 -0400

--
 .../METRON/CURRENT/package/files/bro_index.template   |  2 +-
 .../METRON/CURRENT/package/files/error_index.template |  2 +-
 .../METRON/CURRENT/package/files/metaalert_index.template |  2 +-
 .../METRON/CURRENT/package/files/snort_index.template |  2 +-
 .../METRON/CURRENT/package/files/yaf_index.template   |  2 +-
 .../e2e/mock-data/alerts_ui_e2e_index.template|  2 +-
 .../app/alerts/alert-details/alert-details.component.ts   |  2 +-
 .../src/app/alerts/alerts-list/alerts-list.component.ts   |  2 +-
 .../alerts-list/table-view/table-view.component.html  | 10 +-
 .../alerts/alerts-list/table-view/table-view.component.ts |  2 +-
 .../metron-alerts/src/app/model/alert-source.ts   |  2 +-
 .../controller/MetaAlertControllerIntegrationTest.java|  4 ++--
 metron-platform/metron-elasticsearch/README.md|  8 
 .../ElasticsearchMetaAlertIntegrationTest.java| 10 +-
 .../java/org/apache/metron/indexing/dao/MetaAlertDao.java |  2 +-
 .../apache/metron/indexing/dao/InMemoryMetaAlertDao.java  |  2 +-
 16 files changed, 28 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
index 30f2591..17ad4d2 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
@@ -143,7 +143,7 @@
 "uid": {
   "type": "keyword"
 },
-"alert": {
+"metron_alert": {
   "type": "nested"
 },
 "ip_src_addr": {

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
index 00aaf87..d119509 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/error_index.template
@@ -38,7 +38,7 @@
 "error_type": {
   "type": "keyword"
 },
-"alert": {
+"metron_alert": {
   "type": "nested"
 }
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/edec7b18/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
--
diff --git 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
index 24e8357..05d5e32 100644
--- 
a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
+++ 
b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/metaalert_index.template
@@ -36,7 +36,7 @@
   "type": "date",
   "format": 

metron git commit: METRON-1568: Stellar should have a _ special variable which returns the message in map form closes apache/incubator-metron#1021

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 523c38cf6 -> 1b9828e6a


METRON-1568: Stellar should have a _ special variable which returns the message 
in map form closes apache/incubator-metron#1021


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1b9828e6
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1b9828e6
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1b9828e6

Branch: refs/heads/master
Commit: 1b9828e6a68fd39fd308ebb80f1d79cd490ceaec
Parents: 523c38c
Author: cstella 
Authored: Thu Jun 7 10:54:05 2018 -0400
Committer: cstella 
Committed: Thu Jun 7 10:54:05 2018 -0400

--
 .../enrichment/handler/StellarConfig.java   |  11 +-
 .../StellarTransformationTest.java  |  30 ++-
 .../adapters/stellar/StellarAdapterTest.java|  26 ++
 .../integration/EnrichmentIntegrationTest.java  |   2 +
 .../main/config/zookeeper/enrichments/test.json |   1 +
 .../metron/pcap/filter/PcapFieldResolver.java   |   5 +
 metron-stellar/stellar-common/README.md |   1 +
 .../metron/stellar/common/utils/ConcatMap.java  | 256 +++
 .../common/utils/StellarProcessorUtils.java |  51 +++-
 .../metron/stellar/dsl/MapVariableResolver.java |   8 +
 .../metron/stellar/dsl/VariableResolver.java|   1 +
 .../dsl/functions/DataStructureFunctions.java   |   4 +
 .../stellar/common/utils/ConcatMapTest.java |  83 ++
 .../stellar/dsl/functions/BasicStellarTest.java |  18 ++
 14 files changed, 487 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
index 62e0263..9f96a22 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
@@ -19,6 +19,7 @@ package 
org.apache.metron.common.configuration.enrichment.handler;
 
 import org.apache.metron.stellar.common.StellarAssignment;
 import org.apache.metron.stellar.common.StellarProcessor;
+import org.apache.metron.stellar.dsl.VariableResolver;
 import org.json.simple.JSONObject;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -142,8 +143,14 @@ public class StellarConfig implements Config {
   {
 
 Map messageSegment = new HashMap<>();
-for(String variable : stellarFields) {
-  messageSegment.put(variable, message.get(variable));
+if(stellarFields.contains(VariableResolver.ALL_FIELDS)) {
+  //we need to include all of the fields here.
+  messageSegment.putAll(message);
+}
+else {
+  for (String variable : stellarFields) {
+messageSegment.put(variable, message.get(variable));
+  }
 }
 return messageSegment;
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
--
diff --git 
a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
 
b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
index fc91844..3b7c7bb 100644
--- 
a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
+++ 
b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
@@ -78,6 +78,32 @@ public class StellarTransformationTest {
   @Multiline
   public static String badConfig;
 
+  /**
+   { "fieldTransformations" : [
+{ "transformation" : "STELLAR"
+,"output" : [ "new_field"]
+,"config" : {
+  "new_field" : "MAP_GET('source.type', _)"
+}
+}
+]
+  }
+   */
+ @Multiline
+ public static String configAll;
+
+  @Test
+  public void testConfigAll() throws Exception {
+SensorParserConfig c = 
SensorParserConfig.fromBytes(Bytes.toBytes(configAll));
+JSONObject input = new JSONObject();
+input.put("source.type", "test");
+for (FieldTransformer handler : c.getFieldTransformations()) {
+  handler.transformAndUpdate(input, 

[2/2] metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/a89a72c3
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/a89a72c3
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/a89a72c3

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: a89a72c3b2ceb5066fd7a310636507ce274c66ad
Parents: 10271c4 b081e80
Author: cstella 
Authored: Fri Jun 15 20:32:18 2018 -0400
Committer: cstella 
Committed: Fri Jun 15 20:32:18 2018 -0400

--
 .../alert-details/alert-details.component.html  |  4 +--
 .../alert-details/alert-details.component.ts| 24 ++---
 .../alerts-list/alerts-list.component.html  |  4 +--
 .../alerts/alerts-list/alerts-list.component.ts | 13 ++
 .../src/app/alerts/alerts-list/query-builder.ts |  5 ++--
 .../table-view/table-view.component.html| 18 ++---
 .../table-view/table-view.component.ts  | 19 --
 .../tree-view/tree-view.component.html  | 16 ++--
 .../tree-view/tree-view.component.ts| 27 +---
 .../meta-alerts/meta-alerts.component.html  |  2 +-
 .../alerts/meta-alerts/meta-alerts.component.ts | 11 +---
 .../src/app/model/group-request.ts  |  2 +-
 .../src/app/service/global-config.service.ts| 16 +---
 .../metron-alerts/src/app/utils/constants.ts|  3 +--
 .../ElasticsearchMetaAlertIntegrationTest.java  |  5 ++--
 15 files changed, 120 insertions(+), 49 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/a89a72c3/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
--
diff --cc 
metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
index ffd7dd0,c939f04..39d47ac
--- 
a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
+++ 
b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
@@@ -29,8 -30,8 +30,9 @@@ import {Patch} from '../../model/patch'
  import {AlertComment} from './alert-comment';
  import {AuthenticationService} from '../../service/authentication.service';
  import {MetronDialogBox} from '../../shared/metron-dialog-box';
- import {META_ALERTS_SENSOR_TYPE} from '../../utils/constants';
 -import {META_ALERTS_INDEX, META_ALERTS_SENSOR_TYPE} from 
'../../utils/constants';
 +import {CommentAddRemoveRequest} from 
"../../model/comment-add-remove-request";
++import {META_ALERTS_SENSOR_TYPE} from '../../utils/constants';
+ import { GlobalConfigService } from '../../service/global-config.service';
  
  export enum AlertState {
NEW, OPEN, ESCALATE, DISMISS, RESOLVE

http://git-wip-us.apache.org/repos/asf/metron/blob/a89a72c3/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
--
diff --cc 
metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
index 98459db,4496e37..07f7015
--- 
a/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
+++ 
b/metron-interface/metron-alerts/src/app/alerts/alerts-list/alerts-list.component.ts
@@@ -36,10 -36,10 +36,10 @@@ import {AlertSearchDirective} from '../
  import {SearchResponse} from '../../model/search-response';
  import {ElasticsearchUtils} from '../../utils/elasticsearch-utils';
  import {Filter} from '../../model/filter';
- import {THREAT_SCORE_FIELD_NAME, TIMESTAMP_FIELD_NAME, ALL_TIME} from 
'../../utils/constants';
+ import {TIMESTAMP_FIELD_NAME, ALL_TIME} from '../../utils/constants';
  import {TableViewComponent} from './table-view/table-view.component';
  import {Pagination} from '../../model/pagination';
 -import {META_ALERTS_SENSOR_TYPE, META_ALERTS_INDEX} from 
'../../utils/constants';
 +import {META_ALERTS_SENSOR_TYPE} from '../../utils/constants';
  import {MetaAlertService} from '../../service/meta-alert.service';
  import {Facets} from '../../model/facets';
  import { GlobalConfigService } from '../../service/global-config.service';

http://git-wip-us.apache.org/repos/asf/metron/blob/a89a72c3/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.html
--

http://git-wip-us.apache.org/repos/asf/metron/blob/a89a72c3/metron-interface/metron-alerts/src/app/alerts/alerts-list/table-view/table-view.component.ts
--

http://git-wip-us.apache.org/repos/asf/metron/blob/a89a72c3/metron-interface/metron-alerts/src/app/utils/constants.ts

metron git commit: METRON-1617: Make threat triage score function with dots as well as colons closes apache/incubator-metron#1062

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 828ab7134 -> b081e80c0


METRON-1617: Make threat triage score function with dots as well as colons 
closes apache/incubator-metron#1062


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/b081e80c
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/b081e80c
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/b081e80c

Branch: refs/heads/master
Commit: b081e80c07819fdaad0e5014790ddf02b2909c80
Parents: 828ab71
Author: cstella 
Authored: Fri Jun 15 19:59:13 2018 -0400
Committer: cstella 
Committed: Fri Jun 15 19:59:13 2018 -0400

--
 .../alert-details/alert-details.component.html  |  4 +--
 .../alert-details/alert-details.component.ts| 22 ++--
 .../alerts-list/alerts-list.component.html  |  4 +--
 .../alerts/alerts-list/alerts-list.component.ts | 13 ++
 .../src/app/alerts/alerts-list/query-builder.ts |  5 ++--
 .../table-view/table-view.component.html| 18 ++---
 .../table-view/table-view.component.ts  | 19 --
 .../tree-view/tree-view.component.html  | 16 ++--
 .../tree-view/tree-view.component.ts| 27 +---
 .../meta-alerts/meta-alerts.component.html  |  2 +-
 .../alerts/meta-alerts/meta-alerts.component.ts | 11 +---
 .../src/app/model/group-request.ts  |  2 +-
 .../src/app/service/global-config.service.ts| 16 +---
 .../metron-alerts/src/app/utils/constants.ts|  3 +--
 .../ElasticsearchMetaAlertIntegrationTest.java  |  2 ++
 15 files changed, 118 insertions(+), 46 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/b081e80c/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.html
--
diff --git 
a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.html
 
b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.html
index 8b0efae..f8fdc1d 100644
--- 
a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.html
+++ 
b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.html
@@ -34,8 +34,8 @@
 
 
 
- 
- {{ 
alertSource['threat:triage:score'] }} 
+ 
+ {{ getScore(alertSource) }} 
 
 
  {{ 
(alertSource.name && alertSource.name.length > 0)? alertSource.name : alertId | 
centerEllipses:20 }} 

http://git-wip-us.apache.org/repos/asf/metron/blob/b081e80c/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
--
diff --git 
a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
 
b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
index c8d0d7a..c939f04 100644
--- 
a/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
+++ 
b/metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
@@ -18,6 +18,7 @@
 import { Component, OnInit } from '@angular/core';
 import {Router, ActivatedRoute} from '@angular/router';
 import * as moment from 'moment/moment';
+import {Observable, Subscription} from 'rxjs/Rx';
 
 import {SearchService} from '../../service/search.service';
 import {UpdateService} from '../../service/update.service';
@@ -30,6 +31,7 @@ import {AlertComment} from './alert-comment';
 import {AuthenticationService} from '../../service/authentication.service';
 import {MetronDialogBox} from '../../shared/metron-dialog-box';
 import {META_ALERTS_INDEX, META_ALERTS_SENSOR_TYPE} from 
'../../utils/constants';
+import { GlobalConfigService } from '../../service/global-config.service';
 
 export enum AlertState {
   NEW, OPEN, ESCALATE, DISMISS, RESOLVE
@@ -71,6 +73,9 @@ export class AlertDetailsComponent implements OnInit {
   alertFields: string[] = [];
   alertCommentStr = '';
   alertCommentsWrapper: AlertCommentWrapper[] = [];
+  globalConfig: {} = {};
+  globalConfigService: GlobalConfigService;
+  configSubscription: Subscription;
 
   constructor(private router: Router,
   private activatedRoute: ActivatedRoute,
@@ -78,8 +83,9 @@ export class AlertDetailsComponent implements OnInit {
   private updateService: UpdateService,
   private alertsService: AlertsService,
   

metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Repository: metron
Updated Branches:
  refs/heads/feature/METRON-1416-upgrade-solr 476856192 -> 4c4452e59


Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/4c4452e5
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/4c4452e5
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/4c4452e5

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 4c4452e59c93306a049e6879f7b6fed2ee86c74e
Parents: 4768561
Author: cstella 
Authored: Tue Jun 12 21:40:01 2018 -0400
Committer: cstella 
Committed: Tue Jun 12 21:40:01 2018 -0400

--
 .../dao/ElasticsearchSearchDao.java |  30 +--
 .../ElasticsearchMetaAlertIntegrationTest.java  | 223 ++-
 2 files changed, 134 insertions(+), 119 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/4c4452e5/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchSearchDao.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchSearchDao.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchSearchDao.java
index 5725534..5cd0a4d 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchSearchDao.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchSearchDao.java
@@ -443,22 +443,24 @@ public class ElasticsearchSearchDao implements SearchDao {
   private List getGroupResults(GroupRequest groupRequest, int 
index, Aggregations aggregations, Map commonColumnMetadata) {
 List groups = groupRequest.getGroups();
 String field = groups.get(index).getField();
-Terms terms = aggregations.get(getGroupByAggregationName(field));
 List searchResultGroups = new ArrayList<>();
-for(Bucket bucket: terms.getBuckets()) {
-  GroupResult groupResult = new GroupResult();
-  groupResult.setKey(formatKey(bucket.getKey(), 
commonColumnMetadata.get(field)));
-  groupResult.setTotal(bucket.getDocCount());
-  Optional scoreField = groupRequest.getScoreField();
-  if (scoreField.isPresent()) {
-Sum score = 
bucket.getAggregations().get(getSumAggregationName(scoreField.get()));
-groupResult.setScore(score.getValue());
-  }
-  if (index < groups.size() - 1) {
-groupResult.setGroupedBy(groups.get(index + 1).getField());
-groupResult.setGroupResults(getGroupResults(groupRequest, index + 1, 
bucket.getAggregations(), commonColumnMetadata));
+if(aggregations != null) {
+  Terms terms = aggregations.get(getGroupByAggregationName(field));
+  for (Bucket bucket : terms.getBuckets()) {
+GroupResult groupResult = new GroupResult();
+groupResult.setKey(formatKey(bucket.getKey(), 
commonColumnMetadata.get(field)));
+groupResult.setTotal(bucket.getDocCount());
+Optional scoreField = groupRequest.getScoreField();
+if (scoreField.isPresent()) {
+  Sum score = 
bucket.getAggregations().get(getSumAggregationName(scoreField.get()));
+  groupResult.setScore(score.getValue());
+}
+if (index < groups.size() - 1) {
+  groupResult.setGroupedBy(groups.get(index + 1).getField());
+  groupResult.setGroupResults(getGroupResults(groupRequest, index + 1, 
bucket.getAggregations(), commonColumnMetadata));
+}
+searchResultGroups.add(groupResult);
   }
-  searchResultGroups.add(groupResult);
 }
 return searchResultGroups;
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/4c4452e5/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
 
b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
index 99f0490..224ec56 100644
--- 
a/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
+++ 
b/metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
@@ -1,20 +1,21 @@
+
 /*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for 

[1/2] metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Repository: metron
Updated Branches:
  refs/heads/feature/METRON-1416-upgrade-solr 0717cfc25 -> 476856192


Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/e7233b0f
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/e7233b0f
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/e7233b0f

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: e7233b0f82f372e6b1339de56a115fa7e4581f07
Parents: 4170887
Author: cstella 
Authored: Mon Jun 11 21:41:06 2018 -0400
Committer: cstella 
Committed: Tue Jun 12 09:46:46 2018 -0400

--
 .../org/apache/metron/common/Constants.java |  1 +
 .../elasticsearch/dao/ElasticsearchDao.java |  4 ++
 .../dao/ElasticsearchMetaAlertDao.java  | 30 +-
 .../dao/ElasticsearchMetaAlertUpdateDao.java|  4 +-
 .../indexing/dao/metaalert/MetaAlertConfig.java | 43 
 .../AbstractLuceneMetaAlertUpdateDao.java   |  3 ++
 .../dao/metaalert/MetaAlertIntegrationTest.java |  2 +-
 .../AbstractLuceneMetaAlertUpdateDaoTest.java   | 28 +
 .../metron/solr/dao/SolrMetaAlertDao.java   | 29 +
 .../metron/solr/dao/SolrMetaAlertSearchDao.java | 15 ---
 .../metron/solr/dao/SolrMetaAlertUpdateDao.java |  2 +-
 .../SolrMetaAlertIntegrationTest.java   | 28 +
 12 files changed, 129 insertions(+), 60 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/e7233b0f/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
index f74660c..4a8bea2 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
@@ -28,6 +28,7 @@ public class Constants {
   public static final long DEFAULT_CONFIGURED_BOLT_TIMEOUT = 5000;
   public static final String SENSOR_TYPE = "source.type";
   public static final String SENSOR_TYPE_FIELD_PROPERTY = "source.type.field";
+  public static final String THREAT_SCORE_FIELD_PROPERTY = 
"threat.triage.score.field";
   public static final String ENRICHMENT_TOPIC = "enrichments";
   public static final String INDEXING_TOPIC = "indexing";
   public static final String ERROR_STREAM = "error";

http://git-wip-us.apache.org/repos/asf/metron/blob/e7233b0f/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
index eae0a39..3eb86ce 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
@@ -85,6 +85,10 @@ public class ElasticsearchDao implements IndexDao {
 //uninitialized.
   }
 
+  public AccessConfig getAccessConfig() {
+return accessConfig;
+  }
+
   @Override
   public synchronized void init(AccessConfig config) {
 if (this.client == null) {

http://git-wip-us.apache.org/repos/asf/metron/blob/e7233b0f/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
--
diff --git 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
index f73a640..649077e 100644
--- 
a/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
+++ 
b/metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
@@ -68,6 +68,7 @@ import java.util.Map.Entry;
 import java.util.Optional;
 import java.util.Set;
 import java.util.UUID;
+import java.util.function.Supplier;
 import java.util.stream.Collectors;
 
 import static org.apache.metron.common.Constants.GUID;
@@ -83,10 +84,8 @@ public class ElasticsearchMetaAlertDao implements 
MetaAlertDao {
   public static final String THREAT_TRIAGE_FIELD = 
MetaAlertConstants.THREAT_FIELD_DEFAULT
   

[2/3] metron git commit: METRON-1588 Migrate storm-kafka-client to 1.2.1 closes apache/incubator-metron#1039

[cestella]

METRON-1588 Migrate storm-kafka-client to 1.2.1 closes 
apache/incubator-metron#1039


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/828ab713
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/828ab713
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/828ab713

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 828ab71346b458b73cb14d42d68e7471ead7fa4e
Parents: 7757046
Author: Jungtaek Lim 
Authored: Thu Jun 14 12:07:05 2018 -0400
Committer: cstella 
Committed: Thu Jun 14 12:07:05 2018 -0400

--
 NOTICE  |   6 +
 dependencies_with_url.csv   |   4 +
 metron-platform/metron-api/pom.xml  |   4 -
 .../parsers/topology/ParserTopologyBuilder.java |   2 +-
 .../metron-storm-kafka-override/pom.xml |  83 -
 .../KafkaSpoutRetryExponentialBackoff.java  | 336 +++
 .../storm/kafka/spout/internal/Timer.java   |   9 +-
 metron-platform/metron-storm-kafka/pom.xml  |   5 -
 pom.xml |   4 +-
 9 files changed, 435 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/NOTICE
--
diff --git a/NOTICE b/NOTICE
index ff6550b..5227768 100644
--- a/NOTICE
+++ b/NOTICE
@@ -10,3 +10,9 @@
This product includes software developed by Chef Software 
(https://www.chef.io)
Copyright (c) 2012-2015, Chef Software, Inc.
 
+   This includes derived works from the Apache Storm (ASLv2 licensed) project 
(https://github.com/apache/storm):
+   Copyright 2015 The Apache Software Foundation
+   The derived work is adapted from
+ org/apache/storm/kafka/spout/KafkaSpoutRetryExponentialBackoff.java
+ org/apache/storm/kafka/spout/internal/Timer.java
+   and can be found in the org.apache.storm.kafka package

http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/dependencies_with_url.csv
--
diff --git a/dependencies_with_url.csv b/dependencies_with_url.csv
index df3bcd2..438ce3e 100644
--- a/dependencies_with_url.csv
+++ b/dependencies_with_url.csv
@@ -119,15 +119,19 @@ 
com.fasterxml.jackson.core:jackson-annotations:jar:2.2.3:compile,ASLv2,http://wi
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.7.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.8.3:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.9.0:compile,ASLv2,http://github.com/FasterXML/jackson
+com.fasterxml.jackson.core:jackson-annotations:jar:2.9.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-core:jar:2.2.3:compile,ASLv2,http://wiki.fasterxml.com/JacksonHome
+com.fasterxml.jackson.core:jackson-core:jar:2.6.3:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.6.6:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.7.4:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.8.3:compile,ASLv2,https://github.com/FasterXML/jackson-core
+com.fasterxml.jackson.core:jackson-core:jar:2.9.4:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.9.5:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-databind:jar:2.2.3:compile,ASLv2,http://wiki.fasterxml.com/JacksonHome
 
com.fasterxml.jackson.core:jackson-databind:jar:2.4.3:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.7.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.8.3:compile,ASLv2,http://github.com/FasterXML/jackson
+com.fasterxml.jackson.core:jackson-databind:jar:2.9.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.9.5:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.6:compile,ASLv2,http://wiki.fasterxml.com/JacksonForCbor
 
com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.6.6:compile,ASLv2,http://wiki.fasterxml.com/JacksonForSmile

http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/metron-platform/metron-api/pom.xml
--
diff --git a/metron-platform/metron-api/pom.xml 
b/metron-platform/metron-api/pom.xml
index e3bf12d..59b1622 100644
--- a/metron-platform/metron-api/pom.xml
+++ b/metron-platform/metron-api/pom.xml
@@ -31,10 +31,6 @@

[3/3] metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/0e484279
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/0e484279
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/0e484279

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 0e4842799f8a3afa15074cb121bdbd205e50fb58
Parents: 4c4452e 828ab71
Author: cstella 
Authored: Thu Jun 14 12:18:43 2018 -0400
Committer: cstella 
Committed: Thu Jun 14 12:18:43 2018 -0400

--
 NOTICE  |   6 +
 dependencies_with_url.csv   |   4 +
 metron-platform/metron-api/pom.xml  |   4 -
 .../parsers/topology/ParserTopologyBuilder.java |   2 +-
 .../metron-storm-kafka-override/pom.xml |  83 -
 .../KafkaSpoutRetryExponentialBackoff.java  | 336 +++
 .../storm/kafka/spout/internal/Timer.java   |   9 +-
 metron-platform/metron-storm-kafka/pom.xml  |   5 -
 pom.xml |   4 +-
 site/documentation/index.md |  14 +-
 10 files changed, 442 insertions(+), 25 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/0e484279/dependencies_with_url.csv
--

http://git-wip-us.apache.org/repos/asf/metron/blob/0e484279/pom.xml
--

[1/3] metron git commit: METRON-1612 Fix website download links (justinleet) closes apache/metron#1058

[cestella]

Repository: metron
Updated Branches:
  refs/heads/feature/METRON-1416-upgrade-solr 4c4452e59 -> 0e4842799


METRON-1612 Fix website download links (justinleet) closes apache/metron#1058


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/77570464
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/77570464
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/77570464

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 77570464b4db205c3a3b95fcc3a0e8f7f6a1c540
Parents: 32d132b
Author: justinleet 
Authored: Mon Jun 11 10:35:09 2018 -0400
Committer: leet 
Committed: Mon Jun 11 10:35:09 2018 -0400

--
 site/documentation/index.md | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/77570464/site/documentation/index.md
--
diff --git a/site/documentation/index.md b/site/documentation/index.md
index 585f656..f842a67 100644
--- a/site/documentation/index.md
+++ b/site/documentation/index.md
@@ -82,18 +82,18 @@ title: Apache Metron Documentation
 http://www.apache.org/dyn/closer.cgi/metron/0.5.0/apache-metron-0.5.0.tar.gz;
 target="new">
 apache-metron-0.5.0.tar.gz
 
- [ https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.asc;
 target="new">
+ [ https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.asc; 
target="new">
 PGP
  ]
- [ https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha1;
 target="new">
-SHA
+ [ https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha1; 
target="new">
+SHA1
  ]
- [ https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha512;
 target="new">
+ [ https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha256;
 target="new">
+SHA256
+ ]
+ [ https://www.apache.org/dist/metron/0.5.0/apache-metron-0.5.0.tar.gz.sha512;
 target="new">
 SHA512
  ]
- [ https://dist.apache.org/repos/dist/release/metron/0.5.0/apache-metron-0.5.0.tar.gz.md5;
 target="new">
-MD5
-  ]
  [  Book ]
 
 

metron git commit: METRON-1588 Migrate storm-kafka-client to 1.2.1 closes apache/incubator-metron#1039

[cestella]

Repository: metron
Updated Branches:
  refs/heads/master 77570464b -> 828ab7134


METRON-1588 Migrate storm-kafka-client to 1.2.1 closes 
apache/incubator-metron#1039


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/828ab713
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/828ab713
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/828ab713

Branch: refs/heads/master
Commit: 828ab71346b458b73cb14d42d68e7471ead7fa4e
Parents: 7757046
Author: Jungtaek Lim 
Authored: Thu Jun 14 12:07:05 2018 -0400
Committer: cstella 
Committed: Thu Jun 14 12:07:05 2018 -0400

--
 NOTICE  |   6 +
 dependencies_with_url.csv   |   4 +
 metron-platform/metron-api/pom.xml  |   4 -
 .../parsers/topology/ParserTopologyBuilder.java |   2 +-
 .../metron-storm-kafka-override/pom.xml |  83 -
 .../KafkaSpoutRetryExponentialBackoff.java  | 336 +++
 .../storm/kafka/spout/internal/Timer.java   |   9 +-
 metron-platform/metron-storm-kafka/pom.xml  |   5 -
 pom.xml |   4 +-
 9 files changed, 435 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/NOTICE
--
diff --git a/NOTICE b/NOTICE
index ff6550b..5227768 100644
--- a/NOTICE
+++ b/NOTICE
@@ -10,3 +10,9 @@
This product includes software developed by Chef Software 
(https://www.chef.io)
Copyright (c) 2012-2015, Chef Software, Inc.
 
+   This includes derived works from the Apache Storm (ASLv2 licensed) project 
(https://github.com/apache/storm):
+   Copyright 2015 The Apache Software Foundation
+   The derived work is adapted from
+ org/apache/storm/kafka/spout/KafkaSpoutRetryExponentialBackoff.java
+ org/apache/storm/kafka/spout/internal/Timer.java
+   and can be found in the org.apache.storm.kafka package

http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/dependencies_with_url.csv
--
diff --git a/dependencies_with_url.csv b/dependencies_with_url.csv
index df3bcd2..438ce3e 100644
--- a/dependencies_with_url.csv
+++ b/dependencies_with_url.csv
@@ -119,15 +119,19 @@ 
com.fasterxml.jackson.core:jackson-annotations:jar:2.2.3:compile,ASLv2,http://wi
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.7.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.8.3:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-annotations:jar:2.9.0:compile,ASLv2,http://github.com/FasterXML/jackson
+com.fasterxml.jackson.core:jackson-annotations:jar:2.9.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-core:jar:2.2.3:compile,ASLv2,http://wiki.fasterxml.com/JacksonHome
+com.fasterxml.jackson.core:jackson-core:jar:2.6.3:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.6.6:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.7.4:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.8.3:compile,ASLv2,https://github.com/FasterXML/jackson-core
+com.fasterxml.jackson.core:jackson-core:jar:2.9.4:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-core:jar:2.9.5:compile,ASLv2,https://github.com/FasterXML/jackson-core
 
com.fasterxml.jackson.core:jackson-databind:jar:2.2.3:compile,ASLv2,http://wiki.fasterxml.com/JacksonHome
 
com.fasterxml.jackson.core:jackson-databind:jar:2.4.3:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.7.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.8.3:compile,ASLv2,http://github.com/FasterXML/jackson
+com.fasterxml.jackson.core:jackson-databind:jar:2.9.4:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.core:jackson-databind:jar:2.9.5:compile,ASLv2,http://github.com/FasterXML/jackson
 
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.6:compile,ASLv2,http://wiki.fasterxml.com/JacksonForCbor
 
com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.6.6:compile,ASLv2,http://wiki.fasterxml.com/JacksonForSmile

http://git-wip-us.apache.org/repos/asf/metron/blob/828ab713/metron-platform/metron-api/pom.xml
--
diff --git a/metron-platform/metron-api/pom.xml 
b/metron-platform/metron-api/pom.xml
index e3bf12d..59b1622 100644
--- a/metron-platform/metron-api/pom.xml
+++ 

[50/50] [abbrv] metron git commit: Merge branch 'master' into feature/METRON-1416-upgrade-solr

[cestella]

Merge branch 'master' into feature/METRON-1416-upgrade-solr


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/41708876
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/41708876
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/41708876

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 41708876336d009b3a2ff2c533bdf0294af1477b
Parents: f241f87 32d132b
Author: cstella 
Authored: Mon Jun 11 17:50:48 2018 -0400
Committer: cstella 
Committed: Mon Jun 11 17:50:48 2018 -0400

--
 KEYS|59 +
 Upgrading.md| 2 +-
 dev-utilities/release-utils/metron-rc-check |77 +-
 .../release-utils/validate-jira-for-release |75 +-
 metron-analytics/metron-maas-common/pom.xml | 2 +-
 metron-analytics/metron-maas-service/pom.xml| 2 +-
 metron-analytics/metron-profiler-client/pom.xml | 2 +-
 .../profiler/client/stellar/GetProfileTest.java | 3 +-
 .../client/stellar/ProfilerFunctionsTest.java   |15 +-
 metron-analytics/metron-profiler-common/pom.xml | 2 +-
 metron-analytics/metron-profiler/README.md  |15 +
 metron-analytics/metron-profiler/pom.xml| 2 +-
 .../src/main/flux/profiler/remote.yaml  | 3 +-
 metron-analytics/metron-statistics/pom.xml  | 2 +-
 .../StellarStatisticsFunctionsTest.java | 5 +-
 .../sampling/SamplerFunctionsTest.java  | 3 +-
 metron-analytics/pom.xml| 2 +-
 metron-contrib/metron-docker/pom.xml| 2 +-
 metron-contrib/metron-performance/pom.xml   | 2 +-
 metron-contrib/pom.xml  | 2 +-
 metron-deployment/Kerberos-manual-setup.md  | 8 +-
 metron-deployment/README.md |22 +-
 metron-deployment/amazon-ec2/conf/defaults.yml  | 2 +-
 .../ansible/playbooks/docker_probe_install.yml  | 2 +-
 .../roles/ambari_master/defaults/main.yml   | 2 +-
 .../roles/metron_pcapservice/defaults/main.yml  | 2 +-
 metron-deployment/development/centos6/README.md | 8 +-
 .../centos6/ansible/inventory/group_vars/all| 2 +-
 .../development/ubuntu14/README.md  | 6 +-
 .../ubuntu14/ansible/inventory/group_vars/all   | 2 +-
 .../ambari/elasticsearch-mpack/pom.xml  | 4 +-
 .../packaging/ambari/metron-mpack/pom.xml   | 4 +-
 .../CURRENT/package/files/bro_index.template| 2 +-
 .../CURRENT/package/files/error_index.template  | 2 +-
 .../package/files/metaalert_index.template  | 2 +-
 .../CURRENT/package/files/snort_index.template  | 2 +-
 .../CURRENT/package/files/yaf_index.template| 2 +-
 .../CURRENT/package/scripts/metron_service.py   | 5 +
 .../metron-mpack/src/main/resources/mpack.json  | 2 +-
 .../packaging/docker/deb-docker/pom.xml | 4 +-
 .../packaging/docker/rpm-docker/pom.xml | 4 +-
 metron-deployment/pom.xml   | 2 +-
 .../e2e/mock-data/alerts_ui_e2e_index.template  | 2 +-
 metron-interface/metron-alerts/package.json | 2 +-
 metron-interface/metron-alerts/pom.xml  | 2 +-
 .../alert-details/alert-details.component.ts| 2 +-
 .../alerts/alerts-list/alerts-list.component.ts | 2 +-
 .../table-view/table-view.component.html|10 +-
 .../table-view/table-view.component.ts  | 2 +-
 .../metron-alerts/src/app/model/alert-source.ts | 2 +-
 .../src/app/service/global-config.service.ts|10 +-
 metron-interface/metron-config/package.json | 2 +-
 metron-interface/metron-config/pom.xml  | 2 +-
 .../metron-config/scripts/package.json  | 2 +-
 metron-interface/metron-rest-client/pom.xml | 2 +-
 metron-interface/metron-rest/pom.xml| 2 +-
 .../rest/service/impl/SearchServiceImpl.java|10 +-
 .../src/main/resources/application.yml  | 2 +-
 .../metron-rest/src/main/scripts/metron-rest.sh | 5 +-
 .../MetaAlertControllerIntegrationTest.java | 4 +-
 .../service/impl/SearchServiceImplTest.java | 3 +-
 metron-interface/pom.xml| 2 +-
 metron-platform/Performance-tuning-guide.md | 4 +-
 metron-platform/README.md   | 2 +-
 metron-platform/elasticsearch-shaded/pom.xml| 2 +-
 metron-platform/metron-api/pom.xml  | 2 +-
 metron-platform/metron-common/README.md | 5 +
 metron-platform/metron-common/pom.xml   | 2 +-
 .../org/apache/metron/common/Constants.java | 1 +
 .../metron/common/bolt/ConfiguredBolt.java  |14 +-
 .../common/bolt/ConfiguredEnrichmentBolt.java   |11 +-
 .../common/bolt/ConfiguredIndexingBolt.java |13 +-
 

[38/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

  
-  
-  
-Use-cases
-  
-  
-
-
-
-
-
-  
-
-   
-
-
-
- http://maven.apache.org/; title="Built 
by Maven" class="poweredBy">
-
-  
-  
+
+  User Documentation
+Metron
+
+CONTRIBUTING
+Upgrading
+Analytics
+
+Maas-service
+Profiler
+Profiler-client
+Statistics
+
+
+Docker
+Performance
+Deployment
+Alerts
+Config
+Rest
+Platform
+Sensors
+Stellar-3rd-party-example
+Stellar-common
+Stellar-zeppelin
+Use-cases
+
+
+
+  
+  
+
+
+
+
+http://maven.apache.org/; title="Built by Maven" 
class="poweredBy">
+
   
 
-
-
-
-  
-Model Management Infrastructure
+
+
+Model Management Infrastructure
 
 
 Introduction
-One of the main features envisioned and requested is the ability to augment 
the threat intelligence and enrichment processes with insights derived from 
machine learning or statistical models. The challenges with this sort of 
infrastructure are
-
+One of the main features envisioned and requested is the ability to augment 
the threat intelligence and enrichment processes with insights derived from 
machine learning or statistical models.  The challenges with this sort of 
infrastructure are
 
-  
+
 Applying the model may be sufficiently computationally/resource intensive 
that we need to support scaling via load balancing, which will require service 
discovery and management.
-  
 Models require out of band and frequent training to react to growing 
threats and new patterns that emerge.
-  
-Models should be language/environment agnostic as much as possible. These 
should include small-data and big-data libraries and languages.
+Models should be language/environment agnostic as much as possible.  These 
should include small-data and big-data libraries and languages.
 
-To support a high throughput environment that is manageable, it is evident 
that 
-
+To support a high throughput environment that is manageable, it is evident 
that
 
-  
+
 Multiple versions of models will need to be exposed
-  
 Deployment should happen using Yarn to manage resources
-  
 Clients should have new model endpoints pushed to them
 
 
 Architecture
 
 To support these requirements, the following components have been 
created:
-
 
-  
+
 A Yarn application which will listen for model deployment requests and 
upon execution, register their endpoints in zookeeper:
-  
 
-
+
 Operation type: ADD, REMOVE, LIST
-
 Model Name
-
 Model Version
-
 Memory requirements (in megabytes)
-
 Number of instances
-  
-  
+
+
 A command line deployment client which will localize the model payload 
onto HDFS and submit a model request
-  
 A Java client which will interact with zookeeper and receive updates about 
model state changes (new deployments, removals, etc.)
-  
 A series of Stellar functions for interacting with models deployed via the 
Model as a Service infrastructure.
 
 
 maas_service.sh
-The maas_service.sh script starts the Yarn application which will 
listen for requests. Right now the queue for the requests is a distributed 
queue stored in http://curator.apache.org/curator-recipes/distributed-queue.html;>zookeeper
 for convenience.
+The maas_service.sh script starts the Yarn application which will 
listen for requests.  Right now the queue for the requests is a distributed 
queue stored in http://curator.apache.org/curator-recipes/distributed-queue.html;>zookeeper
 for convenience.
 
-
-
-./maas_service.sh
+
+
+./maas_service.sh
 usage: MaaSClient
  -c,--create  Flag to indicate whether to create
   the domain specified with -domain.
@@ -311,23 +177,21 @@ usage: MaaSClient
   given domain
  -zq,--zk_quorum argZookeeper Quorum
  -zr,--zk_root arg  Zookeeper Root
-
+
+
 
 maas_deploy.sh
-The maas_deploy.sh script allows users to deploy models and their 
collateral from their local disk to the cluster. It is assumed that the 
-
+The maas_deploy.sh script allows users to deploy models and their 
collateral from their local disk to the cluster. It is assumed that the
 
-  
+
 Collateral has exactly one .sh script capable of starting the 
endpoint
-  
 The model service executable will expose itself as a URL endpoint (e.g. as 
a REST interface, but not necessarily)
-  
 The model service executable will write out to local disk a JSON blob 
indicating the endpoint (see https://g

[05/50] [abbrv] metron git commit: METRON-1594: KafkaWriter is asynchronous and may lose data on node failure (mmiklavc via mmiklavc) closes apache/metron#1045

[cestella]

METRON-1594: KafkaWriter is asynchronous and may lose data on node failure 
(mmiklavc via mmiklavc) closes apache/metron#1045


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/523c38cf
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/523c38cf
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/523c38cf

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 523c38cf6399e2e3974a51a2cd0fe47e096b0bdf
Parents: b6808f7
Author: mmiklavc 
Authored: Wed Jun 6 15:40:55 2018 -0600
Committer: Michael Miklavcic 
Committed: Wed Jun 6 15:40:55 2018 -0600

--
 metron-analytics/metron-profiler/README.md  |  15 ++
 .../src/main/flux/profiler/remote.yaml  |   3 +-
 metron-platform/metron-common/README.md |   5 +
 .../metron/common/bolt/ConfiguredBolt.java  |  14 +-
 .../common/bolt/ConfiguredEnrichmentBolt.java   |  11 +-
 .../common/bolt/ConfiguredIndexingBolt.java |  13 +-
 .../common/bolt/ConfiguredParserBolt.java   |  13 +-
 .../common/bolt/ConfiguredProfilerBolt.java |   9 +-
 .../common/configuration/Configurations.java|   7 +-
 .../configuration/EnrichmentConfigurations.java |  26 +++
 .../configuration/IndexingConfigurations.java   |  12 +-
 .../configuration/ParserConfigurations.java |   1 +
 .../profiler/ProfilerConfigurations.java|  25 +++
 .../writer/ConfigurationStrategy.java   |  44 +
 .../writer/ConfigurationsStrategies.java| 144 +++
 .../writer/EnrichmentWriterConfiguration.java   | 110 
 .../writer/ParserWriterConfiguration.java   |   6 +-
 .../writer/ProfilerWriterConfiguration.java | 109 
 .../configuration/ParserConfigurationsTest.java | 120 +
 .../profiler/ProfilerConfigTest.java|  13 +-
 .../writer/ConfigurationsStrategiesTest.java|  79 +
 .../EnrichmentWriterConfigurationTest.java  |  54 ++
 .../writer/IndexingWriterConfigurationTest.java |  70 
 .../writer/ParserWriterConfigurationTest.java   |  72 
 .../writer/ProfilerWriterConfigurationTest.java |  54 ++
 .../writer/IndexingWriterConfigurationTest.java |  70 
 .../writer/ParserWriterConfigurationTest.java   |  38 
 metron-platform/metron-enrichment/README.md |  15 +-
 .../main/flux/enrichment/remote-splitjoin.yaml  |   9 +-
 .../main/flux/enrichment/remote-unified.yaml|   9 +-
 .../bolt/BulkMessageWriterBoltTest.java |  64 ---
 .../src/main/flux/indexing/batch/remote.yaml|   4 +-
 .../flux/indexing/random_access/remote.yaml |   4 +-
 metron-platform/metron-parsers/README.md|   6 +-
 .../apache/metron/parsers/bolt/ParserBolt.java  | 108 +++-
 .../metron/parsers/bolt/WriterHandler.java  |  63 +--
 .../metron/parsers/bolt/ParserBoltTest.java | 176 +--
 .../integration/WriterBoltIntegrationTest.java  |   6 +-
 .../metron/writer/bolt/BatchTimeoutHelper.java  |   6 +-
 .../writer/bolt/BulkMessageWriterBolt.java  |  48 ++---
 .../apache/metron/writer/kafka/KafkaWriter.java |  92 +++---
 41 files changed, 1426 insertions(+), 321 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/523c38cf/metron-analytics/metron-profiler/README.md
--
diff --git a/metron-analytics/metron-profiler/README.md 
b/metron-analytics/metron-profiler/README.md
index 79cdd44..1a17e10 100644
--- a/metron-analytics/metron-profiler/README.md
+++ b/metron-analytics/metron-profiler/README.md
@@ -538,6 +538,8 @@ The Profiler runs as an independent Storm topology.  The 
configuration for the P
 | [`profiler.hbase.batch`](#profilerhbasebatch)
 | The number of puts that are written to HBase in a single batch.
 | 
[`profiler.hbase.flush.interval.seconds`](#profilerhbaseflushintervalseconds) | 
The maximum number of seconds between batch writes to HBase.
 | [`topology.kryo.register`](#topologykryoregister)
 | Storm will use Kryo serialization for these classes.
+| [`profiler.writer.batchSize`](#profilerwriterbatchsize)  
 | The number of records to batch when writing to Kakfa.
+| [`profiler.writer.batchTimeout`](#profilerwriterbatchtimeout)
 | The timeout in ms for batching when writing to Kakfa.
 
 
 ### `profiler.input.topic`
@@ -852,6 +854,19 @@ More information on accessing profile data can be found in 
the [Profiler Client]
 
 More information on using the [`STATS_*` functions in Stellar can be found 
here](../../metron-platform/metron-common).
 
+### `profiler.writer.batchSize`
+
+*Default*: 15
+
+The number of records to batch when writing to Kakfa. This is managed in the 
global configuration and does not require a 

[27/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/packaging/ambari/index.html
--
diff --git a/site/current-book/metron-deployment/packaging/ambari/index.html 
b/site/current-book/metron-deployment/packaging/ambari/index.html
index 8013381..bac5758 100644
--- a/site/current-book/metron-deployment/packaging/ambari/index.html
+++ b/site/current-book/metron-deployment/packaging/ambari/index.html
@@ -1,505 +1,331 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Ambari Management Pack Development
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Ambari Management Pack Development
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Ambari Management Pack Development
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   




   
-  
-
-  
-  
-Deployment
-
-  
-  
-
-  
-  
-Kerberos-ambari-setup
-
-  
-  
-
-  
-  
-Kerberos-manual-setup
-
-  
-  
-
-  
-  

[08/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-stellar/stellar-zeppelin/index.html
--
diff --git a/site/current-book/metron-stellar/stellar-zeppelin/index.html 
b/site/current-book/metron-stellar/stellar-zeppelin/index.html
new file mode 100644
index 000..54bc800
--- /dev/null
+++ b/site/current-book/metron-stellar/stellar-zeppelin/index.html
@@ -0,0 +1,239 @@
+
+
+http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
+  
+
+
+
+
+Metron  Stellar Interpreter for Apache Zeppelin
+
+
+
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
+
+  
+
+  
+
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Stellar Interpreter for Apache Zeppelin
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
+  
+  
+
+  
+
+  User Documentation
+Metron
+
+CONTRIBUTING
+Upgrading
+Analytics
+Docker
+Performance
+Deployment
+Alerts
+Config
+Rest
+Platform
+Sensors
+Stellar-3rd-party-example
+Stellar-common
+Stellar-zeppelin
+Use-cases
+
+
+
+  
+  
+
+
+
+
+http://maven.apache.org/; title="Built by Maven" 
class="poweredBy">
+
+  
+
+
+
+Stellar Interpreter for Apache Zeppelin
+https://zeppelin.apache.org/;>Apache 
Zeppelin is a web-based notebook that enables data-driven, interactive data 
analytics and collaborative documents with SQL, Scala and more.  This project 
provides a means to run the Stellar REPL directly within a Zeppelin 
Notebook.
+
+
+Prerequisites
+Installation
+Usage
+
+
+Prerequisites
+
+
+https://zeppelin.apache.org/;>Apache 
Zeppelin 0.7.3
+This is tested with version 0.7.3.  Other versions may work, but are not 
supported.
+
+
+Installation
+Currently, you need to manually install the Stellar Interpreter in 
Zeppelin. In the future this step could be automated by the Metron Mpack.
+To install the Stellar Interpreter in your Apache Zeppelin installation, 
follow these instructions.  This is paraphrased from the https://zeppelin.apache.org/docs/latest/development/writingzeppelininterpreter.html#install-your-interpreter-binary;>Zeppelin
 docs.
+
+
+
+
+Build and install Metron. Metron and its dependencies will be retrieved 
from your local Maven repository.
+
+
+
+cd $METRON_HOME
+mvn clean install -DskipTests
+
+
+
+
+If you do not already have Zeppelin installed, https://zeppelin.apache.org/download.html;>download and unpack Apache 
Zeppelin.  Then change directories to the root of your Zeppelin 
download.
+
+
+
+cd $ZEPPELIN_HOME
+
+
+
+
+Use Zeppelins installation utility to install the Stellar 
Interpreter.
+If Zeppelin was already installed, make sure that it is stopped before 
running this command.  Update the version, 0.5.0 in the example 
below, to whatever is appropriate for your environment.
+
+
+
+bin/install-interpreter.sh --name stellar --artifact 
org.apache.metron:stellar-zeppelin:0.5.0
+
+
+
+
+Start Zeppelin.
+
+
+
+bin/zeppelin-daemon.sh start
+
+
+
+
+Navigate to Zeppelin running at http://localhost:8080/;>http://localhost:8080/.  The Stellar 
Interpreter should be ready for use with a basic set of functions.
+
+
+
+Usage
+
+
+
+
+Create a new notebook.
+
+
+
+
+Click on Notebook  Create new note.
+
+
+
+Set the default Interpreter to stellar.
+When creating the notebook, if you define stellar as the default 
interpreter, then there is no need to enter %stellar at the top of 
each code block.
+If stellar is not the default interpreter, then you must enter 
%stellar at the top of a code block containing Stellar code.
+
+
+
+
+
+In the first block, add the following Stellar, then click Run.
+
+
+
+2 in [2,3,4]
+
+
+
+
+In the next block, check which functions are available to you.
+When executing Stellars magic functions, you must explicitly define 
which interpreter should be used in the code block.  If you define 
stellar as the default interpreter when creating a notebook, 
then this is only required when using Stellars magic functions.
+
+
+
+%stellar
+
+%functions
+
+
+You will only see the functions defined within 
stellar-common since that is the only library that we added to the 
interpreter.
+
+
+
+Add additional Stellar functions to your session.
+
+
+
+
+Go back to the Stellar interpreter configuration and add another dependency 
as follows.
+
+
+
+org.apache.metron:metron-statistics:0.5.0
+
+
+
+
+Go back to your notebook and run %functions again.  You will now 
see the additional functions defined within 

[16/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-indexing/index.html
--
diff --git a/site/current-book/metron-platform/metron-indexing/index.html 
b/site/current-book/metron-platform/metron-indexing/index.html
index 3d79323..6d4b8d6 100644
--- a/site/current-book/metron-platform/metron-indexing/index.html
+++ b/site/current-book/metron-platform/metron-indexing/index.html
@@ -1,378 +1,200 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Indexing
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Indexing
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Indexing
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-   

[02/50] [abbrv] metron git commit: METRON-1553 Validate JIRA Script Error (nickwallen) closes apache/metron#1013

[cestella]

METRON-1553 Validate JIRA Script Error (nickwallen) closes apache/metron#1013


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/45e3ed87
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/45e3ed87
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/45e3ed87

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 45e3ed87ca4aceda7c8387389fc3a286ee5264d9
Parents: dbcc52c
Author: nickwallen 
Authored: Tue Jun 5 11:50:58 2018 -0400
Committer: nickallen 
Committed: Tue Jun 5 11:50:58 2018 -0400

--
 .../release-utils/validate-jira-for-release | 75 
 1 file changed, 46 insertions(+), 29 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/45e3ed87/dev-utilities/release-utils/validate-jira-for-release
--
diff --git a/dev-utilities/release-utils/validate-jira-for-release 
b/dev-utilities/release-utils/validate-jira-for-release
index df5776c..5fd4367 100755
--- a/dev-utilities/release-utils/validate-jira-for-release
+++ b/dev-utilities/release-utils/validate-jira-for-release
@@ -137,35 +137,52 @@ for i in "$@"; do
   esac
 done
 
-WORKDIR="~/tmp"
 
 # ensure all required values are set
 if [ -z "$VERSION" ]; then
-   echo "Missing -v/--version is is required"
-   exit 1
+  help
+  echo "Missing -v/--version is is required"
+  exit 1
 fi
 if [ -z "$START" ]; then
-   echo "Missing -s/--start which is required"
-   exit 1
+  help
+  echo "Missing -s/--start which is required"
+  exit 1
 fi
 if [ -z "$END" ]; then
-   echo "Missing -e/--end which is required"
-   exit 1
+  help
+  echo "Missing -e/--end which is required"
+  exit 1
 fi
 if [ -z "$REPO" ]; then
-   echo "Missing -r/--repo which is required"
-   exit 1
+  help
+  echo "Missing -r/--repo which is required"
+  exit 1
 fi
 if [ -z "$BRANCH" ]; then
-   echo "Missing -b/--branch which is required"
-   exit 1
+  help
+  echo "Missing -b/--branch which is required"
+  exit 1
 fi
 
-# clone the metron repo and fetch all tags
-git clone $REPO "metron-$VERSION"
+TMPDIR="$HOME/tmp"
+WORKDIR="$TMPDIR/metron-$VERSION"
+
+# warn the user if the working directory exists
+if [ -d "$WORKDIR" ]; then
+  read -p "  directory exists [$WORKDIR].  overwrite existing repo? [yN] " -n 
1 -r
+  echo
+  if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+exit 1
+  fi
+fi
+
+# fetch the repo and all tags
+rm -rf "$WORKDIR"
+git clone $REPO "$WORKDIR"
+cd "$WORKDIR"
 git checkout $BRANCH
-cd "$WORKDIR/metron-$VERSION"
-git fetch --all --tags
+git fetch --tags
 
 # find all JIRAs that have been committed since the last release
 GET_JIRAS="git log $START..$END --oneline | grep -E -o 'METRON[- ]*[0-9]+'"
@@ -177,21 +194,21 @@ printf "$FORMAT_STR" "JIRA" "STATUS" "FIX VERSION" 
"ASSIGNEE" "FIX"
 # for each JIRA since the last release tag...
 eval $GET_JIRAS | while read JIRA ; do
 
-   # fetch the JIRA content
-   
URL="https://issues.apache.org/jira/si/jira.issueviews:issue-xml/$JIRA/$JIRA.xml;
-CONTENT=`curl -s $URL`
+  # fetch the JIRA content
+  
URL="https://issues.apache.org/jira/si/jira.issueviews:issue-xml/$JIRA/$JIRA.xml;
+  CONTENT=`curl -s $URL`
 
-   # painfully extract some fields
-   STATUS=`echo "$CONTENT" | grep "]*>" | sed 
's/^.*]*>//' | sed 's/<.status>.*$//'`
-   ASSIGNEE=`echo "$CONTENT" | grep "]*>" | sed 
's/^.*]*>//' | sed 's/<.assignee>.*$//'`
-   FIXV=`echo "$CONTENT" | grep "]*>" | sed 
's/^.*]*>//' | sed 's/<.fixVersion>.*$//'`
+  # painfully extract some fields
+  STATUS=`echo "$CONTENT" | grep "]*>" | sed 's/^.*]*>//' 
| sed 's/<.status>.*$//'`
+  ASSIGNEE=`echo "$CONTENT" | grep "]*>" | sed 
's/^.*]*>//' | sed 's/<.assignee>.*$//'`
+  FIXV=`echo "$CONTENT" | grep "]*>" | sed 
's/^.*]*>//' | sed 's/<.fixVersion>.*$//'`
 
-   # the link is only populated, if there is something to fix
-   LINK=""
-   if [ "$FIXV" != "$VERSION" ] || [ "$STATUS" != "Done" ]; then
-   LINK="https://issues.apache.org/jira/browse/$JIRA;
-   fi
+  # the link is only populated, if there is something to fix
+  LINK=""
+  if [ "$FIXV" != "$VERSION" ] || [ "$STATUS" != "Done" ]; then
+LINK="https://issues.apache.org/jira/browse/$JIRA;
+  fi
 
-   # show the JIRA
-   printf "$FORMAT_STR" "$JIRA" "$STATUS" "$FIXV" "$ASSIGNEE" "$LINK"
+  # show the JIRA
+  printf "$FORMAT_STR" "$JIRA" "$STATUS" "$FIXV" "$ASSIGNEE" "$LINK"
 done

[06/50] [abbrv] metron git commit: METRON-1568: Stellar should have a _ special variable which returns the message in map form closes apache/incubator-metron#1021

[cestella]

METRON-1568: Stellar should have a _ special variable which returns the message 
in map form closes apache/incubator-metron#1021


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1b9828e6
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1b9828e6
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1b9828e6

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 1b9828e6a68fd39fd308ebb80f1d79cd490ceaec
Parents: 523c38c
Author: cstella 
Authored: Thu Jun 7 10:54:05 2018 -0400
Committer: cstella 
Committed: Thu Jun 7 10:54:05 2018 -0400

--
 .../enrichment/handler/StellarConfig.java   |  11 +-
 .../StellarTransformationTest.java  |  30 ++-
 .../adapters/stellar/StellarAdapterTest.java|  26 ++
 .../integration/EnrichmentIntegrationTest.java  |   2 +
 .../main/config/zookeeper/enrichments/test.json |   1 +
 .../metron/pcap/filter/PcapFieldResolver.java   |   5 +
 metron-stellar/stellar-common/README.md |   1 +
 .../metron/stellar/common/utils/ConcatMap.java  | 256 +++
 .../common/utils/StellarProcessorUtils.java |  51 +++-
 .../metron/stellar/dsl/MapVariableResolver.java |   8 +
 .../metron/stellar/dsl/VariableResolver.java|   1 +
 .../dsl/functions/DataStructureFunctions.java   |   4 +
 .../stellar/common/utils/ConcatMapTest.java |  83 ++
 .../stellar/dsl/functions/BasicStellarTest.java |  18 ++
 14 files changed, 487 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
--
diff --git 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
index 62e0263..9f96a22 100644
--- 
a/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
+++ 
b/metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/enrichment/handler/StellarConfig.java
@@ -19,6 +19,7 @@ package 
org.apache.metron.common.configuration.enrichment.handler;
 
 import org.apache.metron.stellar.common.StellarAssignment;
 import org.apache.metron.stellar.common.StellarProcessor;
+import org.apache.metron.stellar.dsl.VariableResolver;
 import org.json.simple.JSONObject;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -142,8 +143,14 @@ public class StellarConfig implements Config {
   {
 
 Map messageSegment = new HashMap<>();
-for(String variable : stellarFields) {
-  messageSegment.put(variable, message.get(variable));
+if(stellarFields.contains(VariableResolver.ALL_FIELDS)) {
+  //we need to include all of the fields here.
+  messageSegment.putAll(message);
+}
+else {
+  for (String variable : stellarFields) {
+messageSegment.put(variable, message.get(variable));
+  }
 }
 return messageSegment;
   }

http://git-wip-us.apache.org/repos/asf/metron/blob/1b9828e6/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
--
diff --git 
a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
 
b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
index fc91844..3b7c7bb 100644
--- 
a/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
+++ 
b/metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/StellarTransformationTest.java
@@ -78,6 +78,32 @@ public class StellarTransformationTest {
   @Multiline
   public static String badConfig;
 
+  /**
+   { "fieldTransformations" : [
+{ "transformation" : "STELLAR"
+,"output" : [ "new_field"]
+,"config" : {
+  "new_field" : "MAP_GET('source.type', _)"
+}
+}
+]
+  }
+   */
+ @Multiline
+ public static String configAll;
+
+  @Test
+  public void testConfigAll() throws Exception {
+SensorParserConfig c = 
SensorParserConfig.fromBytes(Bytes.toBytes(configAll));
+JSONObject input = new JSONObject();
+input.put("source.type", "test");
+for (FieldTransformer handler : c.getFieldTransformations()) {
+  handler.transformAndUpdate(input, Context.EMPTY_CONTEXT());
+}
+Assert.assertEquals(2, 

[10/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-sensors/pycapa/index.html
--
diff --git a/site/current-book/metron-sensors/pycapa/index.html 
b/site/current-book/metron-sensors/pycapa/index.html
index 8c46464..f81aee9 100644
--- a/site/current-book/metron-sensors/pycapa/index.html
+++ b/site/current-book/metron-sensors/pycapa/index.html
@@ -1,313 +1,199 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Pycapa
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Pycapa
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Pycapa
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-   

   

[13/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-parsers/parser-testing.html
--
diff --git 
a/site/current-book/metron-platform/metron-parsers/parser-testing.html 
b/site/current-book/metron-platform/metron-parsers/parser-testing.html
index 0edfd53..ff22013 100644
--- a/site/current-book/metron-platform/metron-parsers/parser-testing.html
+++ b/site/current-book/metron-platform/metron-parsers/parser-testing.html
@@ -1,356 +1,180 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Parser Contribution and Testing
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Parser Contribution and Testing
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Parser Contribution and Testing
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-

[42/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/maven-base.css
--
diff --git a/site/current-book/css/maven-base.css 
b/site/current-book/css/maven-base.css
new file mode 100644
index 000..322efae
--- /dev/null
+++ b/site/current-book/css/maven-base.css
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+body {
+  margin: 0px;
+  padding: 0px;
+}
+table {
+  padding:0px;
+  width: 100%;
+  margin-left: -2px;
+  margin-right: -2px;
+}
+acronym {
+  cursor: help;
+  border-bottom: 1px dotted #feb;
+}
+table.bodyTable th, table.bodyTable td {
+  padding: 2px 4px 2px 4px;
+  vertical-align: top;
+}
+div.clear{
+  clear:both;
+  visibility: hidden;
+}
+div.clear hr{
+  display: none;
+}
+#bannerLeft, #bannerRight {
+  font-size: xx-large;
+  font-weight: bold;
+}
+#bannerLeft img, #bannerRight img {
+  margin: 0px;
+}
+.xleft, #bannerLeft img {
+  float:left;
+}
+.xright, #bannerRight {
+  float:right;
+}
+#banner {
+  padding: 0px;
+}
+#breadcrumbs {
+  padding: 3px 10px 3px 10px;
+}
+#leftColumn {
+ width: 170px;
+ float:left;
+ overflow: auto;
+}
+#bodyColumn {
+  margin-right: 1.5em;
+  margin-left: 197px;
+}
+#legend {
+  padding: 8px 0 8px 0;
+}
+#navcolumn {
+  padding: 8px 4px 0 8px;
+}
+#navcolumn h5 {
+  margin: 0;
+  padding: 0;
+  font-size: small;
+}
+#navcolumn ul {
+  margin: 0;
+  padding: 0;
+  font-size: small;
+}
+#navcolumn li {
+  list-style-type: none;
+  background-image: none;
+  background-repeat: no-repeat;
+  background-position: 0 0.4em;
+  padding-left: 16px;
+  list-style-position: outside;
+  line-height: 1.2em;
+  font-size: smaller;
+}
+#navcolumn li.expanded {
+  background-image: url(../images/expanded.gif);
+}
+#navcolumn li.collapsed {
+  background-image: url(../images/collapsed.gif);
+}
+#navcolumn li.none {
+  text-indent: -1em;
+  margin-left: 1em;
+}
+#poweredBy {
+  text-align: center;
+}
+#navcolumn img {
+  margin-top: 10px;
+  margin-bottom: 3px;
+}
+#poweredBy img {
+  display:block;
+  margin: 20px 0 20px 17px;
+}
+#search img {
+margin: 0px;
+display: block;
+}
+#search #q, #search #btnG {
+border: 1px solid #999;
+margin-bottom:10px;
+}
+#search form {
+margin: 0px;
+}
+#lastPublished {
+  font-size: x-small;
+}
+.navSection {
+  margin-bottom: 2px;
+  padding: 8px;
+}
+.navSectionHead {
+  font-weight: bold;
+  font-size: x-small;
+}
+.section {
+  padding: 4px;
+}
+#footer {
+  padding: 3px 10px 3px 10px;
+  font-size: x-small;
+}
+#breadcrumbs {
+  font-size: x-small;
+  margin: 0pt;
+}
+.source {
+  padding: 12px;
+  margin: 1em 7px 1em 7px;
+}
+.source pre {
+  margin: 0px;
+  padding: 0px;
+}
+#navcolumn img.imageLink, .imageLink {
+  padding-left: 0px;
+  padding-bottom: 0px;
+  padding-top: 0px;
+  padding-right: 2px;
+  border: 0px;
+  margin: 0px;
+}

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/maven-theme.css
--
diff --git a/site/current-book/css/maven-theme.css 
b/site/current-book/css/maven-theme.css
new file mode 100644
index 000..4e2bdfb
--- /dev/null
+++ b/site/current-book/css/maven-theme.css
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+body {
+  padding: 0px 0px 10px 0px;
+}
+body, td, select, input, li{
+  font-family: Verdana, Helvetica, Arial, 

[43/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/css/apache-maven-fluido-1.7.min.css
--
diff --git a/site/current-book/css/apache-maven-fluido-1.7.min.css 
b/site/current-book/css/apache-maven-fluido-1.7.min.css
new file mode 100644
index 000..b436baf
--- /dev/null
+++ b/site/current-book/css/apache-maven-fluido-1.7.min.css
@@ -0,0 +1,17 @@
+/*!
+ * Bootstrap v2.3.2
+ *
+ * Copyright 2013 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world by @mdo and @fat.
+ 
*/.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0
 
a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin
 dotted #333;outline:5px auto 
-webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{width:auto\9;height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas
 img,.google-maps img{max-widt
 
h:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,html
 
input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}label,select,button,input[type="button"],input[type="reset"],input[type="submit"],input[type="radio"],input[type="checkbox"]{cursor:pointer}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}@media
 
print{*{color:#000!important;text-shadow:none!important;background:transparent!important;box-shadow:none!important}a,a:visited{text-decoration:underline}a[href]:after{content:"
 (" attr(href) ")"}abbr[title]:after{content:" (" attr(title) 
 ")"}.ir 
a:after,a[href^="javascript:"]:after,a[href^="#"]:after{content:""}pre,blockquote{border:1px
 solid 
#999;page-break-inside:avoid}thead{display:table-header-group}tr,img{page-break-inside:avoid}img{max-width:100%!important}@page{margin:.5cm}p,h2,h3{orphans:3;widows:3}h2,h3{page-break-after:avoid}}body{margin:0;font-family:"Helvetica
 
Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover,a:focus{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px
 solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0 1px 3px 
rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px 
rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-hei
 
ght:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:20px}.container,.navbar-static-top
 .container,.navbar-fixed-top .container,.navbar-fixed-bottom 
.container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offset1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid
 
[class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127
 
659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid
 [class*="span"]:first-child{margin-left:0}.row-fluid .controls-row 
[class*="span"]+[class*="span"]{margin-left:2.127659574468085%}.row-fluid 

[14/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-parsers/index.html
--
diff --git a/site/current-book/metron-platform/metron-parsers/index.html 
b/site/current-book/metron-platform/metron-parsers/index.html
index 23955ac..807a24e 100644
--- a/site/current-book/metron-platform/metron-parsers/index.html
+++ b/site/current-book/metron-platform/metron-parsers/index.html
@@ -1,379 +1,211 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Parsers
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Parsers
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Parsers
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-   

[20/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-platform/metron-api/index.html
--
diff --git a/site/current-book/metron-platform/metron-api/index.html 
b/site/current-book/metron-platform/metron-api/index.html
index 690ab94..ddcaec5 100644
--- a/site/current-book/metron-platform/metron-api/index.html
+++ b/site/current-book/metron-platform/metron-api/index.html
@@ -1,346 +1,161 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Metron PCAP Service
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Metron PCAP Service
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Metron PCAP Service
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   



 
-  
-
-  
-  
-Deployment
-  
-  
-  
-
-  
-  
-Alerts
-
-  
-  
-
-  
-  
-Config
-
-  
-  
-
-  
-  
-Rest
-
-   

[28/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
--
diff --git 
a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
 
b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
index eeadfe6..5cf9775 100644
--- 
a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
+++ 
b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html
@@ -1,387 +1,230 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   




   
-  
-
-  
-  
-Deployment
-
-  
-  
-
-  
-  
-Kerberos-ambari-setup
-
-  
-  
-
-  
-  
-Kerberos-manual-setup
-
-  
-  

[34/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-statistics/index.html
--
diff --git a/site/current-book/metron-analytics/metron-statistics/index.html 
b/site/current-book/metron-analytics/metron-statistics/index.html
index 29084e8..0c3515d 100644
--- a/site/current-book/metron-analytics/metron-statistics/index.html
+++ b/site/current-book/metron-analytics/metron-statistics/index.html
@@ -1,756 +1,599 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Statistics and Mathematical Functions
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Statistics and Mathematical Functions
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Statistics and Mathematical Functions
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   

 
-  
-
-  
-  
-Analytics
-
-  
-  
-
-  
-  
-Maas-service
-
-  
-  
-
-  
-  
-Profiler
-
-  
-  
-
-  
-  
-Profiler-client
-
-
-  
-
-Statistics
-  
-  
-  
-
-  
-  
-HLLP
-
-  
-
-  
-
-  
-  
-
-  
-  
-Docker
-
-   

   

[46/50] [abbrv] metron git commit: METRON-1572 Enhance KAFKA_PUT function (nickwallen) closes apache/metron#1024

[cestella]

METRON-1572 Enhance KAFKA_PUT function (nickwallen) closes apache/metron#1024


Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/40796c06
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/40796c06
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/40796c06

Branch: refs/heads/feature/METRON-1416-upgrade-solr
Commit: 40796c06ad96ae45dd853925fbae8c26509f6c2f
Parents: ae1d3eb
Author: nickwallen 
Authored: Fri Jun 8 08:39:20 2018 -0400
Committer: nickallen 
Committed: Fri Jun 8 08:39:20 2018 -0400

--
 .../metron/management/KafkaFunctions.java   | 87 +++-
 .../KafkaFunctionsIntegrationTest.java  | 21 +
 2 files changed, 89 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/metron/blob/40796c06/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
--
diff --git 
a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
 
b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
index 316e19d..a0c92eb 100644
--- 
a/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
+++ 
b/metron-platform/metron-management/src/main/java/org/apache/metron/management/KafkaFunctions.java
@@ -22,6 +22,7 @@ import org.apache.kafka.clients.consumer.ConsumerRecord;
 import org.apache.kafka.clients.consumer.KafkaConsumer;
 import org.apache.kafka.clients.producer.KafkaProducer;
 import org.apache.kafka.clients.producer.ProducerRecord;
+import org.apache.kafka.clients.producer.RecordMetadata;
 import org.apache.kafka.common.PartitionInfo;
 import org.apache.kafka.common.TopicPartition;
 import org.apache.kafka.common.serialization.StringDeserializer;
@@ -37,7 +38,7 @@ import org.slf4j.LoggerFactory;
 
 import java.lang.invoke.MethodHandles;
 import java.util.ArrayList;
-import java.util.Arrays;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
@@ -46,6 +47,9 @@ import java.util.Optional;
 import java.util.Properties;
 import java.util.Set;
 import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
 
 import static java.lang.String.format;
 import static org.apache.metron.stellar.dsl.Context.Capabilities.GLOBAL_CONFIG;
@@ -332,18 +336,26 @@ public class KafkaFunctions {
   /**
* KAFKA_PUT
*
-   * Sends messages to a Kafka topic.
+   * Sends messages to a Kafka topic.
*
-   * Example: Put two messages on the topic 'topic'.
+   * Example: Put two messages on the topic 'topic'.
+   * 
+   *  {@code
*  KAFKA_PUT('topic', ["message1", "message2"])
+   *  }
+   * 
*
-   * Example: Put a message on a topic and also define an alternative Kafka 
broker.
+   * Example: Put a message on a topic and also define an alternative Kafka 
broker.
+   * 
+   *  {@code
*  KAFKA_PUT('topic', ["message1"], { "bootstrap.servers": 
"kafka-broker-1:6667" })
+   *  }
+   * 
*/
   @Stellar(
   namespace = "KAFKA",
   name = "PUT",
-  description = "Sends messages to a Kafka topic.",
+  description = "Sends messages to a Kafka topic. ",
   params = {
   "topic - The name of the Kafka topic.",
   "messages - A list of messages to write.",
@@ -355,45 +367,82 @@ public class KafkaFunctions {
 
 @Override
 public Object apply(List args, Context context) throws 
ParseException {
-
   String topic = ConversionUtils.convert(args.get(0), String.class);
-  List messages = ConversionUtils.convert(args.get(1), List.class);
 
-  // build the properties for kafka
+  List messages;
+  if(args.get(1) instanceof String) {
+// a single message needs sent
+String msg = ConversionUtils.convert(args.get(1), String.class);
+messages = Collections.singletonList(msg);
+
+  } else {
+// a list of messages; all need sent
+messages = ConversionUtils.convert(args.get(1), List.class);
+  }
+
+  // are there any overrides?
   Map overrides = new HashMap<>();
   if(args.size() > 2) {
 overrides = ConversionUtils.convert(args.get(2), Map.class);
   }
-  Properties properties = buildKafkaProperties(overrides, context);
 
   // send the messages
-  try {
-send(topic, messages, properties);
-
-  } catch(InterruptedException | ExecutionException e) {
-throw new ParseException(e.getMessage(), e);
-  }
+  Properties properties = buildKafkaProperties(overrides, context);
+  

[23/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
--
diff --git 
a/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html 
b/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
deleted file mode 100644
index f72df78..000
--- a/site/current-book/metron-deployment/vagrant/quick-dev-platform/index.html
+++ /dev/null
@@ -1,406 +0,0 @@
-
-
-http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
-  
-
-
-
-
-Metron  Quick Development Platform
-
-
-
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
-
-  
-
-  
-
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Quick Development Platform
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
-  
-
-
-  
-
-  
-
-
-
-User Documentation
-   











  


-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   
   
-  
-
-  
-  
-Analytics
-  
-  
-  
-
-  
-  
-Docker
-
-   




   
-  
-
-  
-  
-Deployment
-
-  
-  
-
-  
-  
-Kerberos-ambari-setup
-
-  
-  
-
-  
-  
-Kerberos-manual-setup
-
-  
-  
-
-  
-  
-Amazon-ec2
-
-
-  
-
-  
-  
-Other-examples
-  
-  
-  
-
-  
-  
-Ambari
-
-  
-  
-
-  
-  
-Ansible-docker
-
-  

[36/50] [abbrv] metron git commit: METRON-1607 update public web site to point at 0.5.0 new release (justinleet) closes apache/metron#1053

[cestella]

http://git-wip-us.apache.org/repos/asf/metron/blob/ae1d3eb9/site/current-book/metron-analytics/metron-profiler/index.html
--
diff --git a/site/current-book/metron-analytics/metron-profiler/index.html 
b/site/current-book/metron-analytics/metron-profiler/index.html
index c3bf7ad..fe498af 100644
--- a/site/current-book/metron-analytics/metron-profiler/index.html
+++ b/site/current-book/metron-analytics/metron-profiler/index.html
@@ -1,295 +1,168 @@
 
 
 http://www.w3.org/1999/xhtml; xml:lang="en" lang="en">
   
 
 
-
+
 
 Metron  Metron Profiler
-
+
 
 
-
-  
-
-
-  
-
-$( document ).ready( function() { $( 
'.carousel' ).carousel( { interval: 3500 } ) } );
-  
-
-
-  
-
-
-
-
-  
-
-http://metron.apache.org/; 
id="bannerLeft">
-   
 
-
-  
-  
+
+
+  $( document ).ready( function() { $( '.carousel' ).carousel( { 
interval: 3500 } ) } );
+
+  
+  
+
+  
+http://metron.apache.org/; 
id="bannerLeft">
+
 
   
 
   
 
-
-
-  
-http://www.apache.org; class="externalLink" 
title="Apache">
-Apache
-
-  /
-
-http://metron.apache.org/; class="externalLink" 
title="Metron">
-Metron
-
-  /
-
-
-Documentation
-
-  /
-Metron Profiler
-
-
-
-  Last Published: 
2018-01-03 |
-  Version: 0.4.2
-
-
+  http://www.apache.org; class="externalLink" 
title="Apache">Apache/
+  http://metron.apache.org/; class="externalLink" 
title="Metron">Metron/
+  Documentation/
+Metron Profiler
+| 
Last Published: 2018-06-07
+  Version: 0.5.0
+
   
-
-
   
-
+
   
-
-
-
-User Documentation
-   











  
  
-  
-
-  
-  
-Metron
-
-  
-  
-
-  
-  
-Upgrading
-
-   

 
-  
-
-  
-  
-Analytics
-
-  
-  
-
-  
-  
-Maas-service
-
-  
-  
-
-Profiler
-  
-  
-  
-
-  
-  
-Profiler-client
-
-
-  
-
-  
-  
-Statistics
-  
-  
-
-  
-  
-
-  
-  
-Docker
-
-