date:20171109

[jira] [Commented] (METRON-1302) Split up Indexing Topology into batch and random access sections

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247099#comment-16247099
 ] 

ASF GitHub Bot commented on METRON-1302:


Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/831
  
That sounds great. We had a huge headache to find good tuning parameters 
for "indexing" topology. 


> Split up Indexing Topology into batch and random access sections
> 
>
> Key: METRON-1302
> URL: https://issues.apache.org/jira/browse/METRON-1302
> Project: Metron
>  Issue Type: Improvement
>Reporter: Casey Stella
>
> Currently we have the indexing topology handle writing to both random access 
> indices (e.g. elasticsearch) as well as batch write indices (e.g. hdfs).  We 
> should split these up and configure them separately.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-939) Upgrade ElasticSearch and Kibana

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247096#comment-16247096
 ] 

ASF GitHub Bot commented on METRON-939:
---

Github user mraliagha commented on the issue:

https://github.com/apache/metron/pull/840
  
It might not be completely related to this PR, but since _timestamp is 
removed in ES 5.x, can we add a specific time of indexing at indexing bolt to 
capture time of indexing. It is useful for the purpose of benchmarking as well 
as evaluation of SLA.


> Upgrade ElasticSearch and Kibana
> 
>
> Key: METRON-939
> URL: https://issues.apache.org/jira/browse/METRON-939
> Project: Metron
>  Issue Type: Improvement
>Reporter: Jon Zeolla
>Assignee: Michael Miklavcic
>
> Upgrade ElasticSearch and Kibana (latest is 5.4 as of writing this).  Among 
> other benefits, this allows us to use periods in field names 
> (https://github.com/elastic/elasticsearch/pull/19937/files), which has been 
> available as of 5.0 and 2.4, and the ability to index an IPv6 address 
> properly 
> (https://www.elastic.co/blog/indexing-ipv6-addresses-in-elasticsearch).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-939) Upgrade ElasticSearch and Kibana

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247028#comment-16247028
 ] 

ASF GitHub Bot commented on METRON-939:
---

GitHub user mmiklavc opened a pull request:

https://github.com/apache/metron/pull/840

METRON-939: Upgrade ElasticSearch and Kibana

DO NOT MERGE

This PR builds on the great work in 
https://github.com/apache/metron/pull/619

## Contributor Comments
I wanted to get eyes on this despite some outstanding work left. It was no 
small undertaking moving 3 major versions, so there's quite a bit to review. In 
the future I would like to recommend we make feature branches for changes of 
this magnitude. 

This work upgrades Elasticsearch and Kibana both to 5.6. I'm running a 
local full dev build now, but assuming no failures this will spin up full dev 
with everything, including the Kibana dashboard (had to be completely 
rewritten).

### TODO
- [ ] Another merge with master
- [ ] Update docs
- [ ] Docs for managing Kibana dashboards (no more pickle file - straight 
JSON using ES's bulk load API. This was not as easy as you would think it would 
be.)
- [ ] Test with Kerberos


## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
 
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && build_utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered by building and verifying the site-book? If not then run 
the following commands and the verify changes via 
`site-book/target/site/index.html`:

  ```
  cd site-book
  mvn site
  ```

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mmiklavc/metron elasticsearch562

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/840.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #840


commit 0805d20b95e3cdcf55bb0dfde91a08d8b9f58395
Author: Michael Miklavcic 
Date:   2017-10-02T18:51:23Z

Version changes for MPack

commit 889c1bd5d24ee359d7d8113f500539ea10750a40
Author: Michael Miklavcic 
Date:   2017-10-04T14:55:08Z

save demo code

commit 42f82180208dd7235214810406596bfe896665ab
Author: Michael Miklavcic 
Date:   2017-10-17T12:12:43Z

Working through tests

commit 084db5ab4258a1dacb792ba504c1aeb18838327d
Author:

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246730#comment-16246730
 ] 

ASF GitHub Bot commented on METRON-1289:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/824
  
The 2nd most recent commits updates the MetaAlertDao interface by adding 
additional methods for interacting with metaalerts.   This should make it 
easier to use and make the underlying implementation simpler and easier to 
understand by eliminating the need for complex logic that must catch all of the 
different use cases in the MetaAlertDao.update.  The high-level changes include:
- dedicated methods that allow adding/removing alerts to/from a metaalert
- dedicated method for changing the status of a metaalert
- direct updates through IndexDao.update are no longer allowed on 
metaalerts, the other MetaAlertDao methods must be used instead

The latest commit updates the ElasticsearchMetaAlertIntegrationTest with 
several changes:
- test data setup and test style is now consistent across all tests
- test cases for the new interface methods were added
- old test cases were reconciled with the new test cases
- test coverage for ElasticsearchMetaAlertDao is now close to 100% with 
tests added for all the bugs found so far

I am still working on adding these changes:
- updated javadocs and comments in all relevant classes and tests
- an HBaseDao.getAllLatest implementation and test
- updated ElasticsearchMetaAlertDaoTest adapted for the new MetaAlertDao 
interface (it's currently all commented out for the purpose of running the 
integration tests)

For now reviewers can start looking at the new MetaAlertDao interface 
methods and the accompanying integration tests in 
ElasticsearchMetaAlertIntegrationTest.  Hope to have the rest pushed out soon.


> Alert fields are lost when a MetaAlert is created
> -
>
> Key: METRON-1289
> URL: https://issues.apache.org/jira/browse/METRON-1289
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Assignee: Ryan Merriman
>
> When a MetaAlert is created, the included results are being updated 
> incorrectly with only the "metaalert" field.  This causes subsequent findOne 
> operations to only return the "metaalert field for that alert.  All fields 
> should continue to be present.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246677#comment-16246677
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150106587
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246583#comment-16246583
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150092533
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Created] (METRON-1312) Make implementation of findOne consistent between ES and HBase

2017-11-09 Thread Justin Leet (JIRA)

Justin Leet created METRON-1312:
---

 Summary: Make implementation of findOne consistent between ES and 
HBase
 Key: METRON-1312
 URL: https://issues.apache.org/jira/browse/METRON-1312
 Project: Metron
  Issue Type: Bug
Reporter: Justin Leet


Right now, the implementations differ slightly because of sensorType parameter.

Elasticsearch will take the parameter and use it to determine a type for the 
ids query it runs.  This means that the GUID will be found only if it's in the 
provided sensor (and will return nothing if it's present in another sensor).

HBase completely ignores sensorType and returns it no matter what sensor type.

Given that the GUIDs are universal, we can probably just chop out sensorType 
completely from the API call.  This probably primarily impacts the UI, but if 
anyone is using this endpoint outside of this, it would be a breaking change.

We could also just make the change to the underlying call to ignore the 
parameter completely.  I don't think any discussion around API deprecation and 
versioning has occurred, so this might be safer.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246461#comment-16246461
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150077476
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246456#comment-16246456
 ] 

ASF GitHub Bot commented on METRON-1310:


Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/838


> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246332#comment-16246332
 ] 

ASF GitHub Bot commented on METRON-1310:


Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/838
  
+1 by inspection. Thanks for the fix


> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1311) Service Check Should Check Elasticsearch Index Templates

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246331#comment-16246331
 ] 

ASF GitHub Bot commented on METRON-1311:


GitHub user nickwallen opened a pull request:

https://github.com/apache/metron/pull/839

METRON-1311 Service Check Should Check Elasticsearch Index Templates

The Service Check in Ambari does not validate that the Elasticsearch index 
templates have been installed.  Without these index templates bad things can 
happen.  For example, the Alerts UI will not be able to display any alerts.  
With this PR, the Elasticsearch index templates that are installed by Ambari 
are checked as part of the Metron Service Check.

## Testing

1. Launch a Metron environment like Full Dev

1. Run the Metron Service Check and ensure that it succeeds.  

1. Click on Metron > Metron Indexing > Delete Elasticsearch Templates

1. Run the Metron Service Check and ensure that it fails with an error like 
the following.
```
Missing Elasticsearch index template: name=yaf_index
```

1.  Click on Metron > Metron Indexing > Install Elasticsearch Templates.

1. Run the Metron Service Check and ensure that it succeeds.  

## Pull Request Checklist
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
 
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nickwallen/metron METRON-1311

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/839.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #839


commit 6565b351b0a75aff87acce97afd36978bea00e93
Author: Nick Allen 
Date:   2017-11-09T17:41:48Z

METRON-1310 "Template Delete" Action Deletes Search Indices

commit 621feefe2fc20cb93ea78d0acf36a26c2c9aa137
Author: Nick Allen 
Date:   2017-11-09T19:13:18Z

METRON-1311 Service Check Should Check Elasticsearch Index Templates




> Service Check Should Check Elasticsearch Index Templates
> 
>
> Key: METRON-1311
> URL: https://issues.apache.org/jira/browse/METRON-1311
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The Service Check in Ambari does not validate that the Elasticsearch index 
> templates have been installed.  Without these index templates bad things can 
> happen.  For example, the Alerts UI will not be able to display any alerts.
> The Elasticsearch index templates that are installed by Ambari should also be 
> checked as part of the Metron Service Check.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1309) Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246287#comment-16246287
 ] 

ASF GitHub Bot commented on METRON-1309:


Github user JonZeolla commented on a diff in the pull request:

https://github.com/apache/metron/pull/837#discussion_r150056693
  
--- Diff: 
metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.md ---
@@ -638,15 +638,15 @@ Edit crontab with `# crontab -e` and add:
 
 bro-kafka:
 ```
-# cp -r /root/metron/metron-sensors/bro-plugin-kafka /tmp
-# cd /tmp/bro-plugin-kafka
+# git clone https://github.com/apache/metron-bro-plugin-kafka 
/tmp/metron-bro-plugin-kafka
+# cd /tmp/metron-bro-plugin-kafka
 # rm -rf build/
--- End diff --

Not a bad idea - I was just migrating the instructions to use the new 
location without changing anything else.


> Change metron-deployment to pull the plugin from 
> apache/metron-bro-plugin-kafka
> ---
>
> Key: METRON-1309
> URL: https://issues.apache.org/jira/browse/METRON-1309
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Jon Zeolla
>Assignee: Jon Zeolla
>
> Change the metron-deployment ansible scripts to pull the plugin from 
> apache/metron-bro-plugin-kafka.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (METRON-1311) Service Check Should Check Elasticsearch Index Templates

2017-11-09 Thread Nick Allen (JIRA)

Nick Allen created METRON-1311:
--

 Summary: Service Check Should Check Elasticsearch Index Templates
 Key: METRON-1311
 URL: https://issues.apache.org/jira/browse/METRON-1311
 Project: Metron
  Issue Type: Bug
Affects Versions: 0.4.1
Reporter: Nick Allen
Assignee: Nick Allen
 Fix For: Next + 1


The Service Check in Ambari does not validate that the Elasticsearch index 
templates have been installed.  Without these index templates bad things can 
happen.  For example, the Alerts UI will not be able to display any alerts.

The Elasticsearch index templates that are installed by Ambari should also be 
checked as part of the Metron Service Check.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246261#comment-16246261
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150053090
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246190#comment-16246190
 ] 

ASF GitHub Bot commented on METRON-1310:


Github user cestella commented on the issue:

https://github.com/apache/metron/pull/838
  
yeesh, good catch @nickwallen +1 by inspection.


> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246147#comment-16246147
 ] 

ASF GitHub Bot commented on METRON-1310:


GitHub user nickwallen opened a pull request:

https://github.com/apache/metron/pull/838

METRON-1310 "Template Delete" Action Deletes Search Indices

The "Elasticsearch Template Delete" action within the Indexing service in 
Ambari, actually deletes the search indices.  Per the name of the action, I 
would expect it to only delete the template definitions.  This can be quite a 
fun surprise. :)

## Testing

1. Spin up Full Dev or an equivalent Metron environment.

1. Query Elasticsearch for the currently defined indices.  Take note of all 
the indices that you have defined.
```
curl http://node1:9200/_cat/indices?v
```

1. Query Elasticsearch for the currently defined templates.  There should 
be 5 in all; `snort_index`, `bro_index`, `metaalert_index`, `error_index`, and 
`yaf_index`.
```
curl http://node1:9200/_template | python -mjson.tool
```
1. Open Ambari and go to Metron > Metron Indexing.  Then on the hosts page, 
click on Metron Indexing > Elasticsearch Template Delete.

1. Query Elasticsearch again for the currently defined indices.  Make sure 
none were deleted.

1. Query Elasticsearch again for the currently defined templates.  There 
should be none.
```
curl http://node1:9200/_template | python -mjson.tool
```

1. Open Ambari and go to Metron > Metron Indexing.  Then on the hosts page, 
click on Metron Indexing > Elasticsearch Template Install.

1. Query Elasticsearch again for the currently defined templates.  There 
should be at least 5 in all.
```
curl http://node1:9200/_template | python -mjson.tool
```


## Pull Request Checklist
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
 
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nickwallen/metron METRON-1310

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/838.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #838


commit 6565b351b0a75aff87acce97afd36978bea00e93
Author: Nick Allen 
Date:   2017-11-09T17:41:48Z

METRON-1310 "Template Delete" Action Deletes Search Indices




> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246165#comment-16246165
 ] 

ASF GitHub Bot commented on METRON-1310:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/838#discussion_r150038546
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py
 ---
@@ -141,38 +141,43 @@ def elasticsearch_template_install(self, env):
 
 File(params.meta_index_path,
  mode=0755,
- content=StaticFile('meta_index.mapping')
+ content=StaticFile('meta_index.template')
  )
 
-bro_cmd = ambari_format(
-'curl -s -XPOST http://{es_http_url}/_template/bro_index -d 
@{bro_index_path}')
+bro_cmd = ambari_format('curl -s -XPOST 
http://{es_http_url}/_template/bro_index -d @{bro_index_path}')
 Execute(bro_cmd, logoutput=True)
-snort_cmd = ambari_format(
-'curl -s -XPOST http://{es_http_url}/_template/snort_index -d 
@{snort_index_path}')
+
+snort_cmd = ambari_format('curl -s -XPOST 
http://{es_http_url}/_template/snort_index -d @{snort_index_path}')
 Execute(snort_cmd, logoutput=True)
-yaf_cmd = ambari_format(
-'curl -s -XPOST http://{es_http_url}/_template/yaf_index -d 
@{yaf_index_path}')
+
+yaf_cmd = ambari_format('curl -s -XPOST 
http://{es_http_url}/_template/yaf_index -d @{yaf_index_path}')
 Execute(yaf_cmd, logoutput=True)
-error_cmd = ambari_format(
-'curl -s -XPOST http://{es_http_url}/_template/error_index -d 
@{error_index_path}')
-Execute(error_cmd, logoutput=True)
-error_cmd = ambari_format(
-'curl -s -XPOST http://{es_http_url}/metaalert_index -d 
@{meta_index_path}')
+
+error_cmd = ambari_format('curl -s -XPOST 
http://{es_http_url}/_template/error_index -d @{error_index_path}')
 Execute(error_cmd, logoutput=True)
 
+meta_cmd = ambari_format('curl -s -XPOST 
http://{es_http_url}/_template/metaalert_index -d @{meta_index_path}')
+Execute(meta_cmd, logoutput=True)
+
 def elasticsearch_template_delete(self, env):
 from params import params
 env.set_params(params)
 
-bro_cmd = ambari_format('curl -s -XDELETE 
"http://{es_http_url}/bro_index*;')
+bro_cmd = ambari_format('curl -s -XDELETE 
"http://{es_http_url}/_template/bro_index;')
--- End diff --

Let's delete the template, rather than the entire index.


> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246163#comment-16246163
 ] 

ASF GitHub Bot commented on METRON-1310:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/838#discussion_r150038381
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py
 ---
@@ -195,7 +195,7 @@
 snort_index_path = tmp_dir + "/snort_index.template"
 yaf_index_path = tmp_dir + "/yaf_index.template"
 error_index_path = tmp_dir + "/error_index.template"
-meta_index_path = tmp_dir + "/meta_index.mapping"
+meta_index_path = tmp_dir + "/meta_index.template"
--- End diff --

I renamed `meta_index.mapping` to `meta_index.template` to be consistent 
with the other templates.


> "Template Delete" Action Deletes Search Indices
> ---
>
> Key: METRON-1310
> URL: https://issues.apache.org/jira/browse/METRON-1310
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Nick Allen
>Assignee: Nick Allen
> Fix For: Next + 1
>
>
> The "Elasticsearch Template Delete" service action available within the 
> Indexing service actually deletes the search indices, rather than just 
> deleting the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246132#comment-16246132
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/774
  
Bringing over this from #530, 


https://github.com/apache/nifi/commit/d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6#diff-83e1afb34470ca47809f82aa1caf2138

is the commit in nifi for the nat-utils to diff.


> Bundles and Maven Plugin
> 
>
> Key: METRON-1212
> URL: https://issues.apache.org/jira/browse/METRON-1212
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>
> The first effort will be to land the bundle system and supporting maven 
> plugin on master



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (METRON-1310) "Template Delete" Action Deletes Search Indices

2017-11-09 Thread Nick Allen (JIRA)

Nick Allen created METRON-1310:
--

 Summary: "Template Delete" Action Deletes Search Indices
 Key: METRON-1310
 URL: https://issues.apache.org/jira/browse/METRON-1310
 Project: Metron
  Issue Type: Bug
Affects Versions: 0.4.1
Reporter: Nick Allen
Assignee: Nick Allen
 Fix For: Next + 1


The "Elasticsearch Template Delete" service action available within the 
Indexing service actually deletes the search indices, rather than just deleting 
the template definitions themselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1294) IP addresses are not formatted correctly in facet and group results

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246065#comment-16246065
 ] 

ASF GitHub Bot commented on METRON-1294:


Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/827
  
I submitted a PR against branch that adds a specific test case in the 
integration tests for this defect.  I validated that this test fails in master, 
but passes with your fix.  Feel free to include in your PR.


> IP addresses are not formatted correctly in facet and group results
> ---
>
> Key: METRON-1294
> URL: https://issues.apache.org/jira/browse/METRON-1294
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Assignee: Ryan Merriman
>
> Fields that are of type IP address are being returned in numerical format 
> whereas they should be returned in string format.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245962#comment-16245962
 ] 

ASF GitHub Bot commented on METRON-1289:


Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/824
  
No problem.  I am still actively working on this PR and will add the 
HBaseDao.findAllLatest implementation task to my list.


> Alert fields are lost when a MetaAlert is created
> -
>
> Key: METRON-1289
> URL: https://issues.apache.org/jira/browse/METRON-1289
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Assignee: Ryan Merriman
>
> When a MetaAlert is created, the included results are being updated 
> incorrectly with only the "metaalert" field.  This causes subsequent findOne 
> operations to only return the "metaalert field for that alert.  All fields 
> should continue to be present.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245944#comment-16245944
 ] 

ASF GitHub Bot commented on METRON-1289:


Github user cestella commented on the issue:

https://github.com/apache/metron/pull/824
  
I agree with @justinleet.  I think this is well within the scope of this PR 
since you're introducing a new method at an interface level.  In that 
situation, we should have either a default implementation or a specific 
implementation for every datastore where it makes sense to do the operation.  
In this case, HBase certainly can and should have a `findAllLatest` 
implementation.  As Justin said, I'd be ok with a default implementation at the 
interface level.


> Alert fields are lost when a MetaAlert is created
> -
>
> Key: METRON-1289
> URL: https://issues.apache.org/jira/browse/METRON-1289
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Assignee: Ryan Merriman
>
> When a MetaAlert is created, the included results are being updated 
> incorrectly with only the "metaalert" field.  This causes subsequent findOne 
> operations to only return the "metaalert field for that alert.  All fields 
> should continue to be present.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245893#comment-16245893
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150003217
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245886#comment-16245886
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r150002599
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245871#comment-16245871
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r15166
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245838#comment-16245838
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r149994944
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245826#comment-16245826
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r149993579
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245798#comment-16245798
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r149989360
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245782#comment-16245782
 ] 

ASF GitHub Bot commented on METRON-1212:


Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/774#discussion_r149985390
  
--- Diff: bundles-maven-plugin/README.md ---
@@ -0,0 +1,230 @@
+
+# Apache Metron Bundle Maven Plugin
+
+Apache Metron Bundles Maven Plugin helps to build Bundles Archives to 
support the classloader isolation model.
+
+## Table of Contents
+
+- [Requirements](#requirements)
+- [Building](#building)
+- [Getting Stared](#getting_started)
+- [Getting Help](#getting-help)
+- [License](#license)
+
+## Requirements
+* JDK 1.7 or higher
+* Apache Maven 3.1.0 or higher
+
+## Building 
+
+Building the bundles-maven-plugin module should be rare since it will be 
released infrequently compared to
+the main 'metron' code tree.
+
+- Build with `mvn clean install`
+- Presuming you need to make use of changes to the bundles-maven-plugin 
module, you should next
+  go to the [metron](../metron) directory and follow its instructions. 
+
+## Getting Started
+
+While it is most likely
+that a maven archetype is being utilized to create bundles, as part of a 
toolkit etc, you may want to create on manually, or may need to create a 
project for use in an archetype.
+
+The plugin is utilized by setting the packaging of a maven module to 
'bundle'.
+
+```xml
+bundle
+```
+
+This means that when you package this module, any of it's non-provided 
dependencies will be packaged into the produced bundle ( and all of their 
non-provided dependencies as well).
+Since a library may not always be distributed as part of a bundle with all 
it's dependencies, the bundle module
+shall be a separate module from the actual classes and dependencies to be 
bundled.
+
+A very simple example layout for a project that utilizes bundles would be:
+
+```bash
+├── README.md
+├── pom.xml
+├── testapp
+│   ├── pom.xml
+│   ├── src
+│   │   ├── main
+│   │   │   └── java
+│   │   │   └── org
+│   │   │   └── apache
+│   │   │   └── test
+│   │   │   └── App.java
+│   │   └── test
+│   │   └── java
+│   │   └── org
+│   │   └── apache
+│   │   └── test
+│   │   └── AppTest.java
+└── testappbundle
+├── pom.xml
+```
+Where testappbundle is the bundle module that creates a bundle of testapp, 
and contains the following pom.xml:
+```xml
+
+http://maven.apache.org/POM/4.0.0;
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+test.bundles.plugin
+org.apache.test
+1.0-SNAPSHOT
+  
+  4.0.0
+
+  test.app.bundle
+
+  
+  bundle
+  
+  
+  
+
+  org.apache.test
+  test.app
+  1.0-SNAPSHOT
+
+  
+
+  
+  
+
+  
+
+  org.apache.metron
+  bundles-maven-plugin
+  0.4.2
+  true
+  
+  
+
+  
+
+
+  
+org.apache.metron
+bundles-maven-plugin
+0.4.2
+true
+  
+
+  
+
+```
+When the module is packaged, it packages all of it's  non-provided 
dependencies into the bundles /bundled-dependencies directory.
+Thus, to create a bundle of a module's jar and that jar's non-provided 
dependencies, you add that module to your
+bundle modules dependencies.  You can unzip and examine the bundle in the 
target directory, and verify 
+it's contents, which should be similar to :
+
+```bash
+-> % tree .
+.
+└── META-INF
+├── MANIFEST.MF
+├── bundled-dependencies
+│   ├── log4j-1.2.17.jar
+│   ├── metron-common-0.4.1.jar
+│   ├── slf4j-api-1.7.7.jar
+│   ├── slf4j-log4j12-1.7.7.jar
+│   └── test.app-1.0-SNAPSHOT.jar
+└── maven
+└── org.apache.test
+└── test.app.bundle
+├── pom.properties
+└── pom.xml
+```
+
+This reflects the testapp project, which has these dependencies :
+
+```xml
+
+
+  org.apache.metron
+  metron-common
+  0.4.1
+
+
+  junit
+  junit
+  3.8.1
+  test
+
+  
+```
+metron-common

[jira] [Commented] (METRON-1270) fix for warnings missing @return tag argument in metron-analytics/metron-profiler-common and metron-profiler-client

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245730#comment-16245730
 ] 

ASF GitHub Bot commented on METRON-1270:


Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/810
  
@dbist Looks like this did get merged in (commit 
cf404f77d16daca5692f6197575bcc56c58fc2a9), but the PR is still open.  I don't 
see anything wrong with the commit.  Can you just close this PR after @cestella 
confirms we're in the clear (since I think he merged it)?


> fix for warnings missing @return tag argument in 
> metron-analytics/metron-profiler-common and metron-profiler-client
> ---
>
> Key: METRON-1270
> URL: https://issues.apache.org/jira/browse/METRON-1270
> Project: Metron
>  Issue Type: Bug
>Affects Versions: 0.4.1
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
>
> address the following warnings
> {noformat}
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/ProfileBuilder.java:66:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/hbase/ColumnBuilder.java:43:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/predicates/DateSpecifierPredicate.java:79:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/predicates/DayPredicates.java:109:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/predicates/HolidaysPredicate.java:82:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:50:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:63:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:77:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:91:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:103:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:115:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java:130:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java:61:
>  warning - @return tag has no arguments.
> [WARNING] 
> /Users/aervits/NetBeansProjects/METRON/metron/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java:393:
>  warning - @return tag has no arguments.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1273) Website documentation link should point to the current site-book

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245712#comment-16245712
 ] 

ASF GitHub Bot commented on METRON-1273:


Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/812
  
+1

@JonZeolla You should be good to go, if you're still going to be the one to 
push it through.


> Website documentation link should point to the current site-book
> 
>
> Key: METRON-1273
> URL: https://issues.apache.org/jira/browse/METRON-1273
> Project: Metron
>  Issue Type: Improvement
>Reporter: Jon Zeolla
>Assignee: Jon Zeolla
>Priority: Trivial
>
> Currently, the metron.apache.org site[1] is pointing to the wiki 
> documentation page[2], which is no longer the correct place to point for 
> Metron documentation.  We should repoint it to the current site-book.
> 1:  http://metron.apache.org/documentation/#docshome
> 2:  https://cwiki.apache.org/confluence/display/METRON/Documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1309) Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka

2017-11-09 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245607#comment-16245607
 ] 

ASF GitHub Bot commented on METRON-1309:


Github user JonZeolla commented on the issue:

https://github.com/apache/metron/pull/837
  
This is ready for review.


> Change metron-deployment to pull the plugin from 
> apache/metron-bro-plugin-kafka
> ---
>
> Key: METRON-1309
> URL: https://issues.apache.org/jira/browse/METRON-1309
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Jon Zeolla
>Assignee: Jon Zeolla
>
> Change the metron-deployment ansible scripts to pull the plugin from 
> apache/metron-bro-plugin-kafka.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (METRON-1302) Split up Indexing Topology into batch and random access sections

[jira] [Commented] (METRON-939) Upgrade ElasticSearch and Kibana

[jira] [Commented] (METRON-939) Upgrade ElasticSearch and Kibana

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Created] (METRON-1312) Make implementation of findOne consistent between ES and HBase

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1311) Service Check Should Check Elasticsearch Index Templates

[jira] [Commented] (METRON-1309) Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka

[jira] [Created] (METRON-1311) Service Check Should Check Elasticsearch Index Templates

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Created] (METRON-1310) "Template Delete" Action Deletes Search Indices

[jira] [Commented] (METRON-1294) IP addresses are not formatted correctly in facet and group results

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

[jira] [Commented] (METRON-1289) Alert fields are lost when a MetaAlert is created

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1212) Bundles and Maven Plugin

[jira] [Commented] (METRON-1270) fix for warnings missing @return tag argument in metron-analytics/metron-profiler-common and metron-profiler-client

[jira] [Commented] (METRON-1273) Website documentation link should point to the current site-book

[jira] [Commented] (METRON-1309) Change metron-deployment to pull the plugin from apache/metron-bro-plugin-kafka

33 matches

Site Navigation

Mail list logo

Footer information