[jira] [Commented] (METRON-1825) Upgrade bro to 2.5.5
[ https://issues.apache.org/jira/browse/METRON-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687143#comment-16687143 ] ASF GitHub Bot commented on METRON-1825: Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1237 Yup, that was actually how I found METRON-1826 was an issue. Sorry, I should have noted that. > Upgrade bro to 2.5.5 > > > Key: METRON-1825 > URL: https://issues.apache.org/jira/browse/METRON-1825 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1237: METRON-1825: Upgrade bro to 2.5.5
Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1237 Yup, that was actually how I found METRON-1826 was an issue. Sorry, I should have noted that. ---
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687123#comment-16687123 ] ASF GitHub Bot commented on METRON-1340: Github user gmcdonald closed the pull request at: https://github.com/apache/metron/pull/857 > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #857: METRON-1340: Improve e2e tests for metron alerts
Github user gmcdonald closed the pull request at: https://github.com/apache/metron/pull/857 ---
[jira] [Commented] (METRON-1340) Improve e2e tests for metron alerts
[ https://issues.apache.org/jira/browse/METRON-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687059#comment-16687059 ] ASF GitHub Bot commented on METRON-1340: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/857 I also requested that this be closed in this ticket; https://issues.apache.org/jira/browse/INFRA-17251 > Improve e2e tests for metron alerts > --- > > Key: METRON-1340 > URL: https://issues.apache.org/jira/browse/METRON-1340 > Project: Metron > Issue Type: Bug >Reporter: RaghuMitra >Assignee: RaghuMitra >Priority: Major > > Need to improve e2e tests in the following areas: > - Tests should not be flaky > - Remove the sleep ( This should implicitly make the tests run faster) > - Truncate HBase table 'metron_update' before starting the tests > - Improve the tests descriptions > - Run the tests headless if possible > - Check the node version and browser version before launching the tests > The expected behavior is that there are no intermittent failures. Acceptance > criteria: 5 consecutive runs without failures. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/857 I also requested that this be closed in this ticket; https://issues.apache.org/jira/browse/INFRA-17251 ---
[jira] [Commented] (METRON-1749) Update Angular to latest release in Management UI
[ https://issues.apache.org/jira/browse/METRON-1749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687053#comment-16687053 ] ASF GitHub Bot commented on METRON-1749: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1217 > Update Angular to latest release in Management UI > - > > Key: METRON-1749 > URL: https://issues.apache.org/jira/browse/METRON-1749 > Project: Metron > Issue Type: Improvement >Reporter: Shane Ardell >Assignee: Shane Ardell >Priority: Major > > Currently, the Management UI is on Angular v2. Not only has there been many > updates and improvements to Angular, that release is no longer supported by > the Angular team. This means critical fixes and security patches are no > longer being done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1217: METRON-1749: Update Angular to latest release in ...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1217 ---
[GitHub] metron-bro-plugin-kafka pull request #18: Remove version from show-plugin bt...
GitHub user dopheide-esnet opened a pull request: https://github.com/apache/metron-bro-plugin-kafka/pull/18 Remove version from show-plugin btest output in prep for Bro 2.7 ## Contributor Comments Update show-plugin.bro test to remove the version number from the baseline output. With Bro 2.7, the version will be built from major.minor.patch, which conflicts with the current major.minor in the baseline. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron's Bro kafka writer plugin. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed via: ``` bro-pkg test $GITHUB_USERNAME/metron-bro-plugin-kafka --version $BRANCH ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Apache Metron's [Vagrant full-dev environment](https://github.com/apache/metron/tree/master/metron-deployment/development/centos6) or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/dopheide-esnet/metron-bro-plugin-kafka topic/dopheide/2.7-btest-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron-bro-plugin-kafka/pull/18.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #18 commit 44bfd8b51f7fdda6a8ded6c6e998a9094adc0890 Author: Michael Dopheide Date: 2018-11-14T19:13:17Z Remove version from show-plugin btest output in prep for Bro 2.7 ---
[GitHub] metron issue #526: Metron-846: Add E2E tests for metron management ui
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/526 I opened a ticket to get this closed; https://issues.apache.org/jira/browse/INFRA-17251 ---
[jira] [Commented] (METRON-1762) Change default Elasticsearch field rename behavior to NOOP
[ https://issues.apache.org/jira/browse/METRON-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686958#comment-16686958 ] ASF GitHub Bot commented on METRON-1762: Github user merrimanr closed the pull request at: https://github.com/apache/metron/pull/1181 > Change default Elasticsearch field rename behavior to NOOP > -- > > Key: METRON-1762 > URL: https://issues.apache.org/jira/browse/METRON-1762 > Project: Metron > Issue Type: Improvement >Reporter: Ryan Merriman >Assignee: Ryan Merriman >Priority: Major > > Now that our version of ES supports '.' characters in field names, we should > change the default behavior of the indexing topology to NOT rename fields > with ':' characters. This will make it consistent with the format of data in > other stores (HDFS, Solr, etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1778) Out-of-order timestamps may delay flush in Storm Profiler
[ https://issues.apache.org/jira/browse/METRON-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686957#comment-16686957 ] ASF GitHub Bot commented on METRON-1778: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1197 @JonZeolla That error is a pre-existing condition that is tracked here;[ METRON-1810](https://issues.apache.org/jira/browse/METRON-1810). I am not sure what the root cause is. > Out-of-order timestamps may delay flush in Storm Profiler > - > > Key: METRON-1778 > URL: https://issues.apache.org/jira/browse/METRON-1778 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > When timestamps are received out-of-order there can be cases where a flush > signal should have been signalled, but is not. The flush signal can be > either delayed slightly or occur prematurely. > The smaller the profile period is, the more likely this is to impact the > results. I would not expect this to greatly impact the results of the > Profiler under normal usage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1197: METRON-1778 Out-of-order timestamps may delay flush in S...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1197 @JonZeolla That error is a pre-existing condition that is tracked here;[ METRON-1810](https://issues.apache.org/jira/browse/METRON-1810). I am not sure what the root cause is. ---
[GitHub] metron issue #1237: METRON-1825: Upgrade bro to 2.5.5
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1237 Have you spun this up in Full Dev and tested? ---
[jira] [Commented] (METRON-1825) Upgrade bro to 2.5.5
[ https://issues.apache.org/jira/browse/METRON-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686950#comment-16686950 ] ASF GitHub Bot commented on METRON-1825: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1237 Have you spun this up in Full Dev and tested? > Upgrade bro to 2.5.5 > > > Key: METRON-1825 > URL: https://issues.apache.org/jira/browse/METRON-1825 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1872) Move rat plugin away from snapshot version
[ https://issues.apache.org/jira/browse/METRON-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686949#comment-16686949 ] ASF GitHub Bot commented on METRON-1872: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1264 +1 Thanks @justinleet > Move rat plugin away from snapshot version > -- > > Key: METRON-1872 > URL: https://issues.apache.org/jira/browse/METRON-1872 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Apache Rat 0.13 got released. We were on the SNAPSHOT version before to take > care of a bug. Now that the actual release has happened, we should switch to > that and repo that was being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1264: METRON-1872: Move rat plugin away from snapshot version
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1264 +1 Thanks @justinleet ---
[GitHub] metron issue #1242: METRON-1834: Migrate Elasticsearch from TransportClient ...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1242 Ship it! ---
[jira] [Commented] (METRON-1870) Intermittent Stellar REST test failures
[
https://issues.apache.org/jira/browse/METRON-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686930#comment-16686930
]
ASF GitHub Bot commented on METRON-1870:
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/1263
> Intermittent Stellar REST test failures
> ---
>
> Key: METRON-1870
> URL: https://issues.apache.org/jira/browse/METRON-1870
> Project: Metron
> Issue Type: Bug
>Reporter: Ryan Merriman
>Priority: Major
>
> There have been a few intermittent travis failures reported for the Stellar
> RestGet function:
> restGetShouldTimeoutWithSuppliedTimeout(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
> Time elapsed: 0.336 sec <<< ERROR!
> org.apache.metron.stellar.dsl.ParseException: Unable to parse:
> REST_GET('http://localhost:41396/get', \{ "timeout": 1 } ) due to: Connection
> is not open
> and:
> restGetShouldHandleIOException(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
> Time elapsed: 0.314 sec <<< FAILURE! java.lang.AssertionError: expected
> null, but was:<\{get=success}>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1870) Intermittent Stellar REST test failures
[
https://issues.apache.org/jira/browse/METRON-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686860#comment-16686860
]
ASF GitHub Bot commented on METRON-1870:
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1263
+1 pending successful Travis run. Thanks
> Intermittent Stellar REST test failures
> ---
>
> Key: METRON-1870
> URL: https://issues.apache.org/jira/browse/METRON-1870
> Project: Metron
> Issue Type: Bug
>Reporter: Ryan Merriman
>Priority: Major
>
> There have been a few intermittent travis failures reported for the Stellar
> RestGet function:
> restGetShouldTimeoutWithSuppliedTimeout(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
> Time elapsed: 0.336 sec <<< ERROR!
> org.apache.metron.stellar.dsl.ParseException: Unable to parse:
> REST_GET('http://localhost:41396/get', \{ "timeout": 1 } ) due to: Connection
> is not open
> and:
> restGetShouldHandleIOException(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
> Time elapsed: 0.314 sec <<< FAILURE! java.lang.AssertionError: expected
> null, but was:<\{get=success}>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1834) Migrate Elasticsearch from TransportClient to new Java REST API
[ https://issues.apache.org/jira/browse/METRON-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686861#comment-16686861 ] ASF GitHub Bot commented on METRON-1834: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1242 I'm about to merge this and wanted to give some added feedback. @anandsubbu and I made some additional changes to the tuning options, specifically setting `topology.ackers.executors` equal to the number of inbound Kafka topic partitions, and we were able to see runs with no failures. > Migrate Elasticsearch from TransportClient to new Java REST API > --- > > Key: METRON-1834 > URL: https://issues.apache.org/jira/browse/METRON-1834 > Project: Metron > Issue Type: Improvement >Reporter: Michael Miklavcic >Assignee: Michael Miklavcic >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1242: METRON-1834: Migrate Elasticsearch from TransportClient ...
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1242 I'm about to merge this and wanted to give some added feedback. @anandsubbu and I made some additional changes to the tuning options, specifically setting `topology.ackers.executors` equal to the number of inbound Kafka topic partitions, and we were able to see runs with no failures. ---
[GitHub] metron issue #1263: METRON-1870: Intermittent Stellar REST test failures
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1263 +1 pending successful Travis run. Thanks ---
[jira] [Commented] (METRON-1872) Move rat plugin away from snapshot version
[ https://issues.apache.org/jira/browse/METRON-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686829#comment-16686829 ] ASF GitHub Bot commented on METRON-1872: GitHub user justinleet opened a pull request: https://github.com/apache/metron/pull/1264 METRON-1872: Move rat plugin away from snapshot version ## Contributor Comments Removes the Apache dev snapshot plugin repo, and bumps the rat version to 0.13 instead of 0.13-SNAPSHOT ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/justinleet/metron rat_upgrade Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1264.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1264 commit 4b618f153b04d879a2e2f11960421d73a4853e60 Author: justinjleet Date: 2018-11-14T16:29:26Z remove rat snapshot and repository > Move rat plugin away from snapshot version > -- > > Key: METRON-1872 > URL: https://issues.apache.org/jira/browse/METRON-1872 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Apache Rat 0.13 got released. We were on the SNAPSHOT version before to take > care of a bug. Now that the actual release has happened, we should switch to > that and repo that was being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1264: METRON-1872: Move rat plugin away from snapshot v...
GitHub user justinleet opened a pull request: https://github.com/apache/metron/pull/1264 METRON-1872: Move rat plugin away from snapshot version ## Contributor Comments Removes the Apache dev snapshot plugin repo, and bumps the rat version to 0.13 instead of 0.13-SNAPSHOT ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/justinleet/metron rat_upgrade Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1264.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1264 commit 4b618f153b04d879a2e2f11960421d73a4853e60 Author: justinjleet Date: 2018-11-14T16:29:26Z remove rat snapshot and repository ---
[jira] [Created] (METRON-1872) Move rat plugin away from snapshot version
Justin Leet created METRON-1872: --- Summary: Move rat plugin away from snapshot version Key: METRON-1872 URL: https://issues.apache.org/jira/browse/METRON-1872 Project: Metron Issue Type: Improvement Reporter: Justin Leet Assignee: Justin Leet Apache Rat 0.13 got released. We were on the SNAPSHOT version before to take care of a bug. Now that the actual release has happened, we should switch to that and repo that was being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1263: METRON-1870: Intermittent Stellar REST test failu...
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/1263
METRON-1870: Intermittent Stellar REST test failures
## Contributor Comments
This PR attempts to fix the intermittent test failures reported in the Jira.
For this failure:
```
restGetShouldTimeoutWithSuppliedTimeout(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
Time elapsed: 0.336 sec <<< ERROR!
org.apache.metron.stellar.dsl.ParseException: Unable to parse:
REST_GET('http://localhost:41396/get', { "timeout": 1 } ) due to: Connection is
not open
```
I believe the root cause is that we're getting an IllegalStateException in
some cases where we're expecting an IOException. The HttpGet.abort() method is
called within an ScheduledExecutorService so it's unpredictable when exactly
the request is aborted. I changed the caught exception to be general so
hopefully this gives us the result we expect.
For this failure:
```
restGetShouldHandleIOException(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
Time elapsed: 0.314 sec <<< FAILURE! java.lang.AssertionError: expected null,
but was:<{get=success}>
```
An IOException is being simulated by setting the socket timeout really low
and executing a request against the mock server. Looks like this is not enough
to make it consistently fail. I changed the test to spy on the doGet method
and throw an IOException so that it fails every time.
I was not able to reproduce these problems locally but I think they will
resolve the issues.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/merrimanr/incubator-metron METRON-1870
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1263.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1263
commit b2dd4232e997eef23d0ba7350caa49d52ce5e929
Author: merrimanr
Date: 2018-11-14T15:09:21Z
initial commit
---
[jira] [Commented] (METRON-1870) Intermittent Stellar REST test failures
[
https://issues.apache.org/jira/browse/METRON-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686826#comment-16686826
]
ASF GitHub Bot commented on METRON-1870:
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/1263
METRON-1870: Intermittent Stellar REST test failures
## Contributor Comments
This PR attempts to fix the intermittent test failures reported in the Jira.
For this failure:
```
restGetShouldTimeoutWithSuppliedTimeout(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
Time elapsed: 0.336 sec <<< ERROR!
org.apache.metron.stellar.dsl.ParseException: Unable to parse:
REST_GET('http://localhost:41396/get', { "timeout": 1 } ) due to: Connection is
not open
```
I believe the root cause is that we're getting an IllegalStateException in
some cases where we're expecting an IOException. The HttpGet.abort() method is
called within an ScheduledExecutorService so it's unpredictable when exactly
the request is aborted. I changed the caught exception to be general so
hopefully this gives us the result we expect.
For this failure:
```
restGetShouldHandleIOException(org.apache.metron.stellar.dsl.functions.RestFunctionsTest)
Time elapsed: 0.314 sec <<< FAILURE! java.lang.AssertionError: expected null,
but was:<{get=success}>
```
An IOException is being simulated by setting the socket timeout really low
and executing a request against the mock server. Looks like this is not enough
to make it consistently fail. I changed the test to spy on the doGet method
and throw an IOException so that it fails every time.
I was not able to reproduce these problems locally but I think they will
resolve the issues.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/merrimanr/incubator-metron METRON-1870
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1263.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This
[jira] [Commented] (METRON-1871) Cannot Run Elasticsearch Integration Tests in IDE
[ https://issues.apache.org/jira/browse/METRON-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686690#comment-16686690 ] ASF GitHub Bot commented on METRON-1871: Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1262 I find myself commenting these out every time I run ES integration tests in my IDE. +1 > Cannot Run Elasticsearch Integration Tests in IDE > - > > Key: METRON-1871 > URL: https://issues.apache.org/jira/browse/METRON-1871 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > The Elasticsearch integration tests run successfully from Maven, but they do > not run successfully when executed from within an IDE like Intellij. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1871) Cannot Run Elasticsearch Integration Tests in IDE
[ https://issues.apache.org/jira/browse/METRON-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686672#comment-16686672 ] ASF GitHub Bot commented on METRON-1871: GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/1262 METRON-1871 Cannot Run Elasticsearch Integration Tests in IDE There are some log4j dependencies in the metron-elasticsearch that cause problems when attempting to run the Elasticsearch integration tests from an IDE like Intellij. These dependencies are unnecessary and should be removed for that reason alone. But additionally this fix lets you run the tests from Intellij. This was previously included in #1218 before that was backed out of master. ## Testing 1. Load up the code in an IDE like IntelliJ. 1. Execute the tests in `metron-platform/metron-elasticsearch/src/tests`. 1. The tests should all run successfully. ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1871 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1262 commit f101ab74fe23c5bfbbc822e787c43984d0dbef8f Author: Nick Allen Date: 2018-11-14T15:18:09Z METRON-1871 Cannot Run Elasticsearch Integration Tests in IDE > Cannot Run Elasticsearch Integration Tests in IDE > - > > Key: METRON-1871 > URL: https://issues.apache.org/jira/browse/METRON-1871 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > > The Elasticsearch integration tests run successfully from Maven, but they do > not run successfully when executed from within an IDE like Intellij. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (METRON-1871) Cannot Run Elasticsearch Integration Tests in IDE
[ https://issues.apache.org/jira/browse/METRON-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Allen updated METRON-1871: --- Attachment: Screen Shot 2018-11-14 at 10.26.18 AM.png > Cannot Run Elasticsearch Integration Tests in IDE > - > > Key: METRON-1871 > URL: https://issues.apache.org/jira/browse/METRON-1871 > Project: Metron > Issue Type: Bug >Reporter: Nick Allen >Assignee: Nick Allen >Priority: Major > Attachments: Screen Shot 2018-11-14 at 10.26.18 AM.png > > > The Elasticsearch integration tests run successfully from Maven, but they do > not run successfully when executed from within an IDE like Intellij. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686696#comment-16686696 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 anyone have any ideas of the best way to time these things? > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 anyone have any ideas of the best way to time these things? ---
[jira] [Commented] (METRON-1868) metron-committer-common incorrectly checking REPO_NAME
[ https://issues.apache.org/jira/browse/METRON-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686691#comment-16686691 ] ASF GitHub Bot commented on METRON-1868: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1260 > metron-committer-common incorrectly checking REPO_NAME > -- > > Key: METRON-1868 > URL: https://issues.apache.org/jira/browse/METRON-1868 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Minor > > It appears that REPO_NAME is not being correctly checked. > https://github.com/apache/metron/blob/42068d95192e7346bf0179399193a6afe01d9925/dev-utilities/committer-utils/metron-committer-common#L177 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686693#comment-16686693 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 We could also use more tags, for example I may want to skip building the java, but not skip building the RPMs. Think of a dev flow -> I make my change, run my local tests and want to spin up full dev. It is already built, but needs the rpms, I should be able to make ansible skip the compile/package of java and still do the rpms/debs > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1262: METRON-1871 Cannot Run Elasticsearch Integration Tests i...
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/1262 I find myself commenting these out every time I run ES integration tests in my IDE. +1 ---
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 We could also use more tags, for example I may want to skip building the java, but not skip building the RPMs. Think of a dev flow -> I make my change, run my local tests and want to spin up full dev. It is already built, but needs the rpms, I should be able to make ansible skip the compile/package of java and still do the rpms/debs ---
[GitHub] metron pull request #1260: METRON-1868: metron-committer-common incorrectly ...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1260 ---
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686683#comment-16686683 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @nickwallen That is an option, but not something I would pick as the goal from the outset if you know what I mean. > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @nickwallen That is an option, but not something I would pick as the goal from the outset if you know what I mean. ---
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686670#comment-16686670 ] ASF GitHub Bot commented on METRON-1860: Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1261 I'm going to take a stab at a further look next week. For now I gave it a quick run-up and it was successful. > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #1262: METRON-1871 Cannot Run Elasticsearch Integration ...
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/1262 METRON-1871 Cannot Run Elasticsearch Integration Tests in IDE There are some log4j dependencies in the metron-elasticsearch that cause problems when attempting to run the Elasticsearch integration tests from an IDE like Intellij. These dependencies are unnecessary and should be removed for that reason alone. But additionally this fix lets you run the tests from Intellij. This was previously included in #1218 before that was backed out of master. ## Testing 1. Load up the code in an IDE like IntelliJ. 1. Execute the tests in `metron-platform/metron-elasticsearch/src/tests`. 1. The tests should all run successfully. ## Pull Request Checklist - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? You can merge this pull request into a Git repository by running: $ git pull https://github.com/nickwallen/metron METRON-1871 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1262 commit f101ab74fe23c5bfbbc822e787c43984d0dbef8f Author: Nick Allen Date: 2018-11-14T15:18:09Z METRON-1871 Cannot Run Elasticsearch Integration Tests in IDE ---
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1261 I'm going to take a stab at a further look next week. For now I gave it a quick run-up and it was successful. ---
[jira] [Commented] (METRON-1815) Separate metron-parsers into metron-parsers-common and metron-parsers-storm
[ https://issues.apache.org/jira/browse/METRON-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686668#comment-16686668 ] ASF GitHub Bot commented on METRON-1815: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1249 @justinleet I'm fine with that > Separate metron-parsers into metron-parsers-common and metron-parsers-storm > --- > > Key: METRON-1815 > URL: https://issues.apache.org/jira/browse/METRON-1815 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > In order to expose our parsers to 3rd party components (e.g. the discussions > on NiFi and potentially other platforms like Spark), we should > separate the storm-bits into its own project. The metron-parsers-common > project should contain only parser-oriented code, whereas the > metron-parsers-storm project should contain the storm specific code > (e.g. the parser bolt). > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1249: METRON-1815: Separate metron-parsers into metron-parsers...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1249 @justinleet I'm fine with that ---
[jira] [Created] (METRON-1871) Cannot Run Elasticsearch Integration Tests in IDE
Nick Allen created METRON-1871: -- Summary: Cannot Run Elasticsearch Integration Tests in IDE Key: METRON-1871 URL: https://issues.apache.org/jira/browse/METRON-1871 Project: Metron Issue Type: Bug Reporter: Nick Allen Assignee: Nick Allen The Elasticsearch integration tests run successfully from Maven, but they do not run successfully when executed from within an IDE like Intellij. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1249: METRON-1815: Separate metron-parsers into metron-parsers...
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1249 I don't want to bog this down too much. I'm good with either the recent rev you proposed or simply adding the appropriate notes in the pom internals and dev docs. Chefs choice. ---
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686653#comment-16686653 ] ASF GitHub Bot commented on METRON-1860: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1261 How does the build time compare to the current approach (`mvn clean install -DskipTests`)? Does it change at all since it is now run in a Docker container? How does the time it takes to get Full Dev running compare (whatever is equivalent to `vagrant up`)? > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686649#comment-16686649 ] ASF GitHub Bot commented on METRON-1860: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1261 If we go this approach, why not just replace the existing "Full Dev" environments (both centos and ubuntu) rather than add new environments to support, test, and keep in-sync? > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1261 If we go this approach, why not just replace the existing "Full Dev" environments (both centos and ubuntu) rather than add new environments to support, test, and keep in-sync? ---
[GitHub] metron issue #1249: METRON-1815: Separate metron-parsers into metron-parsers...
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1249 It's a bit inelegant, but what if we do something like: ``` âââ metron-parsing âââ metron-parsers âââ metron-parsers-common âââ metron-parsing-framework-storm ``` Later we could add metron-parsing-framework-common, etc. There's still the slight difference between metron-parsing and metron-parsers, but I think since metron-parsing is the overall parent, and metron-parsers is just the parsers themselves, it's fairly clear. Otherwise, someone might need to break out a thesaurus, because I'm awful at naming things ---
[jira] [Commented] (METRON-1815) Separate metron-parsers into metron-parsers-common and metron-parsers-storm
[ https://issues.apache.org/jira/browse/METRON-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686642#comment-16686642 ] ASF GitHub Bot commented on METRON-1815: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1249 It's a bit inelegant, but what if we do something like: ``` └── metron-parsing ├── metron-parsers ├── metron-parsers-common └── metron-parsing-framework-storm ``` Later we could add metron-parsing-framework-common, etc. There's still the slight difference between metron-parsing and metron-parsers, but I think since metron-parsing is the overall parent, and metron-parsers is just the parsers themselves, it's fairly clear. Otherwise, someone might need to break out a thesaurus, because I'm awful at naming things > Separate metron-parsers into metron-parsers-common and metron-parsers-storm > --- > > Key: METRON-1815 > URL: https://issues.apache.org/jira/browse/METRON-1815 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > In order to expose our parsers to 3rd party components (e.g. the discussions > on NiFi and potentially other platforms like Spark), we should > separate the storm-bits into its own project. The metron-parsers-common > project should contain only parser-oriented code, whereas the > metron-parsers-storm project should contain the storm specific code > (e.g. the parser bolt). > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686616#comment-16686616 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @mmiklavc That is basically correct. Except that the ansible version is 2.5, since it only applies to this build, and allows for the yaml log formatting. Also, in the latest version, the ansible once again does the clean and build as opposed to the script. I had a lot of problems getting the c++ picked up from ansible and moved the build out of it for the time, but the idea was always to have ansible run the metron_build, and that has returned. The reasoning for the prompts to build the vagrant box and the docker -> - if you are using this during development, IE> we are working ON ansible or ON docker, you may fail in the docker or ansible stage without modifying the vm, and thus not need to vagrant up again. Likewise, you may not need to rebuild the docker machine if you have not made changes, _or_ you may in fact need to. I added these flags as I developed. > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1868) metron-committer-common incorrectly checking REPO_NAME
[ https://issues.apache.org/jira/browse/METRON-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686631#comment-16686631 ] ASF GitHub Bot commented on METRON-1868: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1260 +1, sorry I missed this > metron-committer-common incorrectly checking REPO_NAME > -- > > Key: METRON-1868 > URL: https://issues.apache.org/jira/browse/METRON-1868 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Minor > > It appears that REPO_NAME is not being correctly checked. > https://github.com/apache/metron/blob/42068d95192e7346bf0179399193a6afe01d9925/dev-utilities/committer-utils/metron-committer-common#L177 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1260: METRON-1868: metron-committer-common incorrectly checkin...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1260 +1, sorry I missed this ---
[jira] [Commented] (METRON-1868) metron-committer-common incorrectly checking REPO_NAME
[ https://issues.apache.org/jira/browse/METRON-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686625#comment-16686625 ] ASF GitHub Bot commented on METRON-1868: Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1260 If you ran `prepare_commit` and selected the metron-bro-plugin-kafka repo, when you get to `select_code` it would default to metron. > metron-committer-common incorrectly checking REPO_NAME > -- > > Key: METRON-1868 > URL: https://issues.apache.org/jira/browse/METRON-1868 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Minor > > It appears that REPO_NAME is not being correctly checked. > https://github.com/apache/metron/blob/42068d95192e7346bf0179399193a6afe01d9925/dev-utilities/committer-utils/metron-committer-common#L177 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686618#comment-16686618 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 The integration test failure has to do with the profiler tests and seem unrelated. > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @mmiklavc That is basically correct. Except that the ansible version is 2.5, since it only applies to this build, and allows for the yaml log formatting. Also, in the latest version, the ansible once again does the clean and build as opposed to the script. I had a lot of problems getting the c++ picked up from ansible and moved the build out of it for the time, but the idea was always to have ansible run the metron_build, and that has returned. The reasoning for the prompts to build the vagrant box and the docker -> - if you are using this during development, IE> we are working ON ansible or ON docker, you may fail in the docker or ansible stage without modifying the vm, and thus not need to vagrant up again. Likewise, you may not need to rebuild the docker machine if you have not made changes, _or_ you may in fact need to. I added these flags as I developed. ---
[GitHub] metron issue #1261: METRON-1860 [WIP] new developer option for ansible in do...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @nickwallen I did not think of that. I was improving the process that stands today. I think in a world where the posted image exists, we would still want the ability to try the latest ( to verify a fix pre-release etc ). ---
[jira] [Commented] (METRON-1860) Improve full dev experience by using docker and vagrant together
[ https://issues.apache.org/jira/browse/METRON-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686606#comment-16686606 ] ASF GitHub Bot commented on METRON-1860: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1261 @nickwallen I did not think of that. I was improving the process that stands today. I think in a world where the posted image exists, we would still want the ability to try the latest ( to verify a fix pre-release etc ). > Improve full dev experience by using docker and vagrant together > > > Key: METRON-1860 > URL: https://issues.apache.org/jira/browse/METRON-1860 > Project: Metron > Issue Type: Improvement >Reporter: Otto Fowler >Assignee: Otto Fowler >Priority: Major > > Many new users have trouble getting full dev to run correctly. Setting up > the environment to build metron and run the ansible is often a problem. > We already have a docker container for fully building metron, that could also > be used for running ansible. > The idea to explore is the following: > # run vagrant up, but without ansible, just stand up the base image > # run docker to run the ansible playbooks against the vagrant machine > # ??? > # profit -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1868) metron-committer-common incorrectly checking REPO_NAME
[ https://issues.apache.org/jira/browse/METRON-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686603#comment-16686603 ] ASF GitHub Bot commented on METRON-1868: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1260 What did you seen before that is fix now? > metron-committer-common incorrectly checking REPO_NAME > -- > > Key: METRON-1868 > URL: https://issues.apache.org/jira/browse/METRON-1868 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Minor > > It appears that REPO_NAME is not being correctly checked. > https://github.com/apache/metron/blob/42068d95192e7346bf0179399193a6afe01d9925/dev-utilities/committer-utils/metron-committer-common#L177 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1827) Update librdkafka in metron-bro-plugin-kafka
[ https://issues.apache.org/jira/browse/METRON-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686560#comment-16686560 ] ASF GitHub Bot commented on METRON-1827: Github user asfgit closed the pull request at: https://github.com/apache/metron-bro-plugin-kafka/pull/13 > Update librdkafka in metron-bro-plugin-kafka > > > Key: METRON-1827 > URL: https://issues.apache.org/jira/browse/METRON-1827 > Project: Metron > Issue Type: Task >Reporter: Jon Zeolla >Assignee: Jon Zeolla >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron issue #1249: METRON-1815: Separate metron-parsers into metron-parsers...
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1249 I'm ok with changing parsing to something else, just not parsers. ---
[jira] [Commented] (METRON-1815) Separate metron-parsers into metron-parsers-common and metron-parsers-storm
[ https://issues.apache.org/jira/browse/METRON-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686414#comment-16686414 ] ASF GitHub Bot commented on METRON-1815: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1249 I'm ok with changing parsing to something else, just not parsers. > Separate metron-parsers into metron-parsers-common and metron-parsers-storm > --- > > Key: METRON-1815 > URL: https://issues.apache.org/jira/browse/METRON-1815 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > In order to expose our parsers to 3rd party components (e.g. the discussions > on NiFi and potentially other platforms like Spark), we should > separate the storm-bits into its own project. The metron-parsers-common > project should contain only parser-oriented code, whereas the > metron-parsers-storm project should contain the storm specific code > (e.g. the parser bolt). > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
