[jira] [Commented] (METRON-2242) Improve output of validate-jira-for-release script

2019-08-29 Thread Justin Leet (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918954#comment-16918954
 ] 

Justin Leet commented on METRON-2242:
-

Yeah, that's a pretty good summary. This doesn't get run very often, so I'd 
also say that it's not something that needs anything fancy.  It just makes life 
a bit easier.  Anything too complicated is just overkill.

> Improve output of validate-jira-for-release script
> --
>
> Key: METRON-2242
> URL: https://issues.apache.org/jira/browse/METRON-2242
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Priority: Minor
>
> Right now, the validate-jira-for-release outputs the data in a whitespace 
> delimited format. Because values can have whitespace (e.g. "In Progress" or 
> "Justin Leet"), this means that easily sorting by contributor doesn't work. 
> This is important to contacting contributors to update Jira at release time.
> Ideal output would be geared to the release manager being able to easily 
> contact the appropriate people to update their tickets.  Speaking personally, 
> just making the output work with sort is sufficient, but it may be a better 
> default order. Additionally, if it's possible to pull out the contributor's 
> email (e.g. from the relevant commit), that would also make things easier.
> If people want to rearrange the columns or something clever (e.g. have first 
> column be name, so people can easily find themselves in a general list 
> blast), that's another idea.
> Finally, having a more obvious (or default) ability to just pass in version 
> "Next + 1" might make things a bit more clear, but it's not imperative.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[GitHub] [metron] nickwallen commented on issue #1497: METRON-2231 Revert METRON-2175, METRON-2176, METRON-2177 in HDP 3.1 upgrade feature branch

2019-08-29 Thread GitBox
nickwallen commented on issue #1497: METRON-2231 Revert METRON-2175, 
METRON-2176, METRON-2177 in HDP 3.1 upgrade feature branch
URL: https://github.com/apache/metron/pull/1497#issuecomment-526351500
 
 
   I have included the patch for #1499 as this was preventing the CI build from 
completing successfully for this PR.  #1499 should be merged prior to this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Commented] (METRON-2242) Improve output of validate-jira-for-release script

2019-08-29 Thread Otto Fowler (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918930#comment-16918930
 ] 

Otto Fowler commented on METRON-2242:
-

so:

1.  The output should be in a machine friendly format
2. A second script that formats that output *could* be used instead 
3. The script input should have options
4. maybe extend so that you can choose to send people referenced an email from 
template etc

> Improve output of validate-jira-for-release script
> --
>
> Key: METRON-2242
> URL: https://issues.apache.org/jira/browse/METRON-2242
> Project: Metron
>  Issue Type: Improvement
>Reporter: Justin Leet
>Priority: Minor
>
> Right now, the validate-jira-for-release outputs the data in a whitespace 
> delimited format. Because values can have whitespace (e.g. "In Progress" or 
> "Justin Leet"), this means that easily sorting by contributor doesn't work. 
> This is important to contacting contributors to update Jira at release time.
> Ideal output would be geared to the release manager being able to easily 
> contact the appropriate people to update their tickets.  Speaking personally, 
> just making the output work with sort is sufficient, but it may be a better 
> default order. Additionally, if it's possible to pull out the contributor's 
> email (e.g. from the relevant commit), that would also make things easier.
> If people want to rearrange the columns or something clever (e.g. have first 
> column be name, so people can easily find themselves in a general list 
> blast), that's another idea.
> Finally, having a more obvious (or default) ability to just pass in version 
> "Next + 1" might make things a bit more clear, but it's not imperative.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[GitHub] [metron] nickwallen opened a new pull request #1499: METRON-2241 Profiler Integration Test Fails After Storm 1.2.1 Upgrade

2019-08-29 Thread GitBox
nickwallen opened a new pull request #1499: METRON-2241 Profiler Integration 
Test Fails After Storm 1.2.1 Upgrade
URL: https://github.com/apache/metron/pull/1499
 
 
   After the upgrade to Storm 1.2.1, there have been intermittent failures in 
the Storm Profiler integration tests.  
   
   This is due to the way that the storm-kafka integration handles the Kafka 
consumer when a topology is stopped.  This integration test is the only one 
where multiple topologies are stopped and started during the test lifecycle.  
This is likely why it is only a problem for this test case. 
   
   This was identified as a bug in 
[STORM-3013](https://issues.apache.org/jira/browse/STORM-3013) and has been 
patched in storm-kafka version 1.2.3. This PR ensures that we are building 
against a build where this bug has been fixed.
   
   
   ## Pull Request Checklist
   
   Thank you for submitting a contribution to Apache Metron.  
   Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
   Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  
   
   
   In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be 
created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
   - [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   
   ### For code changes:
   - [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
   - [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
   - [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
 ```
 mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
 ```
   
   - [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] Have you verified the basic functionality of the build by building and 
running locally with Vagrant full-dev environment or the equivalent?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered by building and verifying the site-book? If not then run the 
following commands and the verify changes via 
`site-book/target/site/index.html`:
   
 ```
 cd site-book
 mvn site
 ```
   
   - [ ] Have you ensured that any documentation diagrams have been updated, 
along with their source files, using [draw.io](https://www.draw.io/)? See 
[Metron Development 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Development+Guidelines)
 for instructions.
   
    Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   It is also recommended that [travis-ci](https://travis-ci.org) is set up for 
your personal repository such that your branches are built there before 
submitting a pull request.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (METRON-2242) Improve output of validate-jira-for-release script

2019-08-29 Thread Justin Leet (Jira)
Justin Leet created METRON-2242:
---

 Summary: Improve output of validate-jira-for-release script
 Key: METRON-2242
 URL: https://issues.apache.org/jira/browse/METRON-2242
 Project: Metron
  Issue Type: Improvement
Reporter: Justin Leet


Right now, the validate-jira-for-release outputs the data in a whitespace 
delimited format. Because values can have whitespace (e.g. "In Progress" or 
"Justin Leet"), this means that easily sorting by contributor doesn't work. 
This is important to contacting contributors to update Jira at release time.

Ideal output would be geared to the release manager being able to easily 
contact the appropriate people to update their tickets.  Speaking personally, 
just making the output work with sort is sufficient, but it may be a better 
default order. Additionally, if it's possible to pull out the contributor's 
email (e.g. from the relevant commit), that would also make things easier.

If people want to rearrange the columns or something clever (e.g. have first 
column be name, so people can easily find themselves in a general list blast), 
that's another idea.

Finally, having a more obvious (or default) ability to just pass in version 
"Next + 1" might make things a bit more clear, but it's not imperative.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Updated] (METRON-2111) Update public web site to point at 0.7.1 new release

2019-08-29 Thread Justin Leet (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justin Leet updated METRON-2111:

Fix Version/s: Next + 1

> Update public web site to point at 0.7.1 new release
> 
>
> Key: METRON-2111
> URL: https://issues.apache.org/jira/browse/METRON-2111
> Project: Metron
>  Issue Type: Task
>Reporter: Justin Leet
>Assignee: Justin Leet
>Priority: Major
> Fix For: Next + 1
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Update the website to reflect the release



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Updated] (METRON-2241) Profiler Integration Test Fails After Storm 1.2.1 Upgrade

2019-08-29 Thread Nick Allen (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Allen updated METRON-2241:
---
Description: 
The ProfilerIntegrationTest fails after the upgrade to Storm 1.2.1. This is the 
only integration test that starts/stops multiple topologies during testing.
{code:java}
---
 T E S T S
---
Running org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest
2019-08-29 16:35:49 ERROR util:0 - Async loop died!
java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has 
already been closed.
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
at 
org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: This consumer has already been 
closed.
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
at 
org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
at 
org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
at clojure.core$map$fn__4553.invoke(core.clj:2622)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:507)
at clojure.core$seq__4128.invoke(core.clj:137)
at clojure.core$filter$fn__4580.invoke(core.clj:2679)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.next(RT.java:674)
at clojure.core$next__4112.invoke(core.clj:64)
at clojure.core.protocols$fn__6523.invoke(protocols.clj:170)
at 
clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
at 
clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6519)
at clojure.core$into.invoke(core.clj:6600)
at 
org.apache.storm.daemon.executor$metrics_tick.invoke(executor.clj:349)
at 
org.apache.storm.daemon.executor$fn__4975$tuple_action_fn__4981.invoke(executor.clj:522)
at 
org.apache.storm.daemon.executor$mk_task_receiver$fn__4964.invoke(executor.clj:471)
at 
org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
... 7 more
2019-08-29 16:35:49 ERROR executor:0 - 
java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has 
already been closed.
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
at 
org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: This consumer has already been 
closed.
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
at 
org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
at 
org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
at clojure.core$map$fn__4553.invoke(core.clj:2622)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at 

[jira] [Created] (METRON-2241) Profiler Integration Test Fails After Storm 1.2.1 Upgrade

2019-08-29 Thread Nick Allen (Jira)
Nick Allen created METRON-2241:
--

 Summary: Profiler Integration Test Fails After Storm 1.2.1 Upgrade
 Key: METRON-2241
 URL: https://issues.apache.org/jira/browse/METRON-2241
 Project: Metron
  Issue Type: Sub-task
Reporter: Nick Allen
Assignee: Nick Allen


The ProfilerIntegrationTest fails after the upgrade to Storm 1.2.1. This is the 
only integration test that starts/stops multiple topologies during testing.
{code:java}
---
 T E S T S
---
Running org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest
2019-08-29 16:35:49 ERROR util:0 - Async loop died!
java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has 
already been closed.
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
at 
org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: This consumer has already been 
closed.
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
at 
org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
at 
org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
at clojure.core$map$fn__4553.invoke(core.clj:2622)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:507)
at clojure.core$seq__4128.invoke(core.clj:137)
at clojure.core$filter$fn__4580.invoke(core.clj:2679)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.next(RT.java:674)
at clojure.core$next__4112.invoke(core.clj:64)
at clojure.core.protocols$fn__6523.invoke(protocols.clj:170)
at 
clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
at 
clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6519)
at clojure.core$into.invoke(core.clj:6600)
at 
org.apache.storm.daemon.executor$metrics_tick.invoke(executor.clj:349)
at 
org.apache.storm.daemon.executor$fn__4975$tuple_action_fn__4981.invoke(executor.clj:522)
at 
org.apache.storm.daemon.executor$mk_task_receiver$fn__4964.invoke(executor.clj:471)
at 
org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
... 7 more
2019-08-29 16:35:49 ERROR executor:0 - 
java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has 
already been closed.
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
at 
org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
at 
org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: This consumer has already been 
closed.
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
at 
org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
at 

[jira] [Comment Edited] (METRON-2232) Upgrade Hadoop

2019-08-29 Thread Ryan Merriman (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918748#comment-16918748
 ] 

Ryan Merriman edited comment on METRON-2232 at 8/29/19 4:11 PM:


I started working through this and have a branch here:  
[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1].  The 
commit at 
[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]
 has the specific changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
error and attempted to solve it by copying the source for 
FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 


was (Author: rmerriman):
I started working through this and have a branch 
[commit|[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1]]
 .  This 
[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]]
 has the changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
error and attempted to solve it by copying the source for 
FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 

> Upgrade Hadoop
> --
>
> Key: METRON-2232
> URL: https://issues.apache.org/jira/browse/METRON-2232
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Comment Edited] (METRON-2232) Upgrade Hadoop

2019-08-29 Thread Ryan Merriman (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918748#comment-16918748
 ] 

Ryan Merriman edited comment on METRON-2232 at 8/29/19 4:12 PM:


I started working through this and have a branch here:  
[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1].  The 
commit at 
[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]
 has the specific changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
error (https://issues.apache.org/jira/browse/HBASE-22394) and attempted to 
solve it by copying the source for FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 


was (Author: rmerriman):
I started working through this and have a branch here:  
[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1].  The 
commit at 
[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]
 has the specific changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
error and attempted to solve it by copying the source for 
FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 

> Upgrade Hadoop
> --
>
> Key: METRON-2232
> URL: https://issues.apache.org/jira/browse/METRON-2232
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[GitHub] [metron] mmiklavc commented on issue #1495: METRON-2211: [UI] Alerts UI should optionally render timestamp in local time

2019-08-29 Thread GitBox
mmiklavc commented on issue #1495: METRON-2211: [UI] Alerts UI should 
optionally render timestamp in local time
URL: https://github.com/apache/metron/pull/1495#issuecomment-526269692
 
 
   @sardell you must be making people happy with this one - I think this is the 
first time I've ever seen a heart emoji on a PR


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Comment Edited] (METRON-2232) Upgrade Hadoop

2019-08-29 Thread Ryan Merriman (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918748#comment-16918748
 ] 

Ryan Merriman edited comment on METRON-2232 at 8/29/19 4:10 PM:


I started working through this and have a branch 
[commit|[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1]]
 .  This 
[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]]
 has the changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
error and attempted to solve it by copying the source for 
FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 


was (Author: rmerriman):
I started working through this and have a branch 
[here|[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1]]. 
 This 
[commit|[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]]
 has the changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
[error|https://issues.apache.org/jira/browse/HBASE-22394] and attempted to 
solve it by copying the source for FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 

> Upgrade Hadoop
> --
>
> Key: METRON-2232
> URL: https://issues.apache.org/jira/browse/METRON-2232
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (METRON-2232) Upgrade Hadoop

2019-08-29 Thread Ryan Merriman (Jira)


[ 
https://issues.apache.org/jira/browse/METRON-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918748#comment-16918748
 ] 

Ryan Merriman commented on METRON-2232:
---

I started working through this and have a branch 
[here|[https://github.com/merrimanr/incubator-metron/tree/hadoop-upgrade-3.1]]. 
 This 
[commit|[https://github.com/merrimanr/incubator-metron/commit/0446bbb548b88de9df7289b526e2f3c86ff3f4c3]]
 has the changes I made.  

The changes mostly include reorganizing some Maven dependencies and a couple 
code changes to match the newer Hadoop version.  I initially ran into this 
[error|https://issues.apache.org/jira/browse/HBASE-22394] and attempted to 
solve it by copying the source for FanOutOneBlockAsyncDFSOutputHelper.java and 
FanOutOneBlockAsyncDFSOutputSaslHelper.java into the test classpath of affected 
modules.  This worked for metron-hbase-common but I have not been able to get 
it to work with metron-data-management.  Strangely enough my machine reboots 
every time I run tests for that module.  Curious if this happens to others or 
if tests pass.  There are also some test failures in MaasIntegrationTest in the 
metron-maas-service module.  I don't know that module well so I have not been 
able to troubleshoot or find a fix.

This work depends on the HBase upgrade so I have not been able to test this in 
full dev yet because there is not a stable branch that has both the HBase 
changes and MPack changes.

 

 

> Upgrade Hadoop
> --
>
> Key: METRON-2232
> URL: https://issues.apache.org/jira/browse/METRON-2232
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Updated] (METRON-2238) Streaming enrichments regression

2019-08-29 Thread Michael Miklavcic (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Miklavcic updated METRON-2238:
--
Fix Version/s: Next + 1

> Streaming enrichments regression
> 
>
> Key: METRON-2238
> URL: https://issues.apache.org/jira/browse/METRON-2238
> Project: Metron
>  Issue Type: Bug
>Reporter: Michael Miklavcic
>Priority: Major
> Fix For: Next + 1
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I spun up testing the streaming enrichments use case () as part of testing 
> another unrelated PR. The user parser fails with the following stacktrace:
> {code:java}
> 2019-08-27 20:05:59.858 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] 
> [ERROR] 
> java.lang.IllegalAccessError: tried to access method 
> com.google.common.base.Stopwatch.()V from class 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:596)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:580)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:559)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1185)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1152)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1126)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1331)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1155)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:370) 
> ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:321) 
> ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
>  ~[stormjar.jar:?]
>   at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1439) 
> ~[stormjar.jar:?]
>   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1042) 
> ~[stormjar.jar:?]
>   at 
> org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter.write(SimpleHbaseEnrichmentWriter.java:345)
>  ~[stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.applyShouldFlush(BulkWriterComponent.java:179)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.flushAll(BulkWriterComponent.java:152)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.WriterHandler.flush(WriterHandler.java:98) 
> [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.ParserBolt.handleTickTuple(ParserBolt.java:313)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:242) 
> [stormjar.jar:?]
>   at 
> org.apache.storm.daemon.executor$fn__10252$tuple_action_fn__10254.invoke(executor.clj:735)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10171.invoke(executor.clj:469)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.disruptor$clojure_handler$reify__9685.onEvent(disruptor.clj:40)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.daemon.executor$fn__10252$fn__10265$fn__10320.invoke(executor.clj:855)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 

[jira] [Assigned] (METRON-2149) Shaded jar classifier is not consistent

2019-08-29 Thread Michael Miklavcic (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Miklavcic reassigned METRON-2149:
-

Assignee: Ryan Merriman

> Shaded jar classifier is not consistent
> ---
>
> Key: METRON-2149
> URL: https://issues.apache.org/jira/browse/METRON-2149
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Assignee: Ryan Merriman
>Priority: Major
> Fix For: Next + 1
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> The Maven Shade plugin is used in several modules for packaging up code 
> assets into a single, executable jar.  There is an option to use a classifier 
> but it is used the inconsistently:  about half of the modules use it while 
> others don't.  This causes inconsistent mvn dependency resolution and it 
> might be worth investigating a consistent strategy.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Assigned] (METRON-2238) Streaming enrichments regression

2019-08-29 Thread Michael Miklavcic (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Miklavcic reassigned METRON-2238:
-

Assignee: Ryan Merriman

> Streaming enrichments regression
> 
>
> Key: METRON-2238
> URL: https://issues.apache.org/jira/browse/METRON-2238
> Project: Metron
>  Issue Type: Bug
>Reporter: Michael Miklavcic
>Assignee: Ryan Merriman
>Priority: Major
> Fix For: Next + 1
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I spun up testing the streaming enrichments use case () as part of testing 
> another unrelated PR. The user parser fails with the following stacktrace:
> {code:java}
> 2019-08-27 20:05:59.858 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] 
> [ERROR] 
> java.lang.IllegalAccessError: tried to access method 
> com.google.common.base.Stopwatch.()V from class 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:596)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:580)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:559)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1185)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1152)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1126)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1331)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1155)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:370) 
> ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:321) 
> ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
>  ~[stormjar.jar:?]
>   at 
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
>  ~[stormjar.jar:?]
>   at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1439) 
> ~[stormjar.jar:?]
>   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1042) 
> ~[stormjar.jar:?]
>   at 
> org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter.write(SimpleHbaseEnrichmentWriter.java:345)
>  ~[stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.applyShouldFlush(BulkWriterComponent.java:179)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.writer.BulkWriterComponent.flushAll(BulkWriterComponent.java:152)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.WriterHandler.flush(WriterHandler.java:98) 
> [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.ParserBolt.handleTickTuple(ParserBolt.java:313)
>  [stormjar.jar:?]
>   at 
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:242) 
> [stormjar.jar:?]
>   at 
> org.apache.storm.daemon.executor$fn__10252$tuple_action_fn__10254.invoke(executor.clj:735)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10171.invoke(executor.clj:469)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.disruptor$clojure_handler$reify__9685.onEvent(disruptor.clj:40)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>   at 
> org.apache.storm.daemon.executor$fn__10252$fn__10265$fn__10320.invoke(executor.clj:855)
>  [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>  

[jira] [Updated] (METRON-2149) Shaded jar classifier is not consistent

2019-08-29 Thread Michael Miklavcic (Jira)


 [ 
https://issues.apache.org/jira/browse/METRON-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Miklavcic updated METRON-2149:
--
Fix Version/s: Next + 1

> Shaded jar classifier is not consistent
> ---
>
> Key: METRON-2149
> URL: https://issues.apache.org/jira/browse/METRON-2149
> Project: Metron
>  Issue Type: Bug
>Reporter: Ryan Merriman
>Priority: Major
> Fix For: Next + 1
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> The Maven Shade plugin is used in several modules for packaging up code 
> assets into a single, executable jar.  There is an option to use a classifier 
> but it is used the inconsistently:  about half of the modules use it while 
> others don't.  This causes inconsistent mvn dependency resolution and it 
> might be worth investigating a consistent strategy.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[GitHub] [metron] mmiklavc commented on issue #1483: METRON-2217 Migrate current HBase client from HTableInterface to Table

2019-08-29 Thread GitBox
mmiklavc commented on issue #1483: METRON-2217 Migrate current HBase client 
from HTableInterface to Table
URL: https://github.com/apache/metron/pull/1483#issuecomment-526245828
 
 
   @tigerquoll what all are you running? This leaves me with 1 slot available 
still
   
![image](https://user-images.githubusercontent.com/658443/63955284-b16e6d80-ca41-11e9-8bfa-af19d765d3aa.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] mmiklavc commented on issue #1477: METRON-2199: [UI] Add ability to turn off query building in Alerts UI search input

2019-08-29 Thread GitBox
mmiklavc commented on issue #1477: METRON-2199: [UI] Add ability to turn off 
query building in Alerts UI search input
URL: https://github.com/apache/metron/pull/1477#issuecomment-526243889
 
 
   +1, thanks @sardell 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] anandsubbu commented on issue #1489: METRON-2221: Notebook import fails through Zeppelin REST API

2019-08-29 Thread GitBox
anandsubbu commented on issue #1489: METRON-2221: Notebook import fails through 
Zeppelin REST API
URL: https://github.com/apache/metron/pull/1489#issuecomment-526184609
 
 
   Travis issue seems unrelated to this PR. Retry #2 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] anandsubbu opened a new pull request #1489: METRON-2221: Notebook import fails through Zeppelin REST API

2019-08-29 Thread GitBox
anandsubbu opened a new pull request #1489: METRON-2221: Notebook import fails 
through Zeppelin REST API
URL: https://github.com/apache/metron/pull/1489
 
 
   ## Contributor Comments
   This PR modifies the zeppelin notebook import methods to use `curl` instead 
of `python-requests`.
   
   ## Testing Steps
   1. Spin up full dev
   2. Add Zeppelin service
   3. Import notebooks using Ambari -> Metron -> Service Actions -> Zeppelin 
Notebook Import
   
   All metron related zeppelin notebooks should be visible when you login to 
the Zeppelin UI. 
   
   ## Testing Done
   * Verified on both full dev and multi-node cluster and found that zeppelin 
notebooks are being imported properly.
   
   ## Pull Request Checklist
   
   Thank you for submitting a contribution to Apache Metron.  
   Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
   Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  
   
   
   In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? If not one needs to be 
created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
   - [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
   
   
   ### For code changes:
   - [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
   - [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
   - [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
 ```
 mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
 ```
   
   - [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] Have you verified the basic functionality of the build by building and 
running locally with Vagrant full-dev environment or the equivalent?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered by building and verifying the site-book? If not then run the 
following commands and the verify changes via 
`site-book/target/site/index.html`:
   
 ```
 cd site-book
 mvn site
 ```
   
   - [ ] Have you ensured that any documentation diagrams have been updated, 
along with their source files, using [draw.io](https://www.draw.io/)? See 
[Metron Development 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Development+Guidelines)
 for instructions.
   
    Note:
   Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
   It is also recommended that [travis-ci](https://travis-ci.org) is set up for 
your personal repository such that your branches are built there before 
submitting a pull request.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] anandsubbu closed pull request #1489: METRON-2221: Notebook import fails through Zeppelin REST API

2019-08-29 Thread GitBox
anandsubbu closed pull request #1489: METRON-2221: Notebook import fails 
through Zeppelin REST API
URL: https://github.com/apache/metron/pull/1489
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] ruffle1986 edited a comment on issue #1495: METRON-2211: [UI] Alerts UI should optionally render timestamp in local time

2019-08-29 Thread GitBox
ruffle1986 edited a comment on issue #1495: METRON-2211: [UI] Alerts UI should 
optionally render timestamp in local time
URL: https://github.com/apache/metron/pull/1495#issuecomment-526122189
 
 
   Tested it manually and everything worked fine. That's a nice one actually. 
+1 (non-binding)  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] ruffle1986 edited a comment on issue #1477: METRON-2199: [UI] Add ability to turn off query building in Alerts UI search input

2019-08-29 Thread GitBox
ruffle1986 edited a comment on issue #1477: METRON-2199: [UI] Add ability to 
turn off query building in Alerts UI search input
URL: https://github.com/apache/metron/pull/1477#issuecomment-526119013
 
 
   Tested it manually. Worked fine for me. Well done!  +1 (non-binding)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] tigerquoll commented on issue #1483: METRON-2217 Migrate current HBase client from HTableInterface to Table

2019-08-29 Thread GitBox
tigerquoll commented on issue #1483: METRON-2217 Migrate current HBase client 
from HTableInterface to Table
URL: https://github.com/apache/metron/pull/1483#issuecomment-526140555
 
 
   For the test script: full dev still needed another supervisor slot in Storm 
to allow the squid enrichment topology to run.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] ruffle1986 commented on issue #1495: METRON-2211: [UI] Alerts UI should optionally render timestamp in local time

2019-08-29 Thread GitBox
ruffle1986 commented on issue #1495: METRON-2211: [UI] Alerts UI should 
optionally render timestamp in local time
URL: https://github.com/apache/metron/pull/1495#issuecomment-526122189
 
 
   Tested it manually and everything worked fine. That's a nice one actually. 
+1  


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [metron] ruffle1986 commented on issue #1477: METRON-2199: [UI] Add ability to turn off query building in Alerts UI search input

2019-08-29 Thread GitBox
ruffle1986 commented on issue #1477: METRON-2199: [UI] Add ability to turn off 
query building in Alerts UI search input
URL: https://github.com/apache/metron/pull/1477#issuecomment-526119013
 
 
   Tested it manually. Worked fine for me. Well done!  +1


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services