[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 Further, the reason that install worked is not because it needs the artifacts in the cache, per se. It's because of something in the way our multi-module build is structured that isn't

[GitHub] incubator-metron issue #539: METRON-867: In the event that we graduate, remo...

2017-04-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/539 +1. Spun up full dev and verified index is still populating via `curl -XGET "node1:9200/bro*/_count"`. Also spun up the site and grepped for incubator|incubating in the so

[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 fwiw, I ran a local test that seems to work fine. There must a be a subtle difference somewhere in our build, but I haven't found it yet. ``` {18:45}~/devprojects/temp

[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 The .m2 cache is an extension of the local target directories and you're correct that upping the version to 0.4.0 caused the initial issue with missing artifacts. Regardless of maven

[GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 As I stated in https://issues.apache.org/jira/browse/METRON-856, this is a fix that will work given the current configuration, but will quite possibly break with future changes. Given

[GitHub] metron issue #577: METRON-746: Build Custom Checkstyle and IDE formatting se...

2017-07-31 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/577 +1. I get why we didn't include the code formatting rules in the source tree like the checkstyle.xml (covers more than 1 IDE). But we should link to the setup instructions in our dev guide

[GitHub] metron pull request #676: METRON-1072: Cleanup Maven build warnings

2017-08-01 Thread mmiklavc
GitHub user mmiklavc opened a pull request: https://github.com/apache/metron/pull/676 METRON-1072: Cleanup Maven build warnings https://issues.apache.org/jira/browse/METRON-1072 ## Contributor Comments This one is pretty straightforward. Our build outputs a large

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 I see that for parser extensions, the enrichment and indexing configurations have also been moved into this tree. This structure conflates parsers, enrichments, and indexing. Beyond that, I also

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 ``` cat ~/.m2/repository/archetype-catalog.xml http://maven.apache.org/plugins/maven-archetype-plugin/archetype-catalog/1.0.0 http://maven.apache.org/xsd/archetype-catalog-1.0.0.xsd

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Hrm, I completely blew away both archetype catalogs and re-installed our archetype again. It shows up with the correct version. However, the parser folders are still not being created correctly

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Will review your other comments shortly. Some additional questions: I ran the archetype to create a new system parser: ``` [INFO] Parameter: groupId, Value: com.michaelmiklavcic [INFO

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Ok, I see what that's doing now re: metron-parsers. As I was glancing over the project dirs my next question was going to be about what is in metron-parsers vs the extensions, but I follow. I do

[GitHub] metron issue #697: METRON-1110: REST jvm flags property is being overwritten...

2017-08-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/697 +1 by inspection. Thanks for the fix and for tweaking that @merrimanr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] metron issue #688: METRON-1094: MaaS will not start due to classpath error r...

2017-08-11 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/688 +1 by inspection. Thank you for the contribution @cestella --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-11 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Does it make sense to create a feature branch for these PR's? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-14 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 I'm seeing this error spinning up topologies in full-dev. ``` 1:50.278 o.a.s.util [ERROR] Async loop died! java.lang.RuntimeException: Grok parser Error: Grok parser unable

[GitHub] metron issue #697: METRON-1110: REST jvm flags property is being overwritten...

2017-08-14 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/697 Can we normalize this to use the same approach as is done with these 2 properties for enrichment and indexing? ``` enrichment_topology_worker_childopts = client_jaas_arg if security_enabled

[GitHub] metron issue #685: METRON-1087: Adjust license headers to be comments instea...

2017-08-14 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/685 This Travis error with metron-alerts and nodejs is clearly not because of your changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-10 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Hey @ottobackwards. I see no benefit to getting this committed prior to or separate from shoring up the docs. I'm going to work on something to get the ball rolling as I work through some use

[GitHub] metron issue #599: METRON-975: Normalize logging and switch to common idiom ...

2017-07-17 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/599 I would love to get this in --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] metron issue #599: METRON-975: Normalize logging and switch to common idiom ...

2017-07-17 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/599 @merrimanr Conflicts resolved --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] metron issue #657: METRON-1048 Intermittent Test Failure for SimpleHBaseEnri...

2017-07-18 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/657 Another issue with this test that complicates things a bit more is that these are static calls which require more than vanilla Mockito. @jjmeyer0 might be right here. Do we really need these tests

[GitHub] metron issue #657: METRON-1048 Intermittent Test Failure for SimpleHBaseEnri...

2017-07-18 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/657 One problem I see with this test is that it sets up the mock in a BeforeClass annotated init. That's generally frowned upon for mock tests that use verify() because expectations for the number

[GitHub] metron issue #657: METRON-1048 Intermittent Test Failure for SimpleHBaseEnri...

2017-07-18 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/657 Found this useful for duplicating the problem locally. ``` # build everything depended on by metron-parsers cd metron mvn clean install -pl metron-platform/metron-parsers -am

[GitHub] metron issue #614: METRON-992: Create performance tuning guide

2017-07-18 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/614 Any other additions, comments, questions, concerns on this? I will close this out tomorrow unless I hear otherwise. --- If your project is set up for it, you can reply to this email and have your

[GitHub] metron issue #658: METRON-1048 Removed SimpleHBaseEnrichmentWriterLoggingTes...

2017-07-19 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/658 I'm fine with that, +1. I think we have an idea what the problem was so we'll know how to fix it if we ever run into it again. --- If your project is set up for it, you can reply to this email

[GitHub] metron pull request #665: METRON-1060: Add performance timing logging to enr...

2017-07-25 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/665#discussion_r129308721 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/GenericEnrichmentBolt.java --- @@ -179,6 +180,7 @@ public void

[GitHub] metron issue #577: METRON-746: Build Custom Checkstyle and IDE formatting se...

2017-07-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/577 Hey @justinleet, apologies for this slipping between the cracks. I'll run this again and see if there's anything specific I can find. --- If your project is set up for it, you can reply

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-06-29 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Ok, I was getting ready to bail on this for the day but couldn't stop. I tried one more thing - setting the maven-archetype-plugin version to **2.4** and it works as expected. Something must

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-06-29 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 @ottobackwards removing the extra project element allows me to build the project now. However, I'm still unable to use the archetype. It neither shows up in the local catalog listing nor allows me

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-06-29 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Archetype plugin version is the only change ``` http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation=

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-29 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r124722777 --- Diff: metron-maven-archetypes/metron-maven-parser-extension-archetype/src/main/resources/archetype-resources/metron-parser-__parserName__-bundle/pom.xml

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-28 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r124611122 --- Diff: metron-maven-archetypes/README.md --- @@ -0,0 +1,56 @@ +# Metron Maven Archetypes + +These are [Apache Maven Archetypes](http

[GitHub] incubator-metron issue #550: METRON-890: Intermittent unit test errors in sh...

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/550 Is the belief here that killing the topologies first might allow full cluster shutdown and cleanup? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-04-27 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 Don't merge this - need to move this doc change to the new manual kerberos doc in metron-deployment dir. A fix for a bad merge is pending. --- If your project is set up for it, you can

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Should that be in the test src directory? `metron-interface/metron-rest/src/test/resources/README.vm` --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Agreed this isn't Kerberos related. I just wasn't clear what the file was and why it is in test. Any reason the utility can't go in src/main along with the readme? Separate Jira

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 Merge with master again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr Just curious, what's the difference between a vm file and an md file? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] incubator-metron issue #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/521 Oh wow, didn't expect that. Thanks @nickwallen looks great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-04 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Ok, I decided to try yet another combo here. I set the plugin version back to 3.0.1 and re-installed the archetype. Now I get a new file `~/.m2/repository/archetype-catalog.xml` with the archetype

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-04 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 So another wrinkle here - when running the archetype plugin from a non-project (separate, clean) directory, it's going to default to 3.0.1. I'm trying this up in full dev right now and running

[GitHub] metron issue #665: METRON-1060: Add performance timing logging to enrichment...

2017-07-28 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/665 @ottobackwards Just submitted. I've updated the testing notes above as well. Note the added documentation in the metron-common README. --- If your project is set up for it, you can reply

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-02 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 I'm back to looking through this again. A couple questions: 1) I was under the impression that our local copy of the nar functionality would be a hard fork that would naturally deviate from

[GitHub] metron issue #676: METRON-1072: Cleanup Maven build warnings

2017-08-02 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/676 Agreed @ottobackwards - we still have quite a bit of work around the warnings related to shading and relocating jars. Thanks for the review! --- If your project is set up for it, you can reply

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-08 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 This looks much better now. **Note, I also changed the archetype packaging version to 3.0.1 as well since this is the latest.** ``` {11:39}/tmp ➭ mvn archetype:generate

[GitHub] metron issue #685: METRON-1087: Adjust license headers to be comments instea...

2017-08-09 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/685 +1 per inspection, once merge conflicts are resolved. I didn't pour over each file, but I did at least scan every single change and it looks good. Clicking the "load diff" option

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-07 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Running the generate command in CentOS results in defaulting to plugin version 3.0.1 when run from a directory without a pom. In Mac, I'm getting 2.4 for some reason. I haven't found any settings

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-07 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 I just noticed that you had addressed my findings around the plugin version as well. So I think going back to 3.0.1 is ok for now. We do need to figure out what is going on with Mac OS before

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-08-08 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 I think I've figured out what's going on with the plugin versions. I was under the impression that Maven updated its local metadata on at least a daily basis. It would appear that this is not true

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-05-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 @nickwallen can you elaborate on this fix/config a bit? I think we should definitely add this detail to the doc. It looks like you've created a yaf user and principal here. Any additional

[GitHub] metron issue #584: METRON-950: Migrate storm-kafka-client to 1.1

2017-05-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/584 OK, works for me --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] metron issue #584: METRON-950: Migrate storm-kafka-client to 1.1

2017-05-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/584 Just to clarify for others looking at this, there is a `Timer` class in `storm-kafka-client-1.1.0` which depends on a `Time` class (no 'r') in storm-core. In 1.0.3, the main version of Storm we

[GitHub] metron pull request #593: METRON-958: PCAP Query job throws exception when n...

2017-05-17 Thread mmiklavc
GitHub user mmiklavc opened a pull request: https://github.com/apache/metron/pull/593 METRON-958: PCAP Query job throws exception when no files returned by time range query https://issues.apache.org/jira/browse/METRON-958 When the PCAP topology is run against a Kafka topic

[GitHub] metron issue #585: METRON-936: Fixes to pcap for performance and testing

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/585 I found some additional issues with error handling in the HDFSWriterCallback. So I fixed this to throw an IllegalArgumentException when the key is null, but that revealed further problems in our

[GitHub] metron issue #584: METRON-950: Migrate storm-kafka-client to 1.1

2017-05-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/584 @nickwallen Yeah, that's where I landed. Looking in our pcap helper class, I see the following. ``` public static MetronEthernetDecoder createDecoder() { MetronEthernetDecoder

[GitHub] metron issue #584: METRON-950: Migrate storm-kafka-client to 1.1

2017-05-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/584 When I spin this up in full dev, half the records don't output with the pcap_inspector. I'm running: `pycapa --producer --interface eth0 --kafka-broker node1:6667 --kafka-topic pcap --max

[GitHub] metron pull request #569: METRON-936: PCAP topology properties in flux not c...

2017-05-12 Thread mmiklavc
Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/569 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-20 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r123085517 --- Diff: bundles-maven-plugin/src/main/java/org/apache/metron/BundleMojo.java --- @@ -0,0 +1,741 @@ +/* + * Licensed to the Apache Software

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-20 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r123084463 --- Diff: bundles-maven-plugin/src/main/resources/META-INF/plexus/components.xml --- @@ -0,0 +1,52

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-20 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r123080563 --- Diff: metron-platform/metron-extensions/metron-parser-extensions/metron-parser-bro-extension/metron-parser-bro/pom.xml --- @@ -0,0 +1,165

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-20 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r123033739 --- Diff: bundles-maven-plugin/.gitignore --- @@ -0,0 +1,16 @@ +target +.project +.settings +.classpath +nbactions.xml +nb

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-20 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r123090723 --- Diff: metron-analytics/metron-maas-service/README.md --- @@ -138,7 +138,7 @@ Now that we have a deployed model, let's adjust the configurations

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-26 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r124111690 --- Diff: metron-platform/metron-extensions/metron-parser-extensions/metron-parser-bro-extension/metron-parser-bro-assembly/src/main/assembly/assembly.xml

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

2017-06-26 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r124111822 --- Diff: metron-analytics/metron-maas-service/README.md --- @@ -138,7 +138,7 @@ Now that we have a deployed model, let's adjust the configurations

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

2017-06-27 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/530 Looking into it now, but building locally I see this: ``` $ mvn clean install -DskipTests -T 2C ... [INFO] elasticsearch-shaded ... SKIPPED [INFO

[GitHub] metron issue #593: METRON-958: PCAP Query job throws exception when no files...

2017-05-20 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/593 Thanks @merrimanr! Good observation on the start/end date. Part of the problem is that we don't know how large the file time ranges are or when they will roll over, so we might still miss some

[GitHub] metron issue #593: METRON-958: PCAP Query job throws exception when no files...

2017-05-18 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/593 Added fix for the actual empty result set problem ``` INFO mr.PcapJob: No files to process with specified date range. No results returned. ``` --- If your project is set up

[GitHub] metron issue #593: METRON-958: PCAP Query job throws exception when no files...

2017-05-19 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/593 Here are some instructions for getting pycapa setup ``` Installing pycapa env vars PYCAPA_HOME=/opt/pycapa PYTHON27_HOME =/opt/rh/python27/root

[GitHub] metron issue #584: METRON-950: Migrate storm-kafka-client to 1.1

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/584 +1. Data flowed through to ES as expected, and I was able to spin up pcap. Along with @justinleet having tested the other topologies, I'm happy with the results. Great work @cestella! --- If your

[GitHub] metron pull request #585: METRON-936: Fixes to pcap for performance and test...

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/585#discussion_r116848618 --- Diff: metron-platform/metron-pcap-backend/src/main/java/org/apache/metron/spout/pcap/PartitionHDFSWriter.java --- @@ -183,14 +219,14 @@ private void

[GitHub] metron issue #585: METRON-936: Fixes to pcap for performance and testing

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/585 I see that. re: keys and methods for retrieving and saving them. I'll save refactoring and cleaning that up to a separate PR. --- If your project is set up for it, you can reply to this email

[GitHub] metron pull request #585: METRON-936: Fixes to pcap for performance and test...

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/585#discussion_r116865536 --- Diff: metron-platform/metron-pcap-backend/src/main/java/org/apache/metron/spout/pcap/PartitionHDFSWriter.java --- @@ -102,14 +106,43 @@ public void

[GitHub] metron pull request #585: METRON-936: Fixes to pcap for performance and test...

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/585#discussion_r116866077 --- Diff: metron-platform/metron-pcap-backend/src/main/java/org/apache/metron/spout/pcap/HDFSWriterCallback.java --- @@ -116,7 +117,11 @@ public

[GitHub] metron pull request #585: METRON-936: Fixes to pcap for performance and test...

2017-05-16 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/585#discussion_r116883407 --- Diff: metron-platform/metron-pcap-backend/src/main/java/org/apache/metron/spout/pcap/deserializer/KeyValueDeserializer.java --- @@ -28,6 +26,17

[GitHub] metron issue #572: METRON-633: Create better logging for HbaseEnrichmentWrit...

2017-05-31 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/572 Agreed with @justinleet about switching away from isDebugEnabled(). Except for instances where you're doing something that has a fair amount of overhead in evaluating the pre-formatted strings

[GitHub] metron issue #507: METRON-819: Document kafka console producer parameter for...

2017-05-31 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/507 Ok, I grabbed the latest master branch and force pushed out a new commit with the doc changes to this branch. @nickwallen @justinleet can you take a peek at this and see if there isn't

[GitHub] metron pull request #572: METRON-633: Create better logging for HbaseEnrichm...

2017-05-31 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/572#discussion_r119442810 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/writer/SimpleHbaseEnrichmentWriter.java --- @@ -169,16 +188,19

[GitHub] metron issue #577: METRON-746: Build Custom Checkstyle and IDE formatting se...

2017-05-31 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/577 @justinleet nothing in particular - what I noticed was that auto-formatting would format me right into breaking the checkstyle checks. I'd like to figure out how to get the two working consistently

[GitHub] metron issue #507: METRON-819: Document kafka console producer parameter for...

2017-05-31 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/507 @justinleet - I'm changing this entirely per @nickwallen's comments about the jaas file. Coming shortly. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] metron pull request #614: METRON-992: Create performance tuning guide

2017-06-08 Thread mmiklavc
GitHub user mmiklavc opened a pull request: https://github.com/apache/metron/pull/614 METRON-992: Create performance tuning guide https://issues.apache.org/jira/browse/METRON-992 This guide covers performance tuning the Metron topologies. I will be leaving this up

[GitHub] metron pull request #614: METRON-992: Create performance tuning guide

2017-06-08 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/614#discussion_r120979059 --- Diff: metron-platform/Performance-tuning-guide.md --- @@ -0,0 +1,326 @@ +# Metron Performance Tunining Guide + +## Overview

[GitHub] metron pull request #614: METRON-992: Create performance tuning guide

2017-06-08 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/614#discussion_r120979346 --- Diff: metron-platform/Performance-tuning-guide.md --- @@ -0,0 +1,326 @@ +# Metron Performance Tunining Guide + +## Overview

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-05-01 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 I didn't have any problems using the Kafka shell commands without a JAAS file. That was on single node Vagrant, but I'm not sure there should be much of a difference in this case

[GitHub] incubator-metron issue #459: METRON-726: Clean up mvn site generation

2017-05-02 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/459 Still +1. The build works when splitting those phases. I'm still not sure what Maven is doing with our particular build that requires the split. I don't think it's in scope for this PR

[GitHub] incubator-metron pull request #566: METRON-932: Change HLLP Stellar function...

2017-05-05 Thread mmiklavc
GitHub user mmiklavc reopened a pull request: https://github.com/apache/incubator-metron/pull/566 METRON-932: Change HLLP Stellar functions to accept empty lists https://issues.apache.org/jira/browse/METRON-932 I changed the unit tests and README to reflect the new

[GitHub] incubator-metron issue #566: METRON-932: Change HLLP Stellar functions to ac...

2017-05-05 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/566 Kick --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron pull request #566: METRON-932: Change HLLP Stellar function...

2017-05-05 Thread mmiklavc
Github user mmiklavc closed the pull request at: https://github.com/apache/incubator-metron/pull/566 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron issue #569: METRON-936: PCAP topology properties in flux no...

2017-05-08 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/569 Kick Travis - appears to be a test failure in the rest api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] incubator-metron pull request #569: METRON-936: PCAP topology properties in ...

2017-05-08 Thread mmiklavc
GitHub user mmiklavc reopened a pull request: https://github.com/apache/incubator-metron/pull/569 METRON-936: PCAP topology properties in flux not correctly using pcap.properties Properties weren't being exposed and required users to modify the flux file directly for pcap

[GitHub] incubator-metron pull request #569: METRON-936: PCAP topology properties in ...

2017-05-08 Thread mmiklavc
Github user mmiklavc closed the pull request at: https://github.com/apache/incubator-metron/pull/569 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron issue #459: METRON-726: Clean up mvn site generation

2017-05-02 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/459 Pulled down the latest version of this branch: ``` Downloading: https://repo.maven.apache.org/maven2/org/apache/metron/metron-test-utilities/0.4.0/metron-test-utilities-0.4.0.jar

[GitHub] metron issue #590: METRON-953: Complete migration away from incubator-metron...

2017-05-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/590 +1, thanks for submitting this @justinleet I validated the full-dev build with a Zeppelin installation and PCAP dashboard install per the referenced PR above. I also built the new site

[GitHub] metron pull request #593: METRON-958: PCAP Query job throws exception when n...

2017-05-19 Thread mmiklavc
GitHub user mmiklavc reopened a pull request: https://github.com/apache/metron/pull/593 METRON-958: PCAP Query job throws exception when no files returned by time range query https://issues.apache.org/jira/browse/METRON-958 When the PCAP topology is run against a Kafka

[GitHub] metron pull request #760: METRON-1188: Ambari global configuration managemen...

2017-09-15 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/760#discussion_r139190738 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -76,7 +80,7 @@ public static

[GitHub] metron issue #760: METRON-1188: Ambari global configuration management broke...

2017-09-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/760 > Can you reference : http://mail-archives.apache.org/mod_mbox/metron-dev/201701.mbox/%3cCAPpQHK1svDAx7R-s7x2Q0kJR6=5d+KDmR6a+ZmSo-5=wcmo...@mail.gmail.com%3e > Which I think is the discus

[GitHub] metron issue #751: METRON-1180: Make Stellar Shell accept zookeeper quorum a...

2017-09-13 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/751 +1 by inspection. Nice fix ---

[GitHub] metron issue #760: METRON-1188: Ambari global configuration management broke...

2017-09-15 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/760 Just pushed a unit test fix to get the build working again. The other code changes and doc requests from @ottobackwards and @justinleet soon to follow. Thanks for the feedback guys! @ottobackwards

[GitHub] metron pull request #760: METRON-1188: Ambari global configuration managemen...

2017-09-16 Thread mmiklavc
GitHub user mmiklavc reopened a pull request: https://github.com/apache/metron/pull/760 METRON-1188: Ambari global configuration management broken ## Contributor Comments This PR addresses https://issues.apache.org/jira/browse/METRON-1188 It also references, but does

[GitHub] metron pull request #760: METRON-1188: Ambari global configuration managemen...

2017-09-16 Thread mmiklavc
Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/760 ---

  1   2   3   4   >