[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-20 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/421 METRON-283 Migrate Geo Enrichment outside of MySQL ## MySQL Removed Drops MySQL entirely from the project. This is done for a couple reasons outlined in a discussion thread on the

[GitHub] incubator-metron issue #421: METRON-283 Migrate Geo Enrichment outside of My...

2017-01-20 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/421 Expanding the Stellar GEO_GET function to allow for just returning a map of specified fields (if more than one), or just the direct value (if just one field requested) would be very

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97769238 --- Diff: LICENSE --- @@ -210,6 +210,12 @@ This product bundles some test examples from the Stix project (metron-platform/m This

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97769348 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97769416 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/stellar/StellarAdapter.java --- @@ -81,7 +81,6

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97770743 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/GenericEnrichmentBolt.java --- @@ -161,6 +167,7

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97771001 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/ThreatIntelJoinBolt.java --- @@ -63,12 +65,15

[GitHub] incubator-metron issue #421: METRON-283 Migrate Geo Enrichment outside of My...

2017-01-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/421 As a note to anyone coming in late, at least one comment (David's) is still relevant, but hidden behind a collapsed outdated diff, because the line he commented on needed to be de

[GitHub] incubator-metron issue #421: METRON-283 Migrate Geo Enrichment outside of My...

2017-01-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/421 Forgot to comment on it, but the GEO_GET call was updated a bit ago per my comment about lists and fields. That change should also be reviewed and given any feedback. --- If your

[GitHub] incubator-metron issue #418: METRON-666 Fix javadoc doclint errors

2017-01-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/418 I'm +1 on this, and definitely glad to see it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] incubator-metron pull request #423: METRON-270: Add Zeppelin to the platform

2017-01-25 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/423 METRON-270: Add Zeppelin to the platform Adds Zeppelin to the Ambari Management Pack portion. Adding to the Ansible / quick dev portion is happening a bit in parallel with Nick's

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97816667 --- Diff: metron-deployment/roles/snort/files/snort.conf --- @@ -586,7 +586,6 @@ include $RULE_PATH/community.rules # include $RULE_PATH

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97818189 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/GeoEnrichmentFunctions.java --- @@ -0,0 +1,110

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97819095 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/geo/GeoLiteDatabase.java --- @@ -0,0 +1,184

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97824391 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/GenericEnrichmentBolt.java --- @@ -149,9 +154,10

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97826728 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/geo/GeoLiteDatabase.java --- @@ -0,0 +1,184

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97828446 --- Diff: metron-deployment/packaging/docker/rpm-docker/SPECS/metron.spec --- @@ -317,6 +316,8 @@ This package installs the Metron Profiler

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97829967 --- Diff: metron-platform/metron-data-management/README.md --- @@ -250,3 +250,18 @@ The parameters for the utility are as follows: | -l

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97837151 --- Diff: metron-deployment/packaging/docker/rpm-docker/SPECS/metron.spec --- @@ -317,6 +316,8 @@ This package installs the Metron Profiler

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97842488 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/GeoEnrichmentFunctions.java --- @@ -0,0 +1,110

[GitHub] incubator-metron issue #421: METRON-283 Migrate Geo Enrichment outside of My...

2017-01-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/421 Updated to replace `EnrichmentAdapter.initializeAdapter()` with `EnrichmentAdapter.initializeAdapter(Map config)`. Also added `updateAdapter(Map config);`. Right now the configs only

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97877931 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/adapters/geo/GeoLiteDatabase.java --- @@ -0,0 +1,184

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97878462 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/GeoEnrichmentFunctions.java --- @@ -0,0 +1,110

[GitHub] incubator-metron pull request #421: METRON-283 Migrate Geo Enrichment outsid...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/421#discussion_r97895278 --- Diff: LICENSE --- @@ -210,6 +210,12 @@ This product bundles some test examples from the Stix project (metron-platform/m This

[GitHub] incubator-metron pull request #424: METRON-672: SolrIndexingIntegrationTest ...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/424#discussion_r97914854 --- Diff: metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/integration/IndexingIntegrationTest.java --- @@ -139,11

[GitHub] incubator-metron pull request #424: METRON-672: SolrIndexingIntegrationTest ...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/424#discussion_r97914890 --- Diff: metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/integration/IndexingIntegrationTest.java --- @@ -184,6

[GitHub] incubator-metron pull request #424: METRON-672: SolrIndexingIntegrationTest ...

2017-01-25 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/424#discussion_r97915042 --- Diff: metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/integration/components/ConfigUploadComponent.java

[GitHub] incubator-metron issue #424: METRON-672: SolrIndexingIntegrationTest fails i...

2017-01-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/424 Thanks for taking the effort to dig into this. Great work. Other than a couple minor comments, I'm very happy with this. --- If your project is set up for it, you can reply to

[GitHub] incubator-metron issue #424: METRON-672: SolrIndexingIntegrationTest fails i...

2017-01-26 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/424 Thanks again for this. I'm +1 on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] incubator-metron issue #421: METRON-283 Migrate Geo Enrichment outside of My...

2017-01-26 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/421 @cestella @dlyle65535 @nickwallen I added one more change to `EnrichmentIntegrationTest` to not create and destroy some tmp stuff that's not being used in the test. Let me kn

[GitHub] incubator-metron pull request #423: METRON-270: Add Zeppelin to the platform

2017-01-26 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/423#discussion_r98042135 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py

[GitHub] incubator-metron pull request #423: METRON-270: Add Zeppelin to the platform

2017-01-26 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/423#discussion_r98042802 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/metainfo.xml --- @@ -187,6 +187,14

[GitHub] incubator-metron issue #427: METRON-676 Create Zeppelin Notebook for YAF Tel...

2017-01-27 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/427 @nickwallen #423 is in (and breaking the README here) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] incubator-metron issue #427: METRON-676 Create Zeppelin Notebook for YAF Tel...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/427 @nickwallen Would it be reasonable to throw links to the screenshots onto the original discuss thread? I'm +1 (by inspection) on the actual implementation, but I'd r

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 This is a fantastic improvement to what we have, and even though I'm just starting to dig in, this looks really good. I'll add any comments I have as I go through it, but thi

[GitHub] incubator-metron pull request #429: METRON-660 [Umbrella] up-to-date version...

2017-01-30 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/429#discussion_r98444660 --- Diff: site-book/src/site/site.xml --- @@ -0,0 +1,124 @@ + + +http://maven.apache.org/DECORATION/1.3.0"; --- End

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 Bullet points get a little weird in some places. (e.g. in the metron-maas-service README). Looks like, if we don't have a newline before the first bullet, it looks like it

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 @mattf-horton Homebrew gawk seems to work out well for me. Not sure what the implementation difference is between the two though. --- If your project is set up for it, you can reply

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 @mattf-horton Screenshots from your build vs mine of code formatting. I assume both are off the latest, because the title name is correct on both (not included in the pics). Both

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 @mattf-horton Obvious awk issue in output (that doesn't show up with gawk), that I should have noted, but apparently just shut my brain off for: ` Fixing up mar

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-01-30 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 @mattf-horton I think we're dealing with different implementations of awk, rather than just a pure versioning issue. I'm guessing it works on that VM, because that version nu

[GitHub] incubator-metron issue #427: METRON-676 Create Zeppelin Notebook for YAF Tel...

2017-01-31 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/427 @nickwallen Thanks for closing off that discussion. This has percolated awhile, and I'm +1 on it. --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-metron pull request #431: METRON-680: GeoLiteDatabase incorrectly ...

2017-01-31 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/431 METRON-680: GeoLiteDatabase incorrectly using country geoname_id instead of city See the discussion [METRON-680]( https://issues.apache.org/jira/browse/METRON-680) for more info

[GitHub] incubator-metron pull request #431: METRON-680: GeoLiteDatabase incorrectly ...

2017-02-01 Thread justinleet
Github user justinleet closed the pull request at: https://github.com/apache/incubator-metron/pull/431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron pull request #433: Metron 680: GeoLiteDatabase incorrectly ...

2017-02-02 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/433 Metron 680: GeoLiteDatabase incorrectly using country geoname_id instead of city Swaps out country's geoname_id for city's geoname_id. Both ids are the same in format (t

[GitHub] incubator-metron issue #433: METRON-680: GeoLiteDatabase incorrectly using c...

2017-02-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/433 @cestella My mistake, fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] incubator-metron issue #433: METRON-680: GeoLiteDatabase incorrectly using c...

2017-02-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/433 @cestella We definitely could. [METRON-679](https://issues.apache.org/jira/browse/METRON-679) has the details about all the other fields we could be passing, including the current

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-02-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 @mattf-horton, thanks for taking the time/effort to migrate the script. Tried it out and it works great. The only thing I'd like to see is the indentation in the Python s

[GitHub] incubator-metron issue #429: METRON-660 [Umbrella] up-to-date versioned docu...

2017-02-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/429 +1, this is great. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] incubator-metron issue #437: METRON-692: Update Upgrading.md for 0.3.0 -> 0....

2017-02-03 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/437 While the migration from MySQL is transparent once data is in place, it requires running geo_enrichment_load.sh the first time to populate the GeoIP data into HDFS (or alternatively

[GitHub] incubator-metron issue #437: METRON-692: Update Upgrading.md for 0.3.0 -> 0....

2017-02-06 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/437 @cestella Let me know if there's anythign to change or adjust that you see in here. I'm not sure if you can get the raw version in GitHub like in Jira, so ping me if

[GitHub] incubator-metron issue #437: METRON-692: Update Upgrading.md for 0.3.0 -> 0....

2017-02-06 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/437 Everything I know of is in here. +1. Thanks for putting this together. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] incubator-metron issue #443: METRON-703: Rev the version from 0.3.0 to 0.3.1

2017-02-07 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/443 Built and installed the mpack. Versioning looks good where it shows up, and everything installed and started up correctly. --- If your project is set up for it, you can reply to this

[GitHub] incubator-metron issue #422: METRON-670 Monit Incorrectly Reports Status

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/422 +1, spun this up in quick dev, and it seems to work well and the UI reports the timeout on status is 60 seconds. Thanks for grabbing this. --- If your project is set up for it, you

[GitHub] incubator-metron issue #422: METRON-670 Monit Incorrectly Reports Status

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/422 @dlyle65535 I'm fine with holding off on it. I wasn't sure of the timing on that, and I have been annoyed by this issue in my own testing in the interim. @nickwalle

[GitHub] incubator-metron pull request #438: METRON-686 Record Rule Set that Fired Du...

2017-02-09 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/438#discussion_r100372924 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/threatintel/triage/ThreatTriageProcessor.java --- @@ -52,15 +74,36

[GitHub] incubator-metron pull request #438: METRON-686 Record Rule Set that Fired Du...

2017-02-09 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/438#discussion_r100395910 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/threatintel/triage/ThreatTriageProcessor.java --- @@ -52,15 +74,36

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 I saw some odd behavior I think is unrelated to this PR itself while testing. I tried to remove the threat triage rule, messed up, fixed it, and then borked my conf variable

[GitHub] incubator-metron pull request #448: Rev MPack Version to 0.3.1.0

2017-02-09 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/448 Rev MPack Version to 0.3.1.0 Updating the version number in the mpack stuff (nothing automated, just manually changed it). Built a new version of the mpack locally, and it has

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 @nickwallen I've been able to repeat this, by leaving reason undefined. Any idea why that would cause the configs to go haywire? --- If your project is set up for it, you can rep

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 Seems like THREAT_TRIAGE_REMOVE just behaves really badly ``` [Stellar]>>> conf := THREAT_TRIAGE_ADD(conf, [triage]) [Stellar]>>> conf := THREAT_

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 After talking with Casey, it's an issue with StellarShell, not this PR. I'll make a ticket and get it done. So feel free to ignore this issue, @nickwallen --- If your

[GitHub] incubator-metron issue #448: Rev MPack Version to 0.3.1.0

2017-02-09 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/448 Went through the Ambari steps to point where install started running. Version numbers look good throughout. --- If your project is set up for it, you can reply to this email and have

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-10 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 +1, Everything looks good. As noted above, both the issue I had and the review comment were addressed by adding new Jiras and aren't needed here. Thanks a lot, Nick. Th

[GitHub] incubator-metron pull request #450: METRON-690: Create a DSL-based timestamp...

2017-02-10 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/450#discussion_r100557554 --- Diff: metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java --- @@ -0,0

[GitHub] incubator-metron pull request #450: METRON-690: Create a DSL-based timestamp...

2017-02-10 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/450#discussion_r100556457 --- Diff: metron-analytics/metron-profiler-client/src/test/java/org/apache/metron/profiler/client/stellar/IntervalPredicateTest.java

[GitHub] incubator-metron pull request #450: METRON-690: Create a DSL-based timestamp...

2017-02-10 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/450#discussion_r100557597 --- Diff: metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java --- @@ -0,0

[GitHub] incubator-metron pull request #450: METRON-690: Create a DSL-based timestamp...

2017-02-10 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/450#discussion_r100571629 --- Diff: metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java --- @@ -0,0

[GitHub] incubator-metron pull request #458: METRON-725: Javadoc is broken by the use...

2017-02-19 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/458 METRON-725: Javadoc is broken by the use of apiNote As noted in the ticket, there are 3 options for fixing this 1. Just delete the offending inherited javadoc 2. Explicitly

[GitHub] incubator-metron pull request #459: METRON-726: Clean up mvn site generation

2017-02-19 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/459 METRON-726: Clean up mvn site generation ### Summary An initial attempt to add checkstyle expanded a bit once it turned out that `mvn site` appears to not work right now (and

[GitHub] incubator-metron issue #444: METRON-705: Parallelize the build in travis to ...

2017-02-20 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/444 +1, I was able to run this up in Travis setup on my repo without issue and it ran right in the middle of the time range listed --- If your project is set up for it, you can reply to

[GitHub] incubator-metron issue #458: METRON-725: Javadoc is broken by the use of api...

2017-02-21 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/458 I agree on the "should", and share the same concerns. It would be nice to get everything generated and run regularly, but improving our build is a prereq for that.

[GitHub] incubator-metron issue #459: METRON-726: Clean up mvn site generation

2017-02-21 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/459 Discussion thread has been created on the dev list, per @cestella's suggestion. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] incubator-metron pull request #450: METRON-690: Create a DSL-based timestamp...

2017-02-21 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/450#discussion_r102311236 --- Diff: metron-analytics/metron-profiler-client/src/main/antlr4/org/apache/metron/profiler/client/window/generated/Window.g4 --- @@ -0,0

[GitHub] incubator-metron issue #450: METRON-690: Create a DSL-based timestamp lookup...

2017-02-21 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/450 Ran this up and tested per @cestella instructions, along with testing out the language a bit more. Everything worked well for me. My only gotcha, and this is more a general

[GitHub] incubator-metron pull request #461: METRON-733: Remove Geo database from Par...

2017-02-21 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/461 METRON-733: Remove Geo database from ParserBolt To create the original problem, just run up a parser and make sure there's no geo data on hdfs (by default in /apps/metro

[GitHub] incubator-metron issue #461: METRON-733: Remove Geo database from ParserBolt

2017-02-21 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/461 Updated to drop the local geo database in parsers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] incubator-metron issue #461: METRON-733: Remove Geo database from ParserBolt

2017-02-22 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/461 @nickwallen it is a separate issue. https://issues.apache.org/jira/browse/METRON-734 exists to track and fix it --- If your project is set up for it, you can reply to this email and

[GitHub] incubator-metron pull request #462: DO NOT MERGE METRON-734 Builds failing b...

2017-02-22 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/462 DO NOT MERGE METRON-734 Builds failing because of MaxMind DB transitive dependency The pure exclude solution. Still being tested on Vagrant, but making it available for viewing. You

[GitHub] incubator-metron issue #462: DO NOT MERGE METRON-734 Builds failing because ...

2017-02-22 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/462 Spun up in Vagrant and saw enrichment data pass through (including Geo data) as expected. In addition, GEO_GET in Stellar worked as expected for all data. --- If your project is set

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-22 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 Do we know how much work it is the create the enforcement layer? Is it something we want to hold this up for, or is it something that gets flattened here and a ticket gets made to

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

2017-02-22 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/453 @merrimanr Can we make rawMessage, rawMessage_bytes, and rawMessage_hash to raw_message, raw_message_bytes, and raw_message_hash for consistency with error_type and failed_sensor_type

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

2017-02-22 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/453 @merrimanr Thanks for pointing me there. That looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

2017-02-23 Thread justinleet
GitHub user justinleet reopened a pull request: https://github.com/apache/incubator-metron/pull/462 METRON-734 Builds failing because of MaxMind DB transitive dependency The pure exclude solution. Still being tested on Vagrant, but making it available for viewing. You can merge

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

2017-02-23 Thread justinleet
Github user justinleet closed the pull request at: https://github.com/apache/incubator-metron/pull/462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/462 Apparently https://issues.apache.org/jira/browse/METRON-728 occurs more frequently on travis than my local machine. The Travis running on my personal account already succeed

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Are we settling on "less sporadic"? Like I noted in the ticket, I had the original test run for over a minute (~90 seconds) before the JVM decided to actually be single threa

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Nevermind, I can't read. You ran the whole test 100k times, correct? I'm fine with that. --- If your project is set up for it, you can reply to this email and have your re

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella The more I'm thinking about this, the more I wonder if this test is inherently structured incorrectly. My thinking is that it seems more like we're testing whethe

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella that is a much better way of stating it, and exactly what I was alluding to. I'll look through the new commit. --- If your project is set up for it, you can reply to

[GitHub] incubator-metron pull request #463: METRON-728: ReaderSpliteratorTest fails ...

2017-02-23 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/463#discussion_r102776136 --- Diff: metron-platform/metron-common/src/test/java/org/apache/metron/common/utils/file/ReaderSpliteratorTest.java --- @@ -97,88 +110,73

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 +1, I appreciate you going ahead and taking this ticket, given that I've been bitten by it twice now. Looks great. --- If your project is set up for it, you can reply to this

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella Spy() syntax ends up working differently than mock() from what I can tell. This worked for me ``` Spliterator delegatingSpliterator = spy(spliterator

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 I figured out why From the docs: ``` List list = new LinkedList(); List spy = spy(list); //Impossible: real method is called so spy.get(0

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Yep, my +1 is still in place. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] incubator-metron issue #438: METRON-686 Record Rule Set that Fired During Th...

2017-02-24 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/438 @nickwallen I have slight preference towards flattening, fixing, and unflattening. I'd rather conform to convention and keep things consistent for now. I could pretty easi

[GitHub] incubator-metron issue #464: METRON-740: Normalizing and adding log4j proper...

2017-02-24 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/464 +1 by inspection. Nice to have this setup --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] incubator-metron issue #465: METRON-741: Stellar Field Transformations shoul...

2017-02-24 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/465 +1, by inspection. Thanks for grabbing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

2017-03-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/453 I tried running this up and discovered that there's at least one error that doesn't get caught. Json parsing errors, e.g. if someone gives outright badly formatted messages t

[GitHub] incubator-metron pull request #469: DO NOT MERGE METRON-745: Create Error Da...

2017-03-02 Thread justinleet
GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/469 DO NOT MERGE METRON-745: Create Error Dashboards # DO NOT MERGE ## Summary Based on Ryan's work in https://github.com/apache/incubator-metron/pull/453, I went ahea

  1   2   3   4   >