[GitHub] incubator-metron pull request #539: METRON-867: In the event that we graduat...

2017-04-20 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/539 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron pull request #539: METRON-867: In the event that we graduat...

2017-04-20 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/539 METRON-867: In the event that we graduate, remove incubating from the website and documentation ## Contributor Comments NOTE: * This is not an indication of graduation status

[GitHub] incubator-metron issue #539: METRON-867: In the event that we graduate, remo...

2017-04-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/539 Yeah, just waiting for the press release before we make this too public. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] incubator-metron issue #541: METRON-870: Add filtering by packet payload to ...

2017-04-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/541 It appears that byteseek has a LGPL dependency in gnu trove, a primitive collections library. As a stopgap, I: * excluded the dependency * provided a translation layer inside

[GitHub] incubator-metron pull request #541: METRON-870: Add filtering by packet payl...

2017-04-20 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/541 METRON-870: Add filtering by packet payload to the pcap query ## Contributor Comments Currently we have the ability to filter packets in the pcap query tool by header information

[GitHub] incubator-metron pull request #541: METRON-870: Add filtering by packet payl...

2017-04-21 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/541 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron issue #541: METRON-870: Add filtering by packet payload to ...

2017-04-21 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/541 Hey, thanks for that feedback @nishihatapalmer ! I adjusted to use the suggested searcher. I did have one more question, I'm looking to document the possible regex's available

[GitHub] incubator-metron issue #541: METRON-870: Add filtering by packet payload to ...

2017-04-21 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/541 Currently, I'm using the SequenceMatcher to compile a matching expression and then using a searcher to search in the byte array for that expression (code is [here](https://github.com

[GitHub] incubator-metron issue #534: METRON-861: Allow JVM args to be passed to CLI ...

2017-04-21 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/534 I could get behind that. Anyone else have other ideas? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] incubator-metron pull request #541: METRON-870: Add filtering by packet payl...

2017-04-23 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/541 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron pull request #541: METRON-870: Add filtering by packet payl...

2017-04-23 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/541 METRON-870: Add filtering by packet payload to the pcap query ## Contributor Comments Currently we have the ability to filter packets in the pcap query tool by header information

[GitHub] incubator-metron issue #542: METRON-873: Stellar string literals do not supp...

2017-04-21 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/542 We're not using SCHAR anymore. The current commit should address the backslash issue. Give it a try and let me know what you think. :) --- If your project is set up for it, you can

[GitHub] incubator-metron issue #542: METRON-873: Stellar string literals do not supp...

2017-04-21 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/542 @ottobackwards you're totally right. We desperately are in need of better stellar documentation: * A language reference * A set of introductory lessons in Stellar --- If your

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

2017-04-21 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/542 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

2017-04-21 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/542 METRON-873: Stellar string literals do not support quote escaping ## Contributor Comments Right now, in stellar, we cannot represent a string literal that contains `'foo

[GitHub] metron issue #666: METRON-1051: Enable the ability to update indexed message...

2017-08-02 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/666 Testing Instructions beyond the normal smoke test (i.e. letting data flow through to the indices and checking them). # Preliminaries Set an environment variable to indicate

[GitHub] metron issue #641: METRON-539: added HASH function for stellar.

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/641 Ok, I'm +1 on this by inspection as soon as we have the conflict resolved. Great job, @jjmeyer0 --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] metron issue #666: METRON-1051: Enable the ability to update indexed message...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/666 Ok, architectural coverage is mentioned here, so I think this is ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] metron issue #650: METRON-1038: Stellar should have a better collection of b...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/650 Committed, but I wanted to make a comment about the BiFunction. Honestly, I like the notion of providing candy for 2-arg functions like we did for single-arg functions. I think it'll make

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/667#discussion_r131099946 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java --- @@ -0,0 +1,63

[GitHub] metron issue #666: METRON-1051: Enable the ability to update indexed message...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/666 Just a brief note about the `TableProvider` business. We had cut and pasted a mock HTableProvider through several of our projects. In order to test the `HBaseDao`, I needed yet another

[GitHub] metron issue #675: METRON-379 STELLAR can differentiate between a value pass...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/675 Ok, wait, I think I might've misunderstood. This `DefaultVariableResolver` is just being used for tests, validation and Lambda functions now, is that correct? I think I'm ok

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131170639 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,99 @@ public

[GitHub] metron issue #675: METRON-379 STELLAR can differentiate between a value pass...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/675 Yeah, I'm totally ok with this now. Really sorry about jumping to the wrong conclusion; I saw the errors in the aftermath of the math PR that got in this morning and jumped to the wrong conclusion

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131172968 --- Diff: metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/components/ElasticSearchComponent.java

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131172237 --- Diff: metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchUpdateIntegrationTest.java

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131174351 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,99 @@ public

[GitHub] metron issue #675: METRON-379 STELLAR can differentiate between a value pass...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/675 hah no, a JIRA will be necessary ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] metron issue #675: METRON-379 STELLAR can differentiate between a value pass...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/675 +1 by inspection, this is good work, @ottobackwards Sorry for jumping to conclusions! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131173765 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,99 @@ public

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131172518 --- Diff: metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchUpdateIntegrationTest.java

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131217265 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,99 @@ public

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131246467 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,99 @@ public

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131249786 --- Diff: metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/components/ElasticSearchComponent.java

[GitHub] metron issue #666: METRON-1051: Enable the ability to update indexed message...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/666 Alright, I think all of the concerns thus far are addressed. Let me know if I missed anything @justinleet , et al --- If your project is set up for it, you can reply to this email and have your

[GitHub] metron issue #675: METRON-379 STELLAR can differentiate between a value pass...

2017-08-03 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/675 In my opinion, I don't think we want this behavior. I think it may be too severe given how dirty and sparse our data tends to be. Does this mean that if you have a stellar expression `FOO

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/667#discussion_r131148105 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java --- @@ -0,0 +1,63

[GitHub] metron pull request #677: METRON-1073: Add option to limit fields returned f...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/677#discussion_r131133901 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -81,8 +82,14 @@ public

[GitHub] metron pull request #666: METRON-1051: Enable the ability to update indexed ...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/666#discussion_r131129176 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -102,15 +128,100 @@ public

[GitHub] metron pull request #677: METRON-1073: Add option to limit fields returned f...

2017-08-03 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/677#discussion_r131133566 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -81,8 +82,14 @@ public

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

2017-08-15 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/681 Yeah, it's true, I suppose having `==` double as a `NaN` check is probably not the right thing due to the transitivity of equality (I'd like to `SQRT(-1) == NaN` to be `true` but `SQRT(-1) == SQRT

[GitHub] metron issue #686: METRON-711 STELLAR SHELL Do not set variables when except...

2017-08-15 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/686 +1 by inspection, this is good work! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] metron issue #691: METRON-1104: Build Failure - Due to Bootstrap version cha...

2017-08-11 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/691 I'm +1 on this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] metron issue #695: METRON-1108: Metron configuration tabs are not showing up...

2017-08-14 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/695 +1 by inspection; this one was my bad from an earlier PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

2017-08-15 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/681 Some of our math functions return `NaN` (e.g. `SQRT(-1)`) and being able to do `!= NaN` is a useful thing. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

2017-08-15 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/681 Just thinking, we might want +Infinity and -Infinity too as they're part of the floating point standard. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] metron issue #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-15 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/667 I'm fine with this so far, but will hold for @nickwallen 's +1. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...

2017-08-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Yeah, it seems to me that we might need something like a location for discussion and description of architectural decisions made. --- If your project is set up for it, you can reply to this email

[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...

2017-08-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Ah, for the moment we only have the StixExtractor. We could MAKE a cybox extractor and it could reuse the handler logic, but we haven't done that. --- If your project is set up for it, you can

[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...

2017-08-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Ok, so good questions @ottobackwards . I'll do my best to answer them, but the answer to some of these expands past this PR and to the history of Taxii support for Metron (which was one

[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...

2017-08-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/689 Sure, not a problem. I noted the versions supported in the documentation and linked to the actual schemata for cybox and stix. --- If your project is set up for it, you can reply to this email

[GitHub] metron pull request #689: METRON-1102: Add support for ingesting cybox URI o...

2017-08-10 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/689#discussion_r132481074 --- Diff: metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java --- @@ -38,6 +39,7

[GitHub] metron issue #690: METRON-1091 Package STELLAR shell as stand alone

2017-08-14 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/690 I just want to make sure that we didn't break backwards compatibility and have the shell not deployed as part of the mpack, I guess ;) --- If your project is set up for it, you can reply

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 I just want to follow-up with something a bit more concrete suggestions. I think the beginnings of an abstraction are there. You pulled out a bunch of utility methods from `ElasticsearchWriter

[GitHub] metron issue #651: METRON-1037 Added POWER function

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/651 @mattf-horton Actually, `**` and `math.exp()` are different; the first being an arbitrary power function and the later presuming that we're taking a power of `e` (i.e. `e**x == exp(x)` ). I did

[GitHub] metron issue #650: METRON-1038: Stellar should have a better collection of b...

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/650 Ok, I broke this abstraction out a bit and made it easier to add non-single-arg functions (a la POW, which is a separate PR by @simonellistonball ). I also went ahead and added `ROUND` and `EXP

[GitHub] metron issue #653: METRON-1040 Create Installation Instructions for the Prof...

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/653 Exactly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] metron issue #653: METRON-1040 Create Installation Instructions for the Prof...

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/653 Cool, can we get a blurb about adjusting configs to conform to your system, etc. at the end of this? If so, I'm +1. --- If your project is set up for it, you can reply to this email and have your

[GitHub] metron pull request #650: METRON-1038: Stellar should have a better collecti...

2017-07-13 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/650#discussion_r127242712 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/MathFunctions.java --- @@ -25,17 +25,39 @@ import

[GitHub] metron issue #651: METRON-1037 Added POWER function

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/651 Yeah, that's an argument I was expecting. I'm torn. Yes, I'd prefer ^ or **, but we're following java examples (except where we're following python examples ;). I don't know, I'm torn and could

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 Bear with me, @merrimanr, I am going to submit a PR with the DAO abstraction I was talking about so we can hash it out. I started it in collaboration with @justinleet to ensure the ideas

[GitHub] metron pull request #652: METRON-1039: Add ZIP function to Stellar

2017-07-13 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/652#discussion_r127227967 --- Diff: metron-stellar/stellar-common/README.md --- @@ -711,6 +713,18 @@ In the core language functions, we support basic functional programming primitiv

[GitHub] metron pull request #652: METRON-1039: Add ZIP function to Stellar

2017-07-13 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/652#discussion_r127247191 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/FunctionalFunctionsTest.java --- @@ -24,13 +24,124

[GitHub] metron pull request #650: METRON-1038: Stellar should have a better collecti...

2017-07-13 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/650#discussion_r127242100 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/MathFunctions.java --- @@ -25,17 +25,39 @@ import

[GitHub] metron pull request #651: METRON-1037 Added POWER function

2017-07-13 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/651#discussion_r127263511 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/MathFunctions.java --- @@ -60,4 +60,38 @@ public boolean

[GitHub] metron issue #649: METRON-1035 Added SUM to the rules triage aggregation doc...

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/649 +1 by inspection, nice work --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] metron issue #642: METRON-984 Stellar functions to decode encoded fields or ...

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/642 Ok, I dig this a lot. +1 by inspection. The only thing I ask is that since you're adding a new dependency, would you ensure a smoketest in full-dev and maybe pass through a couple of records

[GitHub] metron issue #621: METRON-1001: Allow metron to ingest parser metadata along...

2017-07-11 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/621 @justinleet any other comments or concerns? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] metron pull request #643: METRON-1026: threatintel_taxii_load.sh throws exce...

2017-07-10 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/643 METRON-1026: threatintel_taxii_load.sh throws exception ## Contributor Comments The delegation wrapping done to enable Stellar transformations on data as it is imported broke the taxii loader

[GitHub] metron pull request #621: METRON-1001: Allow metron to ingest parser metadat...

2017-07-10 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/621#discussion_r126377527 --- Diff: metron-platform/metron-storm-kafka/src/main/java/org/apache/metron/storm/kafka/flux/SimpleStormKafkaBuilder.java --- @@ -176,16 +182,48

[GitHub] metron issue #520: METRON-833: Update MaaS documentation to explain how it i...

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/520 Deconflicted; thanks for the patience. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] metron pull request #550: METRON-890: Intermittent unit test errors in shutt...

2017-07-07 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] metron issue #639: METRON-1013 add command line verification to stellar shel...

2017-07-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/639 +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] metron pull request #650: METRON-1038: Stellar should have a better collecti...

2017-07-13 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/650 METRON-1038: Stellar should have a better collection of basic math operations ## Contributor Comments At the moment the math functions are woefully incomplete. We should add at least

[GitHub] metron issue #652: METRON-1039: Add ZIP function to Stellar

2017-07-18 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/652 @jjmeyer0 I don't have a problem with it, but can you cite any previous work (i.e. another language) that works that way? The question I would have is, should each arg be exploded as separate args

[GitHub] metron issue #652: METRON-1039: Add ZIP function to Stellar

2017-07-18 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/652 Just a follow-up to `ZIP_WITH` with a bit more context, it appears that it's in haskell and scala was going to add it (see discussion [here](https://issues.scala-lang.org/browse/SI-1512

[GitHub] metron issue #599: METRON-975: Normalize logging and switch to common idiom ...

2017-07-18 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/599 I'm damned confused about these test errors. This PR didn't seem to touch anything near the test and I haven't seen that test (which is a mock test) fail intermittently before. I'd love a debrief

[GitHub] metron issue #641: METRON-539: added HASH function for stellar.

2017-07-18 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/641 I just want to chime in and say thanks for the contribution @jjmeyer0 !. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] metron issue #654: METRON-1044: Disabled writers are not acking messages

2017-07-18 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/654 +1 by inspection. This is a big miss when I added writer disabling, so thanks for the cleanup @merrimanr . As a follow-up, we might consider turning acking on by default in the indexing topology

[GitHub] metron issue #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/608 That sounds good; can you create a follow-on JIRA to investigate how to make the properties specified at run-time rather than as macros in code? --- If your project is set up for it, you can reply

[GitHub] metron issue #608: METRON-986 Enhance Fastcapa to Support Intel X520

2017-07-07 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/608 +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 This is good work! Thanks, Ryan. We desperately need this abstraction. I think the confusion may be that this PR is the general abstraction as well as a concrete implementation for one index. I

[GitHub] metron pull request #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-25 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/622#discussion_r129252984 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/hbase/DecodableRowKeyBuilder.java --- @@ -0,0 +1,402

[GitHub] metron pull request #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-25 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/622#discussion_r129252678 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/hbase/DecodableRowKeyBuilder.java --- @@ -0,0 +1,402

[GitHub] metron pull request #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-25 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/622#discussion_r129252112 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/hbase/DecodableRowKeyBuilder.java --- @@ -0,0 +1,402

[GitHub] metron issue #650: METRON-1038: Stellar should have a better collection of b...

2017-07-25 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/650 Any comments or review here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] metron issue #451: METRON-157: Added CEF Parser

2017-07-25 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/451 Hey @Ravi0204 you might have better luck sending an email to the Metron user@ list with that question. This PR is closed and the range of people paying attention to it is constrained

[GitHub] metron issue #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-25 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/622 @mattf-horton Would this approach require scans on read in the critical path? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] metron issue #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-25 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/622 @mattf-horton Wouldn't you have to use the serial number to retrieve profiles? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] metron pull request #673: METRON-1069: Ambari MPack documentation around dev...

2017-07-27 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/673#discussion_r129865954 --- Diff: metron-deployment/packaging/ambari/README.md --- @@ -0,0 +1,86 @@ +# Ambari Management Pack Development +Typically, Ambari Management Pack

[GitHub] metron pull request #673: METRON-1069: Ambari MPack documentation around dev...

2017-07-27 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/673#discussion_r129871944 --- Diff: metron-deployment/packaging/ambari/README.md --- @@ -0,0 +1,86 @@ +# Ambari Management Pack Development +Typically, Ambari Management Pack

[GitHub] metron issue #673: METRON-1069: Ambari MPack documentation around developmen...

2017-07-27 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/673 Also, I'd like to see a small section about best practices that people should consider when they're adding properties, such as: * If you're adding a new table or topic based on your property

[GitHub] metron pull request #673: METRON-1069: Ambari MPack documentation around dev...

2017-07-27 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/673#discussion_r129865653 --- Diff: metron-deployment/packaging/ambari/README.md --- @@ -0,0 +1,86 @@ +# Ambari Management Pack Development +Typically, Ambari Management Pack

[GitHub] metron pull request #643: METRON-1026: threatintel_taxii_load.sh throws exce...

2017-07-26 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/643#discussion_r129532376 --- Diff: metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiLoader.java --- @@ -165,6 +167,19 @@ public

[GitHub] metron issue #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/622 Also, while we're in here, is there a strong reason why the prefixed hash is so large? It's just there for uniformity of distribution, correct? I'd propose a non-cryptographic hash

[GitHub] metron issue #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/622 So, in my mind the feature here is the enablement of batch analytics on the profiles. To that end, I'm in general in favor of a decodable row key. I think that the question really isn't a ToC

[GitHub] metron issue #614: METRON-992: Create performance tuning guide

2017-07-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/614 +1 by inspection; this is right on the money and a good first pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] metron issue #622: METRON-1005 Create Decodable Row Key for Profiler

2017-07-20 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/622 I want to point out that I am also in favor of an audit log for the profiler, but I don't think it's a complete solution for the batch analytics use-case. --- If your project is set up

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-19 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 Ok, this appears to me like good infrastructure to base further index-oriented REST calls atop of. I'm +1; good job, @merrimanr ! --- If your project is set up for it, you can reply

[GitHub] metron pull request #624: [DO NOT MERGE] METRON-1004: Travis CI - Job Exceed...

2017-06-29 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/624#discussion_r124921605 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/KafkaControllerIntegrationTest.java --- @@ -148,6 +150,15 @@ public

  1   2   3   4   5   6   7   8   9   >