[GitHub] metron issue #689: METRON-1102: Add support for ingesting cybox URI observab...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/689 So, I think this is a great addition, but I have some comments. - Where is the documentation for the version of Stix and the Version of Cybox metron supports? - How is the

[GitHub] metron pull request #689: METRON-1102: Add support for ingesting cybox URI o...

Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/689#discussion_r132347288 --- Diff: metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java --- @@ -38,6 +39,7

[GitHub] metron pull request #689: METRON-1102: Add support for ingesting cybox URI o...

GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/689 METRON-1102: Add support for ingesting cybox URI observables from taxii feeds ## Contributor Comments There is value in ingesting URIs from taxii feeds and we should provide support to do so.

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 Test --- *Comments from [Reviewable](https://reviewable.io:443/reviews/apache/metron/530#-:-Kr8-4J5YPoUugdlItUi:bb74njr)* --- If your

Re: Profiler statistics NaN

A couple of things come to mind, in no order * Higher level compositing functions that bring all these things together… maybe packaged as a snazzy extension ;) * A more ‘structured’ version of the stellar shell text file input, where you could configure variables in the file and just run it, so

Re: Profiler statistics NaN

I like it, Otto. I see the recipe idea implemented as a collection of GUI wizards. The user can login to a web interface and choose from a collection of recipes. The user interacts with a recipe via a GUI wizard-like mechanism. The wizard gathers the input needed from the user to implement a

Re: Profiler statistics NaN

Well, we need that too :) What're you thinking, procedures for stellar? On Wed, Aug 9, 2017 at 4:42 PM, Otto Fowler wrote: > What we need, is a way to package up some ‘recipes’ for stellar. > If many people are going to do this operation, then a more friendly set of >

Re: Profiler statistics NaN

What we need, is a way to package up some ‘recipes’ for stellar. If many people are going to do this operation, then a more friendly set of facade functions, or some thing would work. On August 9, 2017 at 16:38:48, Casey Stella (ceste...@gmail.com) wrote: Yeah, I'm leaning toward STATS_ADD or

Re: Profiler statistics NaN

Yeah, I'm leaning toward STATS_ADD or STATS_INIT taking a list of numbers. STATS_MERGE seems confusing. On Wed, Aug 9, 2017 at 4:37 PM, Nick Allen wrote: > Or even change the behavior of STATS_MERGE, too? If STATS_MERGE gets raw > numbers, it wraps those in a Stats object,

Re: Profiler statistics NaN

Or even change the behavior of STATS_MERGE, too? If STATS_MERGE gets raw numbers, it wraps those in a Stats object, then returns it. Then Dima's example would just work as-is. I'm not sure I like that though. Maybe so flexible as to be confusing? Thought I would throw it out as an alternative

Re: Profiler statistics NaN

Oh yeah, duh. Now I'm with you. That would be a good quick hit. The current behavior is a little nutty. If there is a list, it only consumes the first element in the list. I'd expect that it should either do what you describe or complain that it doesn't know how to handle a list. Easy fix

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 Yes, and there were not a lot in NAR to start with --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/530 Sure, no worries. And I didn't intend to imply that testing was inadequate, just suggesting another for completeness. Can't have too many tests :-) --- If your project is set up for it, you

Re: Profiler statistics NaN

outcoming is still a HLLP object, not a statistics object, so doing a STATS_MERGE on a bunch of them wouldn't work either. On Wed, Aug 9, 2017 at 4:15 PM, Nick Allen wrote: > That is another problem. Isn't the simplest answer, to just change this... > > "result":

Re: Profiler statistics NaN

That is another problem. Isn't the simplest answer, to just change this... "result": "HLLP_CARDINALITY(outcoming)" to this... "result": "outcoming" ? On Wed, Aug 9, 2017 at 3:48 PM Casey Stella wrote: > Ok, so the problem here is that your profile is returning integers

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 Yes, I am sorry, I just wanted to point out another test, outside the area that you are currently looking. I did not mean to imply that it negated the need for 1099. Although, if we

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/530 @ottobackwards , re `metron-parser-bundle-tests`, very good to have that test. But it only loads one test bundle, right? so still would be good to implement METRON-1099. Emphasizing that

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 @mmiklavc The documentation also changes with METRON-942, as that includes the REST installation steps. If we can get these two PR's through, then follow on with improved docs, it may make

Re: Profiler statistics NaN

Ok, so the problem here is that your profile is returning integers (specifically HLLP cardinalities) rather than stats objects. When you're doing: STATS_PERCENTILE(STATS_MERGE( PROFILE_GET('host-talks-to', '99.191.183.156', PROFILE_FIXED(10, 'HOURS')), 90) You are calling STATS_MERGE on a

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 @mmiklavc can the new document be a follow on? The jira with your writeup would be a good one. --- If your project is set up for it, you can reply to this email and have your reply appear on

Re: Profiler statistics NaN

It seems that you are using the Profiler Client API correctly from the REPL, but you are using it incorrectly in your triage rules. Change your triage rules to match what you ran in the REPL. Correct: PROFILE_GET( "host-talks-to" , "99.191.183.156", PROFILE_FIXED(300, "MINUTES")) Incorrect:

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 @mmiklavc "useful and overwhelming at the same time". If I had a nickel™. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/530 @mattf-horton I don't know if you have seen it, but there is an integration test that tests parser but ensures that the bundle is loaded and not in the default classloader...

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132284538 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/bundle/BundleDetails.java --- @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/530 @ottobackwards , the \@VisibleForTesting annotation comes from: > import com.google.common.annotations.VisibleForTesting; which I believe comes from ```xml 18.0

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132280796 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/bundle/BundleCoordinates.java --- @@ -0,0 +1,93 @@ +/* + * Licensed to the

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132279911 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/bundle/BundleDetails.java --- @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache

Profiler statistics NaN

Hello Metron Team, I have created following profiler: > { > "profile": "host-talks-to", > "onlyif": "exists(source_ip)", > "foreach": "source_ip", > "init": { > "outcoming": "HLLP_INIT(5, 6)" > }, > "update": { "outcoming": "HLLP_ADD(outcoming, destination_ip)" }, >

[GitHub] metron issue #530: METRON-777 Metron Extension System and Parser Extensions

Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/530 I've opened METRON-1099 for integration tests regarding the two items I'm not sure from code inspection will work right. But I'm not making this review dependent on them because they are a

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/620 +1 nice work @iraghumitra --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132263102 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/BundleClassLoaders.java --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132261444 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/BundleClassLoaders.java --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132258789 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/bundle/BundleDetails.java --- @@ -0,0 +1,189 @@ +/* + * Licensed to the Apache

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132255328 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/bundle/Bundle.java --- @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software

[GitHub] metron pull request #530: METRON-777 Metron Extension System and Parser Exte...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/metron/pull/530#discussion_r132254112 --- Diff: bundles-lib/src/main/java/org/apache/metron/bundles/util/FileSystemManagerFactory.java --- @@ -0,0 +1,99 @@ +/** + * Licensed to the

[GitHub] metron issue #685: METRON-1087: Adjust license headers to be comments instea...

Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/685 +1 per inspection, once merge conflicts are resolved. I didn't pour over each file, but I did at least scan every single change and it looks good. Clicking the "load diff" option repeatedly was

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/620 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] metron issue #580: METRON-942 [NO MERGE UNTIL METRON-777] Rest api and confi...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/580 Maintaining METRON-947 is too much of a pain. I have merged it into this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

Github user simonellistonball commented on the issue: https://github.com/apache/metron/pull/620 +1 I'm good with this. My one niggle will be dealt with by other follow on issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] metron issue #620: Metron-988: UI for viewing alerts generated by Metron

Github user cestella commented on the issue: https://github.com/apache/metron/pull/620 +1 by inspection, great job @iraghumitra --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature