Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/946
@ottobackwards in it's current state, sort of, but you're not required to
turn it on. In the desired (reflection based nifi style state) no, it should
load it and use it if present
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/946#discussion_r173416554
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/metron_service.py
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/945
To be fair, my question is probably just as appropriate on a discuss thread
and a separate ticket out of said thread if it comes to it.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/955
Have the dependencies, License and Notices files been updated?
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/953
Have the dependencies, License and Notices files been updated?
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/952
Have the dependencies, License and Notices files been updated?
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/942
I would say performance trumps complexity of functionality here.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/946
Should we consider a dual client in the same project similar to the
approach in
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-5
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/945
Are we losing anything by moving the scheme from Range to Trie types?,
repeating my comment on
https://github.com/apache/metron/pull/922:
Given that our use case is heavily
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/579
I don't believe I have any further things to add. I'm +1 and keen to see
this get in.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/895
One option here, that would make me less grumpy would be to incorporate the
acl actions with the topic creation actions, which at least prevents nefarious
insiders from using this endpoint
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/934
Given that we're happy to accept limiting ourselves to a Hadoop
distribution, it doesn't seem unfair to limit ourselves to a Solr distribution.
While sticking pure raw Apache has some
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/895
Ok, so we have some authentication, with clear text passwords, but we don't
have any authorization on the end points, which causes compliance issues with
things like access change request.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/895
-1 (non-binding) This is a pen-tester's dream. We currently have no
authentication around this endpoint, and allowing it to actually set acls make
it a serious security hole. That may
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/922
@cestella much neater. Thank you! I'll put my data schema OCD away now.
---
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/922#discussion_r166908797
--- Diff:
metron-platform/metron-solr/src/main/config/schema/error/solrconfig.xml ---
@@ -0,0 +1,1601 @@
+
--- End diff
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/914
My bad: the dependencies file is updated, but not the NOTICES.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/914
And in fairness to you, we don't do the NOTICES properly for anything else
either I see. That's probably a separate scope.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/914
And in more topical an relevant points... we're introducing a net new
dependency here... Do we need to also update the NOTICES file (and the
dependencies_with_url.csv list)?
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/914
@ottobackwards Really want that myth to be a reality... as soon as we can
get the config overwrite problem solved... but this really isn't the ticket to
discuss that on :)
I'm
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/922
Should we tidy up the ordering of the schema files for better legibility
(uniquekey next to the field, dynamic catch alls in a consistent location, some
semantic ordering of the key
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/915
Does anyone anticipate using (or even use) these fields to do things like
monitor the flow and latency of their pipelines? I can imagine people wanting
to build data quality monitoring
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/873
One question: why %timing? I would vote for %time, but that might just be
because I use too much jupyter and like the consistency. Am I missing another
reference for the choice of timing?
---
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/867#discussion_r156794990
--- Diff: metron-analytics/metron-statistics/README.md ---
@@ -53,6 +53,32 @@ functions can be used from everywhere where Stellar is
used
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/867#discussion_r156791019
--- Diff:
metron-analytics/metron-statistics/src/main/java/org/apache/metron/statistics/sampling/UniformSampler.java
---
@@ -0,0 +1,91
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/863#discussion_r156676868
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
---
@@ -229,17 +239,30 @@ public
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
Good catch, my test included the source.type in the select list, and
generalised badly in the instructions. We should include the source.type in the
protected system fields, let me update
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
Actually, to follow up on that... I have a proxy feed, and some proxy use
cases (enrichment, profile, etc). I want to keep my data clean and be explicit
about which fields I pass on, so I
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
If they want to select most, and remove the ones they don't want, then I
would recommend using the remove transformation, or a set null in stellar.
Perhaps regex support might be a nice
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155698109
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -216,6 +216,23 @@ whenever `field2` exists and whose corresponding equal
to 'foo
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155659293
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/SelectTransformationTest.java
---
@@ -0,0
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
Right, that should cover it.
---
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155649024
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -216,6 +216,23 @@ whenever `field2` exists and whose corresponding equal
to 'foo
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155646268
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/field/transformation/SelectTransformation.java
---
@@ -0,0 +1,52
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155645745
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/SelectTransformationTest.java
---
@@ -0,0
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155645303
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -216,6 +216,23 @@ whenever `field2` exists and whose corresponding equal
to 'foo
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
It suddenly occurs to me that we should probably whitelist the
original_string and timestamp fields, so that these are always kept by this
transformation. Does that make sense?
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/861
Somehow I knew you were going to say something about docs will add.
---
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/861#discussion_r155641855
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/field/transformation/SelectTransformationTest.java
---
@@ -0,0
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/861
Implemented SELECT transformer to project fields from parser
## Contributor Comments
This is a simple PR to add FieldTransformation capabilities to the parsers
allowing basic
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/856
@cestella I would say that proposed validate function has to be very much
in a namespace. It feels like a name that would be much more useful for a
function replacing our current approach
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/800#discussion_r145197280
--- Diff: metron-interface/metron-rest/README.md ---
@@ -112,42 +112,42 @@ The following configures the application for MySQL:
1. Install
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/682
@ottobackwards I just re-ran this. We're not fixing versions in the UIs at
the moment, so I would expect this kind of minor variance. This is mainly to
ensure licenses are at least covered
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/788#discussion_r143470175
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
---
@@ -133,6 +173,40 @@ export class
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/788#discussion_r143158180
--- Diff:
metron-interface/metron-alerts/src/app/alerts/alert-details/alert-details.component.ts
---
@@ -133,6 +173,40 @@ export class
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/779#discussion_r141869253
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/RestExceptionHandler.java
---
@@ -45,4 +45,14 @@ private
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/771
@ottobackwards should we push that as a follow up issue for now rather than
expanding the scope of this PR?
---
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/779#discussion_r141844398
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/RestExceptionHandler.java
---
@@ -45,4 +45,14 @@ private
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/776
Yes, it is, not sure how that happened. Probably too many tabs.
---
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/775
Do we need to ensure we're picking this from the right repos? We should
probably use the nodesource rpm repo rather than relying on the centos build,
which is a very very old version
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/776
Do we need to ensure we're picking this from the right repos? We should
probably use the nodesource rpm repo rather than relying on the centos build,
which is a very very old version
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/711
@ottobackwards agreed, this is very separate from the management ui (won't
touch, or be used by anything in the management ui). Also agreed this is a
separate entity, but one
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/722
METRON-1139 Fixed service advisor profilerHost variable
## Contributor Comments
This is a simple variable mis-naming error which doesn't cause any
immediate problems, but prevents
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/711
I'd say the docs belong with the UI docs, since this is pretty much an
endpoint to drive the UI buttons, no?
---
If your project is set up for it, you can reply to this email and have
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/711#discussion_r135813488
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/AlertServiceImpl.java
---
@@ -0,0 +1,57
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/709#discussion_r134235349
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-profiler-env.xml
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/709#discussion_r134235333
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-profiler-env.xml
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/709#discussion_r134233234
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/addon-services/METRON/CURRENT/role_command_order.json
---
@@ -5,11
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/689#discussion_r132347288
--- Diff:
metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/extractor/stix/StixExtractor.java
---
@@ -38,6 +39,7
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/620
+1 I'm good with this. My one niggle will be dealt with by other follow on
issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/599
+1 (non-binding) the sooner we get this in the better. It has performance
benefits, and the longer we wait the more work and trouble it will create. No
reason not to get it done, lots
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/530
@ottobackwards I didn't steal _that_ much of it. Still it shouldn't cause a
problem, as long as we get all the updates since this PR started ported in.
That one has moved a bit from things
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/619
Seems like this is failing on some un-related temporary test failures. Can
we get Travis kicked, and see what's left to do on this?
---
If your project is set up for it, you can reply
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/643#discussion_r129352107
--- Diff:
metron-platform/metron-data-management/src/main/java/org/apache/metron/dataloads/nonbulk/taxii/TaxiiLoader.java
---
@@ -165,6 +167,19
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/650
I would like it to be committed to make it much easier for me to port my PR
to it so +1 (non-binding)
---
If your project is set up for it, you can reply to this email and have your
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/647#discussion_r127474849
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/StormCLIWrapper.java
---
@@ -75,37 +81,50 @@ public int
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/647#discussion_r127472746
--- Diff:
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/StormCLIWrapper.java
---
@@ -75,37 +81,50 @@ public int
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/620#discussion_r127438525
--- Diff:
metron-interface/metron-alerts/src/app/utils/elasticsearch-utils.ts ---
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/620#discussion_r127432908
--- Diff:
metron-interface/metron-alerts/src/app/service/elasticsearch-localstorage-impl.ts
---
@@ -0,0 +1,294 @@
+/**
+ * Licensed
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/651
Yes, I think I'll just refactor this around #650 when that's committed to
keep the workflow simple.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user simonellistonball closed the pull request at:
https://github.com/apache/metron/pull/651
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/651
I get the point @mattf-horton and thought about it, but went this way for
familiarity's sake, a lot of the likely authors of stellar statements are
security analysts who will be more
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/652#discussion_r127244811
--- Diff:
metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/FunctionalFunctionsTest.java
---
@@ -24,13 +24,124
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/652#discussion_r127239254
--- Diff:
metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/FunctionalFunctionsTest.java
---
@@ -24,13 +24,124
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/652#discussion_r127231248
--- Diff: metron-stellar/stellar-common/README.md ---
@@ -711,6 +713,18 @@ In the core language functions, we support basic
functional programming
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/651
METRON-1037 Added POWER function
## Contributor Comments
This is a quick addition to the Math functions. It may be worth revising
following the work @cestella did this morning
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/649
METRON-1035 Added SUM to the rules triage aggregation docs
## Contributor Comments
Quick doc fix verified against code.
## Pull Request Checklist
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/648
METRON-1033 Corrected profiler docs units on expires field
Minor change to update profiler docs
## Contributor Comments
[Please place any comments here. A description
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/599
Is it worth us getting this in sooner rather than later, before we get too
many other bits of logging that will need to be backported?
---
If your project is set up for it, you can reply
GitHub user simonellistonball reopened a pull request:
https://github.com/apache/metron/pull/617
METRON-996 Performance improvement for ASA parser
Moved the compilation of Grok into initialisation and created a map of Grok
instances for each ASA message type. No functional changes
Github user simonellistonball closed the pull request at:
https://github.com/apache/metron/pull/617
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/602
Is there anything preventing this getting merged?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user simonellistonball opened a pull request:
https://github.com/apache/metron/pull/617
METRON-996 Performance improvement for ASA parser
Moved the compilation of Grok into initialisation and created a map of Grok
instances for each ASA message type. No functional changes
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/614#discussion_r120961228
--- Diff: metron-platform/Performance-tuning-guide.md ---
@@ -0,0 +1,326 @@
+# Metron Performance Tunining Guide
+
+## Overview
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/530
Awesome, I guess this should be covered by the integration test suite as
well, which has been kept reasonably up to date with the recent changes.
Anything there you think might deserve
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/530
@ottobackwards there have been a number of tweaks to the parsers since this
first went in, do you anticipate any need to port any of those by hand, or will
this structure pick up changes
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/581#discussion_r118686928
--- Diff: metron-interface/metron-config/scripts/package.json ---
@@ -9,7 +9,8 @@
"http-proxy-middleware": "0.17.4"
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/581#discussion_r118387205
--- Diff: metron-interface/metron-config/scripts/package.json ---
@@ -9,7 +9,8 @@
"http-proxy-middleware": "0.17.4"
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/531
I'd love to see your bro PR expand for this @JonZeolla DHCP is a pretty key
source, and Bro is a great way to extract it from taps. Let me know if there is
anything I can do to help
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/586
Fantastic! Good to see more integration tests in there and great to see
additional data ingested. I'm +1 (non-binding) on this.
---
If your project is set up for it, you can reply
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/586
Looks like there are some good unit tests, though maybe not for every new
type you mention, or that could come in. Seems like decent coverage though. I
wonder, is it worth adding something
Github user simonellistonball closed the pull request at:
https://github.com/apache/incubator-metron/pull/582
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user simonellistonball reopened a pull request:
https://github.com/apache/incubator-metron/pull/582
METRON-948 Corrected license abbreviation in package.json file
## Contributor Comments
[Please place any comments here. A description of the problem/enhancement,
how
Github user simonellistonball commented on the issue:
https://github.com/apache/incubator-metron/pull/579
Yes, that makes sense, but does have some performance implications of
course. A single mapping would have much faster response, so I would question
the original approach
Github user simonellistonball commented on the issue:
https://github.com/apache/incubator-metron/pull/579
A number of the field name changes seem to depart from Metron conventions.
Is there a reason to change these from matching the metron ip_src_addr style
pattern?
---
If your
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/581#discussion_r115962297
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts
Github user simonellistonball commented on the issue:
https://github.com/apache/incubator-metron/pull/531
The Bro parsers is actually pretty generic, and will take whatever json bro
dumps out. From a quick inspection you should just need to configure the bro
instance to send out dhcp
Github user simonellistonball commented on the issue:
https://github.com/apache/incubator-metron/pull/531
As an alternative method for getting DHCP data out of pcap, you might
consider the existing Bro sensor, which essentially does what dhcpdump does,
but for a wider range
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/544#discussion_r112968303
--- Diff: metron-deployment/vagrant/full-dev-platform/README.md ---
@@ -13,7 +13,9 @@ Getting Started
The computer used to deploy
99 matches
Mail list logo
