[jira] [Created] (METRON-2327) SOLR Time routed alias support
Dale Richardson created METRON-2327: --- Summary: SOLR Time routed alias support Key: METRON-2327 URL: https://issues.apache.org/jira/browse/METRON-2327 Project: Metron Issue Type: Improvement Reporter: Dale Richardson Assignee: Dale Richardson Recent versions of SOLR provide for excellent eventTime based collection creation and retirement. This provides for automatically creating collections for each time 'bin', dynamically routing documents tot he correct collection based on event time, and for (optionally) removing collections once they get to a certain age. For running a SOLR-based Metron deploy in production, this will reduce admin overhead greatly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (METRON-2312) Solr collection create/delete scripts assume they are not in a chrooted environment by default
[ https://issues.apache.org/jira/browse/METRON-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dale Richardson updated METRON-2312: Comment: was deleted (was: Will do in the future. ) > Solr collection create/delete scripts assume they are not in a chrooted > environment by default > -- > > Key: METRON-2312 > URL: https://issues.apache.org/jira/browse/METRON-2312 > Project: Metron > Issue Type: Bug >Reporter: Dale Richardson >Assignee: Dale Richardson >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > When installing SOLR cloud, its been highly recommended to use the cluster > zookeeper ensemble rather then installing your in own mini-zk cluster just > for SOLR. For the past several years, its been standard practice to use a > chrooted / namespaced environment for storing solr information in zookeeper. > The practical effects of this is to need to prepend '/solr' to any zookeeper > ensemble URLs. The use of chrooted zookeeper configurations is the default > in both lucidworks/HWX SOLR (from 4.0), and for Cloudera SOLR (not sure which > version but for many years). It has also been the documented recommendation > for Apache SOLR Cloud since approximately version 6.6. > End result is, if Metron is dealing with a SOLR cluster that has been > installed or updated any time in the past couple of years, it is dealing with > a SOLR configuration stored in a chrooted Zookeeper environment. > The problem is the Metron SOLR collection create/destroy scripts assume that > we are not using a CHROOTed environment, and fail badly when the expected > SOLR configuration is not present at the expected location in SOLR. Buried > in the readme is instruction on how to set modify the zookeeper environment > variables before running the script to add chrooted address, and when the > scripts are used by Ambari, they are called using the correct chrooted quorum > URL, because there is a seperate configuration item that can be set to > indicate the chroot zookeeper address for SOLR. > Having just been burnt by this I think we should at least > # Cleanly catch the failure of the zkcli command in the collection scripts > when it queries for zookeeper state that is not present > # If the zkcli error is caught, make a suggestion in the error message to > check for a chrooted SOLR cloud zookeeper configuration. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [metron] mmiklavc commented on issue #1570: METRON-2326 Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
mmiklavc commented on issue #1570: METRON-2326 Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field URL: https://github.com/apache/metron/pull/1570#issuecomment-557299193 One thing we should probably add here is a note in the documentation that says "while you can call ENRICHMENT_GET here... DO NOT CALL ENRICHMENT_GET HERE." i.e. ymmv - it's better to handle this in enrichment and use that field later. Reason being, I'm not sure we handle multi-threading in threat triage the way we do for normal enrichments, so there may be an adverse performance impact. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] nickwallen opened a new pull request #1570: METRON-2326 Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
nickwallen opened a new pull request #1570: METRON-2326 Unable to Call
ENRICHMENT_GET from Threat Triage Rule Reason Field
URL: https://github.com/apache/metron/pull/1570
A Threat Triage Rule's "reason" field can contain executable Stellar to
provide an operator context as to why a rule fired during Threat Triage. This
"reason" field is unable to call any function that requires a `StellarContext`
during initialization. For example, I cannot call the function
`ENRICHMENT_GET`.
## Root Cause
There is some additional Stellar validation performed on a threat triage
rule's reason and rule field in `ThreatTriageConfig.setRiskLevelRules`. In
this validation, no `StellarContext` is initialized which causes the
`ENRICHMENT_GET` function to fail as described.
## The Fix
I have found that the additional validation in
`ThreatTriageConfig.setRiskLevelRules` is not necessary, nor useful, for
validating a rule's reason or rule field.
This validation is incorrectly throwing exceptions in cases like this where
the user has provided a valid Stellar expression. This is because the Stellar
execution environment is not configured and initialized in the same manner as
the one used to actually run the Threat Triage rules at run-time.
Even if we corrected this, I have found that there are no error conditions
that this validation would catch that would not be caught by validation
performed elsewhere.
1. Any error condition discoverable at 'compile-time' is checked before the
triage rule is loaded into Zk. If the user attempts to load invalid rules
using the REPL or `zk_load_configs`, all 'compile-time' errors that
`ThreatTriageConfig.setRiskLevelRules` might catch, are already caught.
For example, if the "reason" expression calls a non-existent function or
uses invalid syntax, an exception will be thrown and the rule will not load.
2. Any other 'run-time' errors can only be checked when a message's fields
are available to the expression. These sorts of errors would not be caught by
the validation in `ThreatTriageConfig.setRiskLevelRules` and instead will be
caught by the `ThreatTriageProcessor`.
For example, if the "reason" expression refers to a field that does not
exist or attempts to divide by a field that has a value of 0.
## Changes
1. Remove the validation of Stellar expressions from
`ThreatTriageConfig.setRiskLevelRules`.
## Acceptance Testing
1. Spin-up the development environment.
1. Turn off all sensor-stubs except snort.
```
service sensor-stubs stop
service sensor-stubs start snort
```
1. Create a file called `user.csv`.
```
[root@node1 ~]# cat user.csv
jdoe,192.168.138.2
jane,192.168.66.1
ciana,192.168.138.158
danixa,95.163.121.204
jim,192.168.66.121
```
1. Create a file called `user-extractor.json`.
```
{
"config": {
"columns": {
"user": 0,
"ip": 1
},
"indicator_column": "ip",
"separator": ",",
"type": "user"
},
"extractor": "CSV"
}
```
1. Import the data that we will use for enrichment.
```
source /etc/default/metron
$METRON_HOME/bin/flatfile_loader.sh -i ./user.csv -t enrichment -c t -e
./user-extractor.json
```
1. Validate that the enrichment loaded successfully.
```
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
[Stellar]>>> ip_dst_addr := "192.168.138.2"
192.168.138.2
[Stellar]>>> ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't')
{ip=192.168.138.2, user=jdoe}
```
1. Create a threat triage rule that attempts an ENRICHMENT_GET.
```
[Stellar]>>> conf := SHELL_EDIT()
{
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "true"
}
}
},
"fieldToTypeMap": {},
"config": {}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {},
"config": {},
"triageConfig": {
"riskLevelRules": [
{
"name": "Rule1",
"comment": "This rule should work.",
"rule": "true",
"reason": "FORMAT('Call to ENRICHMENT_GET=%s',
ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't'))",
"score": "100"
}
],
"aggregator": "MAX",
"aggregationConfig": {}
}
},
"configuration": {}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", conf, "snort")
```
[jira] [Updated] (METRON-2326) Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
[
https://issues.apache.org/jira/browse/METRON-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Allen updated METRON-2326:
---
Summary: Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
(was: Unable to Call ENRICHMENT_GET from Threat Triage Rule 'Reason' Field)
> Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
> --
>
> Key: METRON-2326
> URL: https://issues.apache.org/jira/browse/METRON-2326
> Project: Metron
> Issue Type: Bug
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
>
> A Threat Triage Rule's "reason" field can contain executable Stellar to
> provide an operator context as to why a rule fired during Threat Triage. I
> am unable to call any function that requires a StellarContext during
> initialization, from the 'Reason' field of a Threat Triage Rule. For
> example, I cannot call `ENRICHMENT_GET`.
> h3. Steps to Replicate
> 1. Create a simple file called `user.csv`.
> {code:java}
> [root@node1 ~]# cat user.csv
> jdoe,192.168.138.2
> jane,192.168.66.1
> ciana,192.168.138.158
> danixa,95.163.121.204
> jim,192.168.66.121
> {code}
> 2 . Create a file called `user-extractor.json`.
> {code:java}
> {
> "config": {
> "columns": {
> "user": 0,
> "ip": 1
> },
> "indicator_column": "ip",
> "separator": ",",
> "type": "user"
> },
> "extractor": "CSV"
> }
> {code}
> 3. Import the enrichment data.
> {code:java}
> source /etc/default/metron
> $METRON_HOME/bin/flatfile_loader.sh -i ./user.csv -t enrichment -c t -e
> ./user-extractor.json
> {code}
> 4. Validate that the enrichment loaded successfully.
> {code:java}
> [root@node1 0.7.2]# source /etc/default/metron
> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>
> [Stellar]>>> ip_dst_addr := "192.168.138.2"
> 192.168.138.2
>
> [Stellar]>>> ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't')
> \{ip=192.168.138.2, user=jdoe}
> {code}
> 5. Create a threat triage rule that attempts an ENRICHMENT_GET.
> {code}
> [Stellar]>>> conf := SHELL_EDIT()
> {
> "enrichment": {
> "fieldMap": {
> "stellar": {
> "config": {
> "is_alert": "true"
> }
> }
> },
> "fieldToTypeMap": {},
> "config": {}
> },
> "threatIntel": {
> "fieldMap": {},
> "fieldToTypeMap": {},
> "config": {},
> "triageConfig": {
> "riskLevelRules": [
> {
> "name": "Rule",
> "comment": "This rule does not work when executing the 'reason' field.",
> "rule": "true",
> "reason": "FORMAT('Call to ENRICHMENT_GET=%s', ENRICHMENT_GET('user',
> ip_dst_addr, 'enrichment', 't'))",
> "score": "100"
> }
> ],
> "aggregator": "MAX",
> "aggregationConfig": {}
> }
> },
> "configuration": {}
> }
>
> [Stellar]>>> CONFIG_PUT("ENRICHMENT", conf, "snort")
> {code}
>
> 6. The Storm worker logs for Enrichment show the following error.
> {code:java}
> 2019-11-21 03:54:34.370 o.a.c.f.r.c.TreeCache Curator-TreeCache-4 [ERROR]
> org.apache.metron.jackson.databind.JsonMappingException: Unable to find
> capability GLOBAL_CONFIG; it may not be available in your context.
> at [Source: java.io.ByteArrayInputStream@1f55bdda; line: 24, column: 11]
> (through reference chain:
> org.apache.metron.common.configuration.enrichment.SensorEnrichmentConfig["threatIntel"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatIntelConfig["triageConfig"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatTriageConfig["riskLevelRules"])
> at
> org.apache.metron.jackson.databind.JsonMappingException.from(JsonMappingException.java:262)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:537)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:518)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
>
[jira] [Created] (METRON-2326) Unable to Call ENRICHMENT_GET from Threat Triage Rule 'Reason' Field
Nick Allen created METRON-2326:
--
Summary: Unable to Call ENRICHMENT_GET from Threat Triage Rule
'Reason' Field
Key: METRON-2326
URL: https://issues.apache.org/jira/browse/METRON-2326
Project: Metron
Issue Type: Bug
Reporter: Nick Allen
Assignee: Nick Allen
A Threat Triage Rule's "reason" field can contain executable Stellar to provide
an operator context as to why a rule fired during Threat Triage. I am unable
to call any function that requires a StellarContext during initialization, from
the 'Reason' field of a Threat Triage Rule. For example, I cannot call
`ENRICHMENT_GET`.
h3. Steps to Replicate
1. Create a simple file called `user.csv`.
{code:java}
[root@node1 ~]# cat user.csv
jdoe,192.168.138.2
jane,192.168.66.1
ciana,192.168.138.158
danixa,95.163.121.204
jim,192.168.66.121
{code}
2 . Create a file called `user-extractor.json`.
{code:java}
{
"config": {
"columns": {
"user": 0,
"ip": 1
},
"indicator_column": "ip",
"separator": ",",
"type": "user"
},
"extractor": "CSV"
}
{code}
3. Import the enrichment data.
{code:java}
source /etc/default/metron
$METRON_HOME/bin/flatfile_loader.sh -i ./user.csv -t enrichment -c t -e
./user-extractor.json
{code}
4. Validate that the enrichment loaded successfully.
{code:java}
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
[Stellar]>>> ip_dst_addr := "192.168.138.2"
192.168.138.2
[Stellar]>>> ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't')
\{ip=192.168.138.2, user=jdoe}
{code}
5. Create a threat triage rule that attempts an ENRICHMENT_GET.
{code}
[Stellar]>>> conf := SHELL_EDIT()
{
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "true"
}
}
},
"fieldToTypeMap": {},
"config": {}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {},
"config": {},
"triageConfig": {
"riskLevelRules": [
{
"name": "Rule",
"comment": "This rule does not work when executing the 'reason' field.",
"rule": "true",
"reason": "FORMAT('Call to ENRICHMENT_GET=%s', ENRICHMENT_GET('user',
ip_dst_addr, 'enrichment', 't'))",
"score": "100"
}
],
"aggregator": "MAX",
"aggregationConfig": {}
}
},
"configuration": {}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", conf, "snort")
{code}
6. The Storm worker logs for Enrichment show the following error.
{code:java}
2019-11-21 03:54:34.370 o.a.c.f.r.c.TreeCache Curator-TreeCache-4 [ERROR]
org.apache.metron.jackson.databind.JsonMappingException: Unable to find
capability GLOBAL_CONFIG; it may not be available in your context.
at [Source: java.io.ByteArrayInputStream@1f55bdda; line: 24, column: 11]
(through reference chain:
org.apache.metron.common.configuration.enrichment.SensorEnrichmentConfig["threatIntel"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatIntelConfig["triageConfig"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatTriageConfig["riskLevelRules"])
at
org.apache.metron.jackson.databind.JsonMappingException.from(JsonMappingException.java:262)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:537)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:518)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
~[stormjar.jar:?]
at
org.apache.metron.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3807)
~[stormjar.jar:?]
at
[GitHub] [metron] justinleet commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
justinleet commented on a change in pull request #1554: METRON-2307: Migrate to
JUnit5
URL: https://github.com/apache/metron/pull/1554#discussion_r349243107
##
File path:
metron-platform/metron-solr/metron-solr-common/src/test/java/org/apache/metron/solr/dao/SolrSearchDaoTest.java
##
@@ -255,11 +210,11 @@ public void getAllLatestShouldProperlyReturnDocuments()
throws Exception {
SolrDocumentList snortList = new SolrDocumentList();
snortList.add(snortSolrDoc1);
snortList.add(snortSolrDoc2);
-when(client.getById((Collection) argThat(hasItems("bro-1",
"bro-2")),
+when(client.getById((List)
org.mockito.hamcrest.MockitoHamcrest.argThat(hasItems("bro-1", "bro-2")),
Review comment:
Unfortunately, I don't believe so. The `hasItems` doesn't play nice with the
plain `argThat` interfaces, which necessitates using the `MockitoHamcrest` in
this case. I believe this is only instance of this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [metron] justinleet commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
justinleet commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5 URL: https://github.com/apache/metron/pull/1554#discussion_r349243139 ## File path: metron-platform/metron-solr/metron-solr-common/pom.xml ## @@ -199,10 +199,15 @@ + +org.hamcrest Review comment: No, it's just an artifact of debugging. I double checked by moving it back to the original spot and the tests pass at the `metron-solr` level, not just `metron-solr-common`. I haven't pushed that change up, but I have no problem making it if you want. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] justinleet commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
justinleet commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5 URL: https://github.com/apache/metron/pull/1554#discussion_r349243040 ## File path: pom.xml ## @@ -111,21 +111,21 @@ 5.6.14 1.1.1 3.0.2 -4.12 +5.5.2 +2.2 17.0 12.0 2.2.5 1.7.7 3.7 1.8 6.6.2 -1.10.19 -1.7.0 +3.1.0 3.2.0 2.7.4 2.0.14 3.0.2 -2.18 +3.0.0-M3 Review comment: Bumped it. Wasn't released when I originally moved it. Glancing through the list of Jiras (https://maven.apache.org/surefire/maven-surefire-plugin/jira-report.html), I don't see anything that leaps out at me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5 URL: https://github.com/apache/metron/pull/1554#discussion_r349218988 ## File path: pom.xml ## @@ -111,21 +111,21 @@ 5.6.14 1.1.1 3.0.2 -4.12 +5.5.2 +2.2 17.0 12.0 2.2.5 1.7.7 3.7 1.8 6.6.2 -1.10.19 -1.7.0 +3.1.0 3.2.0 2.7.4 2.0.14 3.0.2 -2.18 +3.0.0-M3 Review comment: Should we consider M4? https://maven.apache.org/surefire/maven-surefire-plugin/index.html > Provided 3 extensions of reporters which can be used to customize XML report, console and file reporters. It is very useful for JUnit5 users. > We reworked the internal implementation so that new commands and events can be easily added. The impl is located in a center point and it is a prerequisite in next versions. > Provided bug fixes for Docker Alpine/BusyBox Linux, JUnit5 and 43 more. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5 URL: https://github.com/apache/metron/pull/1554#discussion_r349192630 ## File path: metron-platform/metron-solr/metron-solr-common/pom.xml ## @@ -199,10 +199,15 @@ + +org.hamcrest Review comment: Is this re-ordering significant? Should we add a comment? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to JUnit5
mmiklavc commented on a change in pull request #1554: METRON-2307: Migrate to
JUnit5
URL: https://github.com/apache/metron/pull/1554#discussion_r349194570
##
File path:
metron-platform/metron-solr/metron-solr-common/src/test/java/org/apache/metron/solr/dao/SolrSearchDaoTest.java
##
@@ -255,11 +210,11 @@ public void getAllLatestShouldProperlyReturnDocuments()
throws Exception {
SolrDocumentList snortList = new SolrDocumentList();
snortList.add(snortSolrDoc1);
snortList.add(snortSolrDoc2);
-when(client.getById((Collection) argThat(hasItems("bro-1",
"bro-2")),
+when(client.getById((List)
org.mockito.hamcrest.MockitoHamcrest.argThat(hasItems("bro-1", "bro-2")),
Review comment:
Not sure about the `MockitoHamcrest` matchers. Can we just use the static
import we already have? `org.mockito.ArgumentMatchers.argThat`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [metron] nickwallen closed pull request #1565: METRON-2321 Remove Legacy AWS Deployment Path
nickwallen closed pull request #1565: METRON-2321 Remove Legacy AWS Deployment Path URL: https://github.com/apache/metron/pull/1565 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] nickwallen commented on issue #1564: METRON-2285 Batch Profiler Cannot Persist Data Sketches
nickwallen commented on issue #1564: METRON-2285 Batch Profiler Cannot Persist Data Sketches URL: https://github.com/apache/metron/pull/1564#issuecomment-557138214 I added the promised test case. Unfortunately, this test case would not have caught this specific bug. I still added it in hopes of catching potential future problems. Now why can't our tests replicate this bug? It appears that the original problem is only triggered when running against the HDP-flavor of Spark and its dependencies. This is the "flavor" that you would get when running Metron in a cluster deployed with Ambari and the MPack. The issue cannot be replicated in the integration tests as they are run against the against the pure, open source version of Spark and its dependencies, which are slightly different. Now the only way I could really prove this theory is if I were to attempt to run all of our integration tests against the HDP-flavor of each of our dependencies. It seems like this would be a rather time-consuming task. It might be really useful to define a separate profile that builds against the HDP-flavor of dependencies that we could use to potentially catch issues like this, but it seems a bit overkill for this specific bug. I am less concerned with the tests here because the fix included in this PR actually removes some of the ugly scaffolding that I had to use (ProfileMeasurementAdapter) to get this working originally. I see this (along with #1556) as the approach I would have preferred to take originally had I been able to get it working at the time. Of course, I could be persuaded otherwise. Let me know if more due diligence is needed here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [metron] sardell commented on issue #1560: METRON-2316: [UI] Drag drop sorting for the selected fields in the Alerts UI
sardell commented on issue #1560: METRON-2316: [UI] Drag drop sorting for the selected fields in the Alerts UI URL: https://github.com/apache/metron/pull/1560#issuecomment-557107220 @ruffle1986 I opened a PR against your branch which adds keyboard accessibility to the up/down arrows. Let's keep them for now, and in the future, if we hear from the users via the mailing list or Slack that it's causing confusion or not helping, we can remove them (or hide them until triggered by a keyboard user navigating through the page). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (METRON-2320) [UI] metron-config npm install failing by copy-ace step
[
https://issues.apache.org/jira/browse/METRON-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979266#comment-16979266
]
Tibor Meller commented on METRON-2320:
--
The post-install script in our package.json file was investigated. All the
copied source files exitst and properly placed under node_modules. The copy
script was also tested and evaluated.
However, this should theoretically work perfectly fine every one out of five
execution the copy script stuck in and fail to copy ace editor plugin files.
This more likely to happen when the copy script already executed successfully
and the developer execute npm ci/install bc a new dependency or package upgrade.
*The issue does not impact travis bc it’s always run npm ci on a clean
workspace.*
> [UI] metron-config npm install failing by copy-ace step
> ---
>
> Key: METRON-2320
> URL: https://issues.apache.org/jira/browse/METRON-2320
> Project: Metron
> Issue Type: Bug
>Reporter: Tibor Meller
>Assignee: Tibor Meller
>Priority: Major
>
> The error message pasted below does not always show up. Depending on the npm
> versions the post-install script just gets stuck or exit with an error.
> Hence the error is caused by ace editor's post install script the install get
> finished properly.
>
> {color:#172b4d} metron-management-ui@0.7.2 postinstall
> D:\workspace\tiborm\metron\metron-interface\metron-config
> > npm run copy-ace & npm run copy-ace-snippets{color}{color:#00}
> {color}{color:#cc}> metron-management-ui@0.7.2 copy-ace
> D:\workspace\tiborm\metron\metron-interface\metron-config
> > cp -f $npm_package_config_node_ace/ext-language_tools.js
> $npm_package_config_node_ace/mode-json.js
> $npm_package_config_node_ace/theme-monokai.js
> $npm_package_config_node_ace/worker-json.js
> src/assets/ace{color}{color:#cc}cp: cannot stat
> '$npm_package_config_node_ace/ext-language_tools.js': No such file or
> directory
> cp: cannot stat '$npm_package_config_node_ace/mode-json.js': No such file or
> directory
> cp: cannot stat '$npm_package_config_node_ace/theme-monokai.js': No such
> file or directory
> cp: cannot stat '$npm_package_config_node_ace/worker-json.js': No such file
> or directory
> npm {color}{color:#c50f1f}ERR!{color}
> {color:#881798}code{color}{color:#cc} ELIFECYCLE
> npm {color}{color:#c50f1f}ERR!{color}
> {color:#881798}errno{color}{color:#cc} 1
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc}
> metron-management-ui@0.7.2 copy-ace: `cp -f
> $npm_package_config_node_ace/ext-language_tools.js
> $npm_package_config_node_ace/mode-json.js
> $npm_package_config_node_ace/theme-monokai.js $npm_package_config_node_ace/w
> orker-json.js src/assets/ace`
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc} Exit status 1
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc}npm
> {color}{color:#c50f1f}ERR!{color}{color:#cc} Failed at the
> metron-management-ui@0.7.2 copy-ace script.
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc} This is probably not a
> problem with npm. There is likely additional logging output
> above.{color}{color:#cc}npm
> {color}{color:#c50f1f}ERR!{color}{color:#cc} A complete log of this run
> can be found in:
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc}
> C:\Users\shua\AppData\Roaming\npm-cache_logs\2019-11-19T12_15_00_922Z-debug.log{color}{color:#cc}>
> metron-management-ui@0.7.2 copy-ace-snippets
> D:\workspace\tiborm\metron\metron-interface\metron-config
> > cp -f $npm_package_config_node_ace/snippets/text.js
> $npm_package_config_node_ace/snippets/json.js
> src/assets/ace/snippets{color}{color:#cc}cp: cannot stat
> '$npm_package_config_node_ace/snippets/text.js': No such file or directory
> cp: cannot stat '$npm_package_config_node_ace/snippets/json.js': No such
> file or directory
> npm {color}{color:#c50f1f}ERR!{color}
> {color:#881798}code{color}{color:#cc} ELIFECYCLE
> npm {color}{color:#c50f1f}ERR!{color}
> {color:#881798}errno{color}{color:#cc} 1
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc}
> metron-management-ui@0.7.2 copy-ace-snippets: `cp -f
> $npm_package_config_node_ace/snippets/text.js
> $npm_package_config_node_ace/snippets/json.js src/assets/ace/snippets`
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc} Exit status 1
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc}npm
> {color}{color:#c50f1f}ERR!{color}{color:#cc} Failed at the
> metron-management-ui@0.7.2 copy-ace-snippets script.
> npm {color}{color:#c50f1f}ERR!{color}{color:#cc} This is probably not a
> problem with npm. There is likely additional logging output above.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
