[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[Otto Fowler (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596101#comment-15596101
 ] 

Otto Fowler commented on METRON-424:


[~cestella] I can't assign yet, can you assign this to yourself and close it?

> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
> Fix For: 0.2.2BETA
>
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582311#comment-15582311
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/303


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
> Fix For: 0.2.2BETA
>
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575540#comment-15575540
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
+1 on inspection


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572418#comment-15572418
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Go for it, man.  It's all yours. :)


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572412#comment-15572412
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
@ottobackwards Sure, why don't you start a DISCUSS thread and lay out how 
you envision that looking?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572409#comment-15572409
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I would like to get the discussion on that going, as adding in new models 
is something near and dear.



> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572411#comment-15572411
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I can't be assigned jiras but I'll take a stab at METRON-502, unless you 
have already completed it ;)


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572407#comment-15572407
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
That JIRA about making the ParserIntegrationTest better is at 
https://issues.apache.org/jira/browse/METRON-502


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572384#comment-15572384
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Yeah, I agree.  I'd like to have a sensor-specific validator as well as a 
global validator.  Right now we have a `MessageFilter`, but it's slightly 
different in intent.  It is intended to pick out which messages we want to pass 
through the parser and drop those that don't meet the criteria.  I'd like to 
see all of these concepts merged into a mapping between stellar statements and 
actions (i.e. DROP, REDIRECT_TO_INVALID, REDIRECT_TO_ERROR) and have sensor 
specific as well as global versions of this.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572366#comment-15572366
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I would prefer changing the IPValidator.  As an aside to this, if someone 
were to want to add in new models to the system I would think Metron would need 
to have the validation mapped to model type and not so hard coded into the 
system.  Some of this would fall out of that effort, unless I'm mistaken


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572361#comment-15572361
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Actually, upon even further consideration, I think we should revert the 
default behavior of the IP validator to return `true` if the field doesn't 
exist.  Handling it in the `FieldValidator` means we cannot express a 
validation function that ensures the existence of a field across all parsers 
(e.g. `timestamp`).

Thoughts?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572305#comment-15572305
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Ok, so there is still an issue here.  Some of the other parsers do not 
create an `ip_src_addr` and `ip_dst_addr` field in their test data (e.g. 
`JSONMapParser`).  In those cases, the `global.json` used fails every message 
as invalid because the default behavior changed for the IP validation code.

As I see it, we have 2 choices:
1. Make the behavior of the IP validator return valid if the field doesn't 
exist.
2. Remove the IP validation check from the integration test global config
3. Force the various parsers to provide those fields.

I suggest 1 upon further consideration.  Global validation across ALL 
parsers is going to have to be tolerant of missing values as not all parsers 
are going to provide every fields.  I think it should be interpreted, "Are 
these fields globally valid if they exist?" rather than "are these fields 
globally valid?"

That being said, I think that the accommodation for this should exist in 
the FieldValidation function, not in the individual validator functions.  I'll 
make a PR to your PR to this effect, but I wanted to make sure everyone was on 
board with that and gather thoughts.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572141#comment-15572141
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Yep, I will. :)


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572105#comment-15572105
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Are you going to open a jira for that?

-- 

On October 13, 2016 at 10:16:13, Casey Stella (notificati...@github.com)
wrote:

@nickwallen  Funny you mention that; yes, I
think we can. We should modify ParserIntegrationTest here


to check the invalid and error queues and dump their contents and fail
immediately.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub

,
or mute the thread


.



> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572081#comment-15572081
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
That is awesome.  Thank you.
I have been looking at the bro test and getting the failures, but I haven’t
gotten hits inside the IP Validator, so i’m a few steps behind.
I will take care of your pull request right way.

The good news, is in true ‘learning by carnage’ fashion, I know know a bit
about the integration tests ;)

On October 13, 2016 at 09:43:23, Casey Stella (notificati...@github.com)
wrote:

I figured out what was going on with this PR and it's kinda interesting, so
forgive the long-winded explanation.

@ottobackwards  is indeed right, it is
not apparent why his PR about a FieldTransformation and Stellar Function
would affect the YafIntegrationTest. Unfortunately, it became apparent when
it consistently failed on this PR in both travis and locally that something
was happening.

It turns out the problem is here

.
The keen observer will notice in the global config we are ensuring that the
src_ip_addr and dst_ip_addr are valid IP addresses for the integration
tests. This, of course, is a typo (that I made :P ). The correct field
names are ip_src_addr and ip_dst_addr.

Now, you may ask:

   - How has this ever worked?
   - Given that it did work before, why is it broken now?

Well, for the first question, it was never correct in the integration
tests, but since we did not exercise the situation where we send in invalid
IP addresses. Furthermore, the default behavior for fields which do not
exist is to return true (probably wrong).

For the second question, @ottobackwards 
has changed the default behavior for fields which do not resolve is to
return false (almost assuredly right).

For clarity to show how I go about debugging these problems, I'll be rather
explicit in how I figured out what was happening:

   - In my git branch, I added @ottobackwards
    's repo as a remote repo and checked
   out the branch:
  - git remote add otto
  https://github.com/ottobackwards/incubator-metron.git
  - git checkout otto/METRON-424
  - git branch METRON-424-debug
  - git checkout METRON-424-debug
   - In Intellij, I ran the YafIntegrationTest to verify that it continued
   to fail
   - I set breakpoints at the exit points of the ParserBolt:
  - Line 135
  

  - Line 147
  

  - Line 131
  

   - It became apparent that we were exiting because the message was not
   valid. I then set breakpoints inside of @ottobackwards
    's code and validated that it was
   doing the right thing, but the fields were coming in as null, which
   means the fields were not in the message.
   - I tracked the global.json down and looked at the field names and there
   the bug became apparent.

I submitted a PR to your PR branch, @ottobackwards
, so if you merge that in, the build
*should* work (fingers crossed ;)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub

,
or mute the thread


.



> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldVal

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572035#comment-15572035
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Interesting. Is there anything we can do to make integration test failures 
like this more transparent in the future?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572048#comment-15572048
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Ok - from what I can see things are written to the kafka topic, but the 
bolt is never called.  I don't see an crashes in the field validation code at 
this time.  Can anyone make sense of these exceptions I get ( I am only running 
the BRO Integration test )



[brointegrationexception.txt](https://github.com/apache/incubator-metron/files/527323/brointegrationexception.txt)



> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572043#comment-15572043
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
@nickwallen Funny you mention that; yes, I think we can.  We should modify 
`ParserIntegrationTest` 
[here](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/integration/ParserIntegrationTest.java#L77)
 to check the invalid and error queues and dump their contents and fail 
immediately.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571956#comment-15571956
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I figured out what was going on with this PR and it's kinda interesting, so 
forgive the long-winded explanation.

@ottobackwards is indeed right, it is not apparent why his PR about a 
FieldTransformation and Stellar Function would affect the `YafIntegrationTest`. 
 Unfortunately, it became apparent when it consistently failed on this PR in 
both travis and locally that something was happening.

It turns out the problem is 
[here](https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-integration-test/src/main/config/zookeeper/global.json#L14).
  The keen observer will notice in the global config we are ensuring that the 
`src_ip_addr` and `dst_ip_addr` are valid IP addresses for the integration 
tests.  This, of course, is a typo (that I made :P ).  The correct field names 
are `ip_src_addr` and `ip_dst_addr`.  
Now, you may ask:
* How has this ever worked?
* Given that it did work before, why is it broken now?

Well, for the first question, it was never correct in the integration 
tests, but since we did not exercise the situation where we send in invalid IP 
addresses.  Furthermore, the default behavior for fields which do not exist is 
to return `true` (probably wrong).

For the second question, @ottobackwards has changed the default behavior 
for fields which do not resolve is to return `false` (almost assuredly right).

For clarity to show how I go about debugging these problems, I'll be rather 
explicit in how I figured out what was happening:
* In my git branch, I added @ottobackwards 's repo as a remote repo and 
checked out the branch:
  * `git remote add otto 
https://github.com/ottobackwards/incubator-metron.git`
  * `git checkout otto/METRON-424`
  * `git branch METRON-424-debug`
  * `git checkout METRON-424-debug`
* In Intellij, I ran the YafIntegrationTest to verify that it continued to 
fail
* I set breakpoints at the exit points of the `ParserBolt`:
  * [Line 
135](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L135)
  * [Line 
147](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L147)
  * [Line 
131](https://github.com/apache/incubator-metron/blob/4a4cb8b117dbb66bbfb4915bca9d871a06682c28/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java#L131)
* It became apparent that we were exiting because the message was not 
valid.  I then set breakpoints inside of @ottobackwards 's code and validated 
that it was doing the right thing, but the fields were coming in as `null`, 
which means the fields were not in the message.
* I tracked the `global.json` down and looked at the field names and there 
the bug became apparent.

I submitted a PR to your PR branch, @ottobackwards, so if you merge that 
in, the build *should* work (fingers crossed ;)


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569679#comment-15569679
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
If that works for you, then great.  

I usually use Maven when I need to run all the tests before a merge or PR.  
When I want to dig into one test and debug something, I just open up Intellij.  
That makes setting the breakpoint and triggering a single test run very easy.  

You just have to watch out because sometimes (rarely) something will work 
in Intellij, but not in Maven or vice versa.  Each builds the code slightly 
differently.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569591#comment-15569591
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I'm running as in 
https://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html 
and attaching with intellij in test, as you describe actually.  It was how to 
kick off the test in debug that got me


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569589#comment-15569589
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I would run one of the tests and put a break point at 
`ParserIntegrationTest.test()` and start walking through it from there.  


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569515#comment-15569515
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Ok - I'm running the integration-test now.  Is there a way to debug it?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569502#comment-15569502
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
FWIW - I am also getting the integration test failures for this PR when I 
run them locally.  Not sure what the problem is though.  I think an exception 
is being swallowed somewhere.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569293#comment-15569293
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
To run the integration tests... Run `mvn integration-test` from Metron's 
root directory.

I do not believe that the Integration tests are run with an `install` or 
`package`, those just run the unit tests.

On a side note, you can see what Travis is doing by looking at the 
`.travisyml` file that is in Metron's root directory.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569278#comment-15569278
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I just outright did a mvn clean install for the entire project and let it 
run while I was doing other stuff.  but integration-test target I believe also 
hits them.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569175#comment-15569175
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Justin, how do I run the integration tests?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569156#comment-15569156
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I pulled the PR down locally, and ran into the same integration test issue. 
 I also ran on master locally and it succeeded.  I'm not sure what the root 
issue is if that's (consistently) the case.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569038#comment-15569038
 ] 

ASF GitHub Bot commented on METRON-424:
---

GitHub user ottobackwards reopened a pull request:

https://github.com/apache/incubator-metron/pull/303

METRON-424 ability to validate ip addresses against both IPV4 and IPV6

Allow the specification of multiple validation types for IP
` 
   {
 "fieldValidations" : [
 {
   "input" : [ "field1", "field2" ]
   ,"validation" : "IP"
   ,"config" : {
   "type" : ["IPV4","IPV6"]
 }
   }
   ]
   }`

The question with this PR is if this is the correct approach.  Is this the 
most consistent approach?  Is it the least surprising?  Would it be better to 
add a ALL keyword such that instead of specifying both types you can just do

"type": "ALL"?

I did it this way to start, but I'm open to change it based on review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/incubator-metron METRON-424

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #303


commit 9064ec0cdc38c54bf3a5309bad200f1f27d3c00a
Author: Otto Fowler 
Date:   2016-10-11T14:57:02Z

METRON-424 ability to validate ip addresses against both IPV4 and IPV6




> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569037#comment-15569037
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards closed the pull request at:

https://github.com/apache/incubator-metron/pull/303


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568930#comment-15568930
 ] 

ASF GitHub Bot commented on METRON-424:
---

GitHub user ottobackwards reopened a pull request:

https://github.com/apache/incubator-metron/pull/303

METRON-424 ability to validate ip addresses against both IPV4 and IPV6

Allow the specification of multiple validation types for IP
` 
   {
 "fieldValidations" : [
 {
   "input" : [ "field1", "field2" ]
   ,"validation" : "IP"
   ,"config" : {
   "type" : ["IPV4","IPV6"]
 }
   }
   ]
   }`

The question with this PR is if this is the correct approach.  Is this the 
most consistent approach?  Is it the least surprising?  Would it be better to 
add a ALL keyword such that instead of specifying both types you can just do

"type": "ALL"?

I did it this way to start, but I'm open to change it based on review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/incubator-metron METRON-424

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #303


commit 9064ec0cdc38c54bf3a5309bad200f1f27d3c00a
Author: Otto Fowler 
Date:   2016-10-11T14:57:02Z

METRON-424 ability to validate ip addresses against both IPV4 and IPV6




> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568929#comment-15568929
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards closed the pull request at:

https://github.com/apache/incubator-metron/pull/303


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566975#comment-15566975
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards closed the pull request at:

https://github.com/apache/incubator-metron/pull/303


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566974#comment-15566974
 ] 

ASF GitHub Bot commented on METRON-424:
---

GitHub user ottobackwards reopened a pull request:

https://github.com/apache/incubator-metron/pull/303

METRON-424 ability to validate ip addresses against both IPV4 and IPV6

Allow the specification of multiple validation types for IP
` 
   {
 "fieldValidations" : [
 {
   "input" : [ "field1", "field2" ]
   ,"validation" : "IP"
   ,"config" : {
   "type" : ["IPV4","IPV6"]
 }
   }
   ]
   }`

The question with this PR is if this is the correct approach.  Is this the 
most consistent approach?  Is it the least surprising?  Would it be better to 
add a ALL keyword such that instead of specifying both types you can just do

"type": "ALL"?

I did it this way to start, but I'm open to change it based on review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/incubator-metron METRON-424

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #303


commit 9064ec0cdc38c54bf3a5309bad200f1f27d3c00a
Author: Otto Fowler 
Date:   2016-10-11T14:57:02Z

METRON-424 ability to validate ip addresses against both IPV4 and IPV6




> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566128#comment-15566128
 ] 

ASF GitHub Bot commented on METRON-424:
---

GitHub user ottobackwards reopened a pull request:

https://github.com/apache/incubator-metron/pull/303

METRON-424 ability to validate ip addresses against both IPV4 and IPV6

Allow the specification of multiple validation types for IP
` 
   {
 "fieldValidations" : [
 {
   "input" : [ "field1", "field2" ]
   ,"validation" : "IP"
   ,"config" : {
   "type" : ["IPV4","IPV6"]
 }
   }
   ]
   }`

The question with this PR is if this is the correct approach.  Is this the 
most consistent approach?  Is it the least surprising?  Would it be better to 
add a ALL keyword such that instead of specifying both types you can just do

"type": "ALL"?

I did it this way to start, but I'm open to change it based on review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/incubator-metron METRON-424

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #303


commit 9064ec0cdc38c54bf3a5309bad200f1f27d3c00a
Author: Otto Fowler 
Date:   2016-10-11T14:57:02Z

METRON-424 ability to validate ip addresses against both IPV4 and IPV6




> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566127#comment-15566127
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards closed the pull request at:

https://github.com/apache/incubator-metron/pull/303


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566095#comment-15566095
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
So, the problem with the tests is this:
```
test(org.apache.metron.parsers.integration.YafIntegrationTest)  Time 
elapsed: 58.442 sec  <<< ERROR!
java.lang.RuntimeException: Too many retries: 11
at 
org.apache.metron.integration.ComponentRunner.process(ComponentRunner.java:140)
at 
org.apache.metron.parsers.integration.ParserIntegrationTest.test(ParserIntegrationTest.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:59)
at 
org.junit.internal.runners.MethodRoadie.runTestMethod(MethodRoadie.java:98)
at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:79)
at 
org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:87)
at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:77)
at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:42)
at 
org.junit.internal.runners.JUnit4ClassRunner.invokeTestMethod(JUnit4ClassRunner.java:88)
at 
org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51)
at 
org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44)
at 
org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27)
at 
org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37)
at 
org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
```

This test failed because of a timeout (probably because the box running it 
is overloaded.  The stream of failures after that point is due to the Parser 
Integration tests not shutting down the infrastructure it spins up (e.g. storm, 
etc.) when the tests fail.  I'd close and immediately reopen this PR and it 
should trigger a test rerun.  It is probably worth while making a JIRA to 
correct the integration tests to be smarter and not spew horrible things.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566004#comment-15566004
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
OK, good then.  I don't have to change anything.
I have no idea why the travis build is failing - it doesn't look todo with 
my changes ( btw there are always so many errors it is hard to look at the log 
).  How does on kick off a new build in the event of a transient error?  Can 
anyone help with what is going on in the log?


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565988#comment-15565988
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
Jon described it well.  I'd prefer "explicit and obvious."


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565980#comment-15565980
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I tend to agree, I prefer `["IPV4", "IPV6"] where we just do an `or` 
between the two.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565971#comment-15565971
 ] 

ASF GitHub Bot commented on METRON-424:
---

Github user JonZeolla commented on the issue:

https://github.com/apache/incubator-metron/pull/303
  
I would personally vote for `"type" : ["IPV4","IPV6"]`.  I usually push for 
explicit and obvious so something like `retrieve_json.py | grep IPV4` would 
work.


> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565676#comment-15565676
 ] 

ASF GitHub Bot commented on METRON-424:
---

GitHub user ottobackwards opened a pull request:

https://github.com/apache/incubator-metron/pull/303

METRON-424 ability to validate ip addresses against both IPV4 and IPV6

Allow the specification of multiple validation types for IP
` 
   {
 "fieldValidations" : [
 {
   "input" : [ "field1", "field2" ]
   ,"validation" : "IP"
   ,"config" : {
   "type" : ["IPV4","IPV6"]
 }
   }
   ]
   }`

The question with this PR is if this is the correct approach.  Is this the 
most consistent approach?  Is it the least surprising?  Would it be better to 
add a ALL keyword such that instead of specifying both types you can just do

"type": "BOTH"?

I did it this way to start, but I'm open to change it based on review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ottobackwards/incubator-metron METRON-424

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/303.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #303


commit 9064ec0cdc38c54bf3a5309bad200f1f27d3c00a
Author: Otto Fowler 
Date:   2016-10-11T14:57:02Z

METRON-424 ability to validate ip addresses against both IPV4 and IPV6




> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-424) Stellar fieldValidation "IP" does not support list values

[Otto Fowler (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565657#comment-15565657
 ] 

Otto Fowler commented on METRON-424:


I have a PR for this, however I'm not sure if we should allow a list as 
requested or add a 'BOTH' keyword.
We'll see what comes out of review

> Stellar fieldValidation "IP" does not support list values
> -
>
> Key: METRON-424
> URL: https://issues.apache.org/jira/browse/METRON-424
> Project: Metron
>  Issue Type: Bug
>Reporter: Neha Sinha
>
> Currently, stellar fieldValidation *IP* accepts only single string 
> value.However we need the ability to specify a list of values,say, we want to 
> validate ip_dst_addr to be of either IPV4 or IPV6 type. This feature needs to 
> be incorporated.
> Desired Behaviour :- User should be able to configure ip validation as below 
> :-
> GLOBAL Config: global
> {
>   "es.clustername": "metron",
>   "es.ip": "metron-test1-10.openstacklocal",
>   "es.port": "9300",
>   "es.date.format": ".MM.dd.HH",
>   "fieldValidations" : [
>   {
> "input" : [ "ip_dst_addr" ],
> "validation" : "IP",
> "config" : {
> "type" : ["IPV6","IPV4"]
>}
>   }
>]
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)