subject:"\[jira\] \[Commented\] \(NIFI\-1893\) Add processor for validating JSON"

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2020-05-20 Thread Carlos (Jira)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112527#comment-17112527
 ] 

Carlos commented on NIFI-1893:
--

 this is still on going? I need to validate json and I want to know if this 
processor is going to be part of the official release some day

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-12-25 Thread HondaWei (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728910#comment-16728910
 ] 

HondaWei commented on NIFI-1893:


Hi [~khill] and [~samox]

Sorry for late reply. I've sent a pull request NIFI-1893. This is my first 
contribution to NiFi and please gives me some suggestion.

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-12-19 Thread Kevin Hill (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725153#comment-16725153
 ] 

Kevin Hill commented on NIFI-1893:
--

Great [~Honda] I'm looking forward to it, as I my next todo involves validating 
some json-schema speced inputs. 

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-12-07 Thread HondaWei (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712596#comment-16712596
 ] 

HondaWei commented on NIFI-1893:


Hi [~samox], thanks for your comment. This processor works well in my local 
NiFi. Now the processor is reviewed by my partner and I am going to upload my 
processor next week. 

Thank you very much!
!image-2018-12-07-18-02-52-813.png!

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
> Attachments: image-2018-12-07-18-02-52-813.png
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-12-06 Thread ASF GitHub Bot (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712238#comment-16712238
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user nileio commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@basvank not sure if any further progress has been done or there is 
alternative solutions for this ? 
someone has suggested using 
https://github.com/java-json-tools/json-schema-validator .. 
Is this issue still being considered ?


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-12-06 Thread Sam Marko (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712205#comment-16712205
 ] 

Sam Marko commented on NIFI-1893:
-

Hi [~hondawei] , i am looking to do the same thing and was looking at the same 
library too.. could you upload your processor please ?

thanks!!

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2018-08-28 Thread HondaWei (JIRA)



[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595846#comment-16595846
 ] 

HondaWei commented on NIFI-1893:


Hi 

I would like to use the following library to implement this processor

[https://github.com/java-json-tools/json-schema-validator]

 

I have already finished it, may someone assign this issue to me?

thank you!

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Priority: Major
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2017-01-03 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795375#comment-15795375
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user basvank commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@pvillard31 @mattyb149 thanks for the information, that didn't become clear 
from this PR.
It seems to be possible to implement in an ASF-friendly way using 
https://github.com/daveclayton/json-schema-validator (ASL 2.0), which depends 
on https://github.com/FasterXML/jackson-core (Apache 2.0) instead of json.org.
I will see if I can find the time to do this myself, but if not it can 
serve as a pointer for others.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2017-01-03 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795338#comment-15795338
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
I believe it had to be removed because it uses or has a dependency on 
JSON.org-licensed JARs, which was determined to be Category X for the Apache 
Software Foundation. It was removed in #1230 under 
[NIFI-2991](https://issues.apache.org/jira/browse/NIFI-2991). I encourage you 
to change the underlying JSON libraries such that the licensing is 
ASF-friendly, then we can review and merge the updated PR.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2017-01-03 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795335#comment-15795335
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@basvank 
Due to licensing issues:
https://issues.apache.org/jira/browse/NIFI-2991

We unfortunately had to remove this processor... we are looking for a new 
solution using another library. However, in the meantime, you can manually 
build this processor using this PR, and add the generated NAR to your NiFi 
instance.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2017-01-03 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795326#comment-15795326
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user basvank commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@pvillard31 (and @bartoszjkwozniak) is it possible that something went 
wrong when merging this feature, maybe because the checks failed? It seems the 
ValidateJson processor did not end up in the master branch or any of the 
releases (I'm on 1.1.1).


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-11-15 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668676#comment-15668676
 ] 

ASF subversion and git services commented on NIFI-1893:
---

Commit 6696529c99b5b564b076e4012abff3a54755370b in nifi's branch 
refs/heads/master from [~joewitt]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6696529 ]

Revert "NIFI-1893 Add processor for validating JSON"

This reverts commit f11682202b57edab585967e033839800d4159f4e.

Signed-off-by: jpercivall 


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-11-15 Thread Joseph Witt (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668443#comment-15668443
 ] 

Joseph Witt commented on NIFI-1893:
---

i have reverted this in the NIFI-2991 work.  It looks like a cool processor so 
hopefully it can be reworked to use a different JSON library.

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-11-15 Thread Joseph Witt (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668200#comment-15668200
 ] 

Joseph Witt commented on NIFI-1893:
---

[~bartoszjkwozniak] [~pvillard] [~mattyb149] Unfortunately we need to revert 
this commit/processor for now.  I've reopened the JIRA.  It depends on a 
library which is now considered category X and also we need to reflect these 
new libs in our license/notice.  Everit for example needs a copyright ref in 
our notice and it is what pulls in the now catx json dependency.

> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513009#comment-15513009
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@pvillard31 you're welcome! Thank you too and @mattyb149 for valuable code 
review.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
> Fix For: 1.1.0
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512986#comment-15512986
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@bartoszjkwozniak I merged it to master, thanks for your contribution!


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
> Fix For: 1.1.0
>
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512980#comment-15512980
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1037


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512978#comment-15512978
 ] 

ASF subversion and git services commented on NIFI-1893:
---

Commit f11682202b57edab585967e033839800d4159f4e in nifi's branch 
refs/heads/master from [~bartoszjkwozniak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f116822 ]

NIFI-1893 Add processor for validating JSON

This closes #1037.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512723#comment-15512723
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
@pvillard31 nope I'm good if you are, I hadn't realized we had started 
reviewing at the same time :)


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512587#comment-15512587
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
LGTM, full build with contrib-check OK.
@mattyb149 do you want to have another look?


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510144#comment-15510144
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79849248
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateJson.java
 ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.nio.file.Paths;
+
+import org.apache.commons.io.IOUtils;
+
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+public class TestValidateJson {
--- End diff --

Sure thing, added


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510145#comment-15510145
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79849231
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StringUtils;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("validate-json-schema-file")
+.displayName("Schema File")
+.description("The path to the Schema file that is to be used 
for validation. Only one of Schema File or Schema Body may be used")
+.required(false)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor SCHEMA_BODY = new 
PropertyDescriptor.Builder()
+.name("validate-json-schema-body")
+.displayName("Schema Body")
+.required(false)
+.description("Json Schema Body that is to be used for 
validation. Only one of Schema File or Schema Body may be used")
+.expressionLanguageSupported(false)
+.addValidator(Validator.VALID)
+.build();
+
+public static final Relationship REL_VALID = new

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510143#comment-15510143
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79849108
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml ---
@@ -249,6 +249,11 @@ language governing permissions and limitations under 
the License. -->
 super-csv
 2.4.0
 
+
+org.everit.json
+org.everit.json.schema
+1.4.0
+
--- End diff --

ahh, I see now. Corrected.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509941#comment-15509941
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on the issue:

https://github.com/apache/nifi/pull/1037
  
For other reviewers, a template to test the processor:
https://gist.github.com/pvillard31/0d8d0d3122ecdd179cc39fbd83d4b9c4


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509924#comment-15509924
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r7986
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateJson.java
 ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.nio.file.Paths;
+
+import org.apache.commons.io.IOUtils;
+
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+public class TestValidateJson {
--- End diff --

Could you add one or two unit tests for the invalid case?


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509925#comment-15509925
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79831139
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StringUtils;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("validate-json-schema-file")
+.displayName("Schema File")
+.description("The path to the Schema file that is to be used 
for validation. Only one of Schema File or Schema Body may be used")
+.required(false)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor SCHEMA_BODY = new 
PropertyDescriptor.Builder()
+.name("validate-json-schema-body")
+.displayName("Schema Body")
+.required(false)
+.description("Json Schema Body that is to be used for 
validation. Only one of Schema File or Schema Body may be used")
+.expressionLanguageSupported(false)
+.addValidator(Validator.VALID)
+.build();
+
+public static final Relationship REL_VALID = new

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509161#comment-15509161
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on the issue:

https://github.com/apache/nifi/pull/1037
  
Thank you for your insight. Pushed new commit on branch.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509159#comment-15509159
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79779583
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509151#comment-15509151
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79779153
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
--- End diff --

Thank you for your

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509141#comment-15509141
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778851
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509145#comment-15509145
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778904
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

sure thing, wasn't aware of that


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation:

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509139#comment-15509139
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778806
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509143#comment-15509143
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778863
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509135#comment-15509135
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778601
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

sure thing, wasn't aware of that


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation:

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509133#comment-15509133
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user bartoszjkwozniak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79778565
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
--- End diff --

Added Schema Body property and custom validator - exactly one of Schema 
File or Schema Body must be set.


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
>

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507100#comment-15507100
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79656833
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Lately the conventional wisdom for the name() method is to choose a 
machine-friendly name (like "validate-json-schema-file") and use displayName() 
for the user-friendly name. This will help when we do more work with 
internationalization. Not a requirement, just a suggestion :)


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp":

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507099#comment-15507099
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79657612
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
--- End diff --

For NiFi clusters, it may not

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507082#comment-15507082
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654106
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
--- End diff --

Could you use both ``.name()`` and ``.displayName()`` as explained here:

https://mail-archives.apache.org/mod_mbox/nifi-dev/201605.mbox/%3c5a6fdf1e-1889-46fe-a3c4-5d2f0a905...@apache.org%3E


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
>

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507083#comment-15507083
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655327
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507084#comment-15507084
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79654708
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
--- End diff --

Would it be possible to have this property to accept both a file and a 
string representation of the schema? If the input given by the user is an 
existing file we use a file as you propose, otherwise we use the string as the 
schema definition. I think that a lot of users would like to pass a schema 
definition without using a file (given that, in cluster mode, the file will 
have to be available on each node). Thoughts?


> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name",

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507081#comment-15507081
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655788
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507080#comment-15507080
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

Github user pvillard31 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1037#discussion_r79655764
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
 ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import org.everit.json.schema.Schema;
+import org.everit.json.schema.ValidationException;
+import org.everit.json.schema.loader.SchemaLoader;
+import org.json.JSONArray;
+import org.json.JSONObject;
+import org.json.JSONTokener;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"json", "schema", "validation"})
+@CapabilityDescription("Validates the contents of FlowFiles against a 
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final PropertyDescriptor SCHEMA_FILE = new 
PropertyDescriptor.Builder()
+.name("Schema File")
+.description("The path to the Schema file that is to be used 
for validation")
+.required(true)
+.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+.description("FlowFiles that are successfully validated 
against the schema are routed to this relationship")
+.build();
+public static final Relationship REL_INVALID = new 
Relationship.Builder()
+.name("invalid")
+.description("FlowFiles that are not valid according to the 
specified schema are routed to this relationship")
+.build();
+
+private List properties;
+private Set relationships;
+private final AtomicReference schemaRef = new 
AtomicReference<>();
+
+@Override
+protected void init(final ProcessorInitializationContext context) {
+final List properties = new ArrayList<>();
+properties.add(SCHEMA_FILE);
+this.properties =

[jira] [Commented] (NIFI-1893) Add processor for validating JSON

2016-09-20 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507030#comment-15507030
 ] 

ASF GitHub Bot commented on NIFI-1893:
--

GitHub user bartoszjkwozniak opened a pull request:

https://github.com/apache/nifi/pull/1037

NIFI-1893 Add processor for validating JSON

Signed-off-by: Bartosz Wozniak 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bartoszjkwozniak/nifi NIFI-1893

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1037.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1037


commit 554d15fa994b74db67e2906de31a8fe001045d95
Author: Bartosz Wozniak 
Date:   2016-09-19T11:37:54Z

NIFI-1893 Add processor for validating JSON

Signed-off-by: Bartosz Wozniak 




> Add processor for validating JSON
> -
>
> Key: NIFI-1893
> URL: https://issues.apache.org/jira/browse/NIFI-1893
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>
> NiFi has a ValidateXml processor to validate incoming XML files against a 
> schema. It would be good to have one to validate JSON files as well.
> For example, an input JSON of:
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load": 15.6
>   }
> }
> Could be validated successfully against the following "schema":
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> There is at least one ASF-friendly library that could be used for 
> implementation: https://github.com/everit-org/json-schema



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

43 matches

Mail list logo