Re: JSON Schema

2016-05-17 Thread Madhukar Thota 
Thanks Matt  for the code and also for opening issue on this.

On Tue, May 17, 2016 at 1:59 PM, Matt Burgess <mattyb...@gmail.com> wrote:

> Madhu,
>
> This is a good idea for a processor (ValidateJson like the existing
> ValidateXml processor), I've written up [1] in Jira for it.
>
> In the meantime, here's a Groovy script you could use in
> ExecuteScript, just need to download the two JAR dependencies ([2] and
> [3]) and add them to your Module Directory property.
>
> import org.everit.json.schema.Schema
> import org.everit.json.schema.loader.SchemaLoader
> import org.json.JSONObject
> import org.json.JSONTokener
>
> flowFile = session.get()
> if(!flowFile) return
>
> jsonSchema = """
> {
>   "type": "object",
>   "required": ["name", "tags", "timestamp", "fields"],
>   "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
>   }
> }
> """
>
> boolean valid = true
> session.read(flowFile, { inputStream ->
>jsonInput = org.apache.commons.io.IOUtils.toString(inputStream,
> java.nio.charset.StandardCharsets.UTF_8)
>JSONObject rawSchema = new JSONObject(new JSONTokener(new
> ByteArrayInputStream(jsonSchema.bytes)))
>Schema schema = SchemaLoader.load(rawSchema)
>try {
>   schema.validate(new JSONObject(jsonInput))
> } catch(ve) {
>   log.error("Doesn't adhere to schema", ve)
>   valid = false
> }
>   } as InputStreamCallback)
>
> session.transfer(flowFile, valid ? REL_SUCCESS : REL_FAILURE)
>
>
> Hope this helps!
>
> Regards,
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-1893
> [2]
> http://mvnrepository.com/artifact/org.everit.json/org.everit.json.schema/1.3.0
> [3] http://mvnrepository.com/artifact/org.json/json/20160212
>
>
> On Tue, May 17, 2016 at 11:44 AM, Madhukar Thota
> <madhukar.th...@gmail.com> wrote:
> > is it possible to do validation of incoming json via http Processor with
> > Json Schema in nifi?
> >
> > Example Json:
> >
> > {
> >   name: "Test",
> >   timestamp: 1463499695,
> >   tags: {
> >"host": "Test_1",
> >"ip" : "1.1.1.1"
> >   },
> >   fields: {
> > "cpu": 10.2,
> > "load: : 15.6
> >   }
> > }
> >
> > JSON schema:
> >
> > "type": "object",
> > "required": ["name", "tags", "timestamp", "fields"],
> > "properties": {
> > "name": {"type": "string"},
> > "timestamp": {"type": "integer"},
> > "tags": {"type": "object", "items": {"type": "string"}},
> > "fields": { "type": "object"}
> > }
>

Re: JSON Schema

2016-05-17 Thread Matt Burgess 
Madhu,

This is a good idea for a processor (ValidateJson like the existing
ValidateXml processor), I've written up [1] in Jira for it.

In the meantime, here's a Groovy script you could use in
ExecuteScript, just need to download the two JAR dependencies ([2] and
[3]) and add them to your Module Directory property.

import org.everit.json.schema.Schema
import org.everit.json.schema.loader.SchemaLoader
import org.json.JSONObject
import org.json.JSONTokener

flowFile = session.get()
if(!flowFile) return

jsonSchema = """
{
  "type": "object",
  "required": ["name", "tags", "timestamp", "fields"],
  "properties": {
"name": {"type": "string"},
"timestamp": {"type": "integer"},
"tags": {"type": "object", "items": {"type": "string"}},
"fields": { "type": "object"}
  }
}
"""

boolean valid = true
session.read(flowFile, { inputStream ->
   jsonInput = org.apache.commons.io.IOUtils.toString(inputStream,
java.nio.charset.StandardCharsets.UTF_8)
   JSONObject rawSchema = new JSONObject(new JSONTokener(new
ByteArrayInputStream(jsonSchema.bytes)))
   Schema schema = SchemaLoader.load(rawSchema)
   try {
  schema.validate(new JSONObject(jsonInput))
} catch(ve) {
  log.error("Doesn't adhere to schema", ve)
  valid = false
}
  } as InputStreamCallback)

session.transfer(flowFile, valid ? REL_SUCCESS : REL_FAILURE)


Hope this helps!

Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-1893
[2] 
http://mvnrepository.com/artifact/org.everit.json/org.everit.json.schema/1.3.0
[3] http://mvnrepository.com/artifact/org.json/json/20160212


On Tue, May 17, 2016 at 11:44 AM, Madhukar Thota
<madhukar.th...@gmail.com> wrote:
> is it possible to do validation of incoming json via http Processor with
> Json Schema in nifi?
>
> Example Json:
>
> {
>   name: "Test",
>   timestamp: 1463499695,
>   tags: {
>"host": "Test_1",
>"ip" : "1.1.1.1"
>   },
>   fields: {
> "cpu": 10.2,
> "load: : 15.6
>   }
> }
>
> JSON schema:
>
> "type": "object",
> "required": ["name", "tags", "timestamp", "fields"],
> "properties": {
> "name": {"type": "string"},
> "timestamp": {"type": "integer"},
> "tags": {"type": "object", "items": {"type": "string"}},
> "fields": { "type": "object"}
> }

JSON Schema

2016-05-17 Thread Madhukar Thota 
is it possible to do validation of incoming json via http Processor with
Json Schema in nifi?

Example Json:

{
  name: "Test",
  timestamp: 1463499695,
  tags: {
   "host": "Test_1",
   "ip" : "1.1.1.1"
  },
  fields: {
    "cpu": 10.2,
"load: : 15.6
  }
}

JSON schema:

"type": "object",
"required": ["name", "tags", "timestamp", "fields"],
"properties": {
"name": {"type": "string"},
"timestamp": {"type": "integer"},
"tags": {"type": "object", "items": {"type": "string"}},
"fields": { "type": "object"}
}