V,
Currently NiFi does not support specifying a schema in JSONSchema
format, you'll want to convert that to an Avro schema for use in
JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
that "stats" schema is supposed to be included in the outgoing object.
I ran it through a utility-under-development [1] and got the following
Avro schema out:
{"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
This doesn't add the "stats" record and specifies "create_date" as a
long versus a BigInteger. I think you might want to use an Avro
logical type of "decimal" [2] depending on what the value is in the
actual JSON object:
{"type":"record","name":"record0","fields":[
{"name":"create_date","type": {"type": "bytes","logicalType":
"decimal","precision": 12,"scale": 0}},
{"name":"id","type":"string"}
]}
If you have a stats object present, this might work:
{"type":"record","name":"record0","fields":[
{"name": "stats", "type" :
{"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
{"name":"create_date","type": {"type": "bytes","logicalType":
"decimal","precision": 12,"scale": 0}},
{"name":"id","type":"string"}
]}
I didn't validate or try these so there may be typos or other
(hopefully minor) mistakes.
Regards,
Matt
[1] https://github.com/fge/json-schema-avro
[2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
On Thu, Aug 30, 2018 at 9:54 AM l vic <[email protected]> wrote:
>
> I have json file for the schema that looks like the following:
>
> {
> "$schema": "http://json-schema.org/draft-04/schema#";,
> "definitions": {
> "stats": {
> "type": "object",
> "additionalProperties": false,
> "properties": {
> "id": {
> "type": "string"
> },
> "bin_qualifier": {
> "type": "string"
> }
> }
> }
> },
> "additionalProperties": false,
> "description": "attributes",
> "type": "object",
> "properties": {
> "id": {
> "type": "string",
> "required": true,
> },
> "create_date": {
> "type": "integer",
> "javaType": "java.math.BigInteger",
> "required": true
> }
> }
> }
>
>
> How can I add this schema for JsonTreeReader?
>
> On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <[email protected]> wrote:
>>
>> The record readers are services, that processors use.
>> When you use a *Record* processor, you will have to select a Reader and a
>> Writer Service, or create one ( which you can do through the UI ).
>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>
>>
>> On August 30, 2018 at 08:48:08, l vic ([email protected]) wrote:
>>
>> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in the
>> list of available processors...
>> Thanks,
>> V
>>
>> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <[email protected]>
>> wrote:
>>>
>>> Hi. Just like CSVRecordReader, we have record reader service for JSON. It's
>>> called JsonTreeReader. You can use AvroSchemaRegistry and provide an Avro
>>> schema (usually generated through InferAvroSchema processor) for your JSON.
>>> Refer:
>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>>>
>>> -
>>> Sivaprasanna
>>>
>>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <[email protected]> wrote:
>>>>
>>>> I need to save two different json messages according to json schemas
>>>> available for each to different relational database tables.
>>>> I saw this blog:
>>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>>> with example using CSVRecordReader for csv->json transformation.
>>>> but what would be RecordReader for schema-based transformation from json?
>>>> Is this a valid approach, or what would be best approach to solve this
>>>> problem?
>>>> I am using: nifi-1.7.1-RC1...
>>>> Thank you,
