Thanks Bryan! I created a 2nd schema for the output, and included just a
single json element in it. -> 'ingredient', in this example.
The 'two' element/column select statement then returned just the single
column/element fount in the output only schema, and the 2nd 'column' (not
in the output schema) was not present, and was not null. That is an
acceptable outcome with clean output.
So to answer your questions, the schema strategy for both the
'jsonPathReader' and 'jsonRecordsSetWriter' was "Use 'Schema Name'
property'. And I had an UpdateAttribute processor which preceded the
QueryRecord that set the 2 schema names, (schema.name, schema.nameOutput).
To create the schema's I edited a json message, and then piped that json
into the 'InferAvroSchema' processor.
the "everything", input schema looks like:
{
"type" : "record",
"name" : "all",
"fields" : [ {
"name" : "recipe",
"type" : "string",
"doc" : "Type inferred from '\"Tacos\"'"
}, {
"name" : "ingredients",
"type" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "ingredients",
"fields" : [ {
"name" : "name",
"type" : "string",
"doc" : "Type inferred from '\"Beef\"'"
} ]
}
},
"doc" : "Type inferred from
'[{\"name\":\"Beef\"},{\"name\":\"Lettuce\"},{\"name\":\"Cheese\"}]'"
}, {
"name" : "inventor",
"type" : {
"type" : "record",
"name" : "inventor",
"fields" : [ {
"name" : "name",
"type" : "string",
"doc" : "Type inferred from '\"Alex\"'"
}, {
"name" : "age",
"type" : "int",
"doc" : "Type inferred from '25'"
} ]
},
"doc" : "Type inferred from '{\"name\":\"Alex\",\"age\":25}'"
} ]
}
and the output schema looked like:
{
"type" : "record",
"name" : "all",
"fields" : [ {
"name" : "ingredients",
"type" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "ingredients",
"fields" : [ {
"name" : "name",
"type" : "string",
"doc" : "Type inferred from '\"Beef\"'"
} ]
}
},
"doc" : "Type inferred from
'[{\"name\":\"Beef\"},{\"name\":\"Lettuce\"},{\"name\":\"Cheese\"}]'"
} ]
}
On Fri, Jun 23, 2017 at 4:54 PM, Bryan Bende <[email protected]> wrote:
> Carl,
>
> I think you need to make a separate schema for the writer that lines
> up with the fields you are selecting in QueryRecord.
>
> You would have a read schema with recipe, ingredients, and inventor
> and then a write schema with only ingredients and recipe (using the
> example where did 'select ingredients, recipe').
>
> It might help if you could share the schema you are using, as well as
> how your reader and writer are obtaining the schema (i.e. the Schema
> Access Strategy of each).
>
> Thanks,
>
> Bryan
>
>
> On Fri, Jun 23, 2017 at 5:22 PM, Carl Berndt <[email protected]> wrote:
> > Hi,
> > I was exploring the QueryRecord this afternoon.
> > I was hoping to use this to just take a couple of json elements from the
> > message that has many json elements ( in the incoming flowfile).
> >
> > I expected "select ingredients, recipe from FLOWFILE"
> > to produce a json with just the "ingredients, recipe" and not include the
> > "inventor". ex:
> > [{"recipe":"Tacos","ingredients":[{"name":"Beef"},
> {"name":"Lettuce"},{"name":"Cheese"}]}]
> > instead I got:
> > [{"recipe":"Tacos","ingredients":[{"name":"Beef"},
> {"name":"Lettuce"},{"name":"Cheese"}],"inventor":null}]
> >
> > Having that "inventor:null" is just noise, and misleading. And, for
> example
> > if I have 25 unused elements that would be 23 Null's....
> >
> > I have the same issue if I select a single column too, for example:
> > select ingredients from FLOWFILE produces:
> > [{"recipe":null,"ingredients":[{"name":"Beef"},{"name":"
> Lettuce"},{"name":"Cheese"}],"inventor":null}]
> >
> > Here is the input
> > {
> > "recipe": "Tacos",
> > "ingredients": [
> > {
> > "name": "Beef"
> > },
> > {
> > "name": "Lettuce"
> > },
> > {
> > "name": "Cheese"
> > }
> > ],
> > "inventor": {
> > "name": "Alex",
> > "age": 25
> > }
> > }
> > I used the InferAvroSchema processor to create the avroSchema....
> > I used the JsonPathReader and have property for each json path
> > ingredients = $.ingredients
> > inventor = $.inventor
> > recipe = $.recipe
> >
> > I did nothing with the "JsonRecordSetWriter".
> >
> > Maybe I should attach a template....
> >
> > Thanks for your help!
> > Carl
> >
> >
>
