Hi all,
I got interested in EvaluateJsonPath, too.
So I tried to figure out how to do that, and found there're few obstacles now.
Input Json =
{"company":"xyz", "rate":[0.02, 0.03, 0.04]}
Then, applying following JsonPaths return:
$.rate.min() => 0.2
$.rate.length() => 3
$.rate.avg() => 0.03
Those aggregation can be tested here:
http://jsonpath.herokuapp.com/
So, we can configure EvaluateJsonPath like:
- Destination: flowfile-attribute
- Return Type: scala
- dynamic properties
-- rate.avg: $.rate.avg()
-- rate.length: $.rate.length()
-- rate.max: $.rate.max()
-- rate.min: $.rate.min()
However, NiFi uses json-path ver 2.0.0 which doesn't understand functions yet.
Function capability was added from json-path 2.1.0.
So currently aggregate functions don't work with EvaluateJsonPath.
I tried to update json-path to 2.2.0, but JsonPathExpressionValidator
got compile error since some Classes have been moved to different
packages.
It needs to be modified.
BTW, for extracting rate array, I was able to do that with following settings:
- Return Type: json
- dynamic properties
-- rate: $.rate
Haven't reached to the goal yet, but I hope this information is helpful.
Thanks,
Koji
On Thu, Apr 7, 2016 at 6:09 AM, Hong Li <[email protected]> wrote:
> Sorry, Aldrin. I forgot to answer your question. My arrays do not share
> fixed number of values.
>
> Thanks.
> Hong
>
>
>
> Hong Li
>
> Centric Consulting
>
> In Balance
> (888) 781-7567 office
> (614) 296-7644 mobile
> www.centricconsulting.com | @Centric
>
>
> On Wed, Apr 6, 2016 at 4:35 PM, Hong Li <[email protected]>
> wrote:
>>
>> I have two goals here for the information embedded in the array.
>>
>> The first goal is to be able to do some kind of aggregation on the array.
>> For example, given {"company":"xyz", "rate":[0.02, 0.03, 0.04]}, if we could
>> have a Json variable "rate", I would like to have something such as
>> ${rate[*]:count()} that would give me a scalar value of the count. Then I
>> can assign the value to a property of a processor under Nifi. But,
>> currently under Nifi, $.rate is not a valid expression. Therefore,
>> $.rate[*]:count() is not valid either. Nifi flow would give me errors and
>> failed if I did that.
>>
>> My second goal is to be able to carry the array forward but turn it into a
>> scalar structure as a single string such as (0.02, 0.03, 0.04). I have
>> experimented the setups you suggested. I explicitly configured the
>> destination as flowfile-content and return type as Json. This time, Nifi
>> flow did not complain about any error any more but it did not capture
>> anything either in the end.
>>
>> I would say my first goal is very important for my project.
>>
>> Thank you all for your help.
>> Hong
>>
>>
>>
>>
>>
>> Hong Li
>>
>> Centric Consulting
>>
>> In Balance
>> (888) 781-7567 office
>> (614) 296-7644 mobile
>> www.centricconsulting.com | @Centric
>>
>>
>> On Wed, Apr 6, 2016 at 2:35 PM, Thad Guidry <[email protected]> wrote:
>>>
>>> Does it store as an attribute if you change the return type from
>>> auto-detect to JSON ?
>>>
>>> Return Type auto-detect
>>>
>>> auto-detect
>>> json
>>> scalar
>>>
>>> Indicates the desired return type of the JSON Path expressions. Selecting
>>> 'auto-detect' will set the return type to 'json' for a Destination of
>>> 'flowfile-content', and 'scalar' for a Destination of 'flowfile-attribute'.
>>>
>>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html
>>>
>>> Depending on your use case.... you might want to store those rate values
>>> as content rather than attributes....but depends on what your trying to
>>> accomplish. You are not limited to changing your flow with just
>>> EvaluateionJsonPath... you can probably add an additional processor after it
>>> to do further filtering or setting of attributes, rather than trying to just
>>> do it all in one shot with the EvaluateJsonPath processor.
>>>
>>> Thad
>>> +ThadGuidry
>>
>>
>
