Hong, Koji, There is a ticket to upgrade this processor to a new version [1] (although the ticket is showing its age by listing 2.1.0).
I had looked into changing it, but the structure of the library validation process and visibility of some of the core classes changed a fair amount and was breaking to the existing processor structure. For some context, some work was done to perform validation, bringing parts of that codebase into the processor to avoid exceptions for illegal patterns. 2.0.0 had some bugs in the library's expression validation process highlighted in NIFI-1189 which caused it to fail at expressions that were in fact valid. I think making use of the introduced functions would most certainly be a great addition but will likely need some refactoring to work with the new package structure of JsonPath. [1] https://issues.apache.org/jira/browse/NIFI-1377 On Thu, Apr 7, 2016 at 6:09 AM, Koji Kawamura <[email protected]> wrote: > Hi all, > > I got interested in EvaluateJsonPath, too. > So I tried to figure out how to do that, and found there're few obstacles > now. > > Input Json = > {"company":"xyz", "rate":[0.02, 0.03, 0.04]} > > Then, applying following JsonPaths return: > > $.rate.min() => 0.2 > $.rate.length() => 3 > $.rate.avg() => 0.03 > > Those aggregation can be tested here: > http://jsonpath.herokuapp.com/ > > So, we can configure EvaluateJsonPath like: > - Destination: flowfile-attribute > - Return Type: scala > - dynamic properties > -- rate.avg: $.rate.avg() > -- rate.length: $.rate.length() > -- rate.max: $.rate.max() > -- rate.min: $.rate.min() > > However, NiFi uses json-path ver 2.0.0 which doesn't understand functions > yet. > Function capability was added from json-path 2.1.0. > So currently aggregate functions don't work with EvaluateJsonPath. > > I tried to update json-path to 2.2.0, but JsonPathExpressionValidator > got compile error since some Classes have been moved to different > packages. > It needs to be modified. > > BTW, for extracting rate array, I was able to do that with following > settings: > - Return Type: json > - dynamic properties > -- rate: $.rate > > Haven't reached to the goal yet, but I hope this information is helpful. > > Thanks, > Koji > > On Thu, Apr 7, 2016 at 6:09 AM, Hong Li <[email protected]> > wrote: > > Sorry, Aldrin. I forgot to answer your question. My arrays do not share > > fixed number of values. > > > > Thanks. > > Hong > > > > > > > > Hong Li > > > > Centric Consulting > > > > In Balance > > (888) 781-7567 office > > (614) 296-7644 mobile > > www.centricconsulting.com | @Centric > > > > > > On Wed, Apr 6, 2016 at 4:35 PM, Hong Li <[email protected]> > > wrote: > >> > >> I have two goals here for the information embedded in the array. > >> > >> The first goal is to be able to do some kind of aggregation on the > array. > >> For example, given {"company":"xyz", "rate":[0.02, 0.03, 0.04]}, if we > could > >> have a Json variable "rate", I would like to have something such as > >> ${rate[*]:count()} that would give me a scalar value of the count. > Then I > >> can assign the value to a property of a processor under Nifi. But, > >> currently under Nifi, $.rate is not a valid expression. Therefore, > >> $.rate[*]:count() is not valid either. Nifi flow would give me errors > and > >> failed if I did that. > >> > >> My second goal is to be able to carry the array forward but turn it > into a > >> scalar structure as a single string such as (0.02, 0.03, 0.04). I have > >> experimented the setups you suggested. I explicitly configured the > >> destination as flowfile-content and return type as Json. This time, > Nifi > >> flow did not complain about any error any more but it did not capture > >> anything either in the end. > >> > >> I would say my first goal is very important for my project. > >> > >> Thank you all for your help. > >> Hong > >> > >> > >> > >> > >> > >> Hong Li > >> > >> Centric Consulting > >> > >> In Balance > >> (888) 781-7567 office > >> (614) 296-7644 mobile > >> www.centricconsulting.com | @Centric > >> > >> > >> On Wed, Apr 6, 2016 at 2:35 PM, Thad Guidry <[email protected]> > wrote: > >>> > >>> Does it store as an attribute if you change the return type from > >>> auto-detect to JSON ? > >>> > >>> Return Type auto-detect > >>> > >>> auto-detect > >>> json > >>> scalar > >>> > >>> Indicates the desired return type of the JSON Path expressions. > Selecting > >>> 'auto-detect' will set the return type to 'json' for a Destination of > >>> 'flowfile-content', and 'scalar' for a Destination of > 'flowfile-attribute'. > >>> > >>> > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.EvaluateJsonPath/index.html > >>> > >>> Depending on your use case.... you might want to store those rate > values > >>> as content rather than attributes....but depends on what your trying to > >>> accomplish. You are not limited to changing your flow with just > >>> EvaluateionJsonPath... you can probably add an additional processor > after it > >>> to do further filtering or setting of attributes, rather than trying > to just > >>> do it all in one shot with the EvaluateJsonPath processor. > >>> > >>> Thad > >>> +ThadGuidry > >> > >> > > >
