[
https://issues.apache.org/jira/browse/NIFI-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14324985#comment-14324985
]
ASF GitHub Bot commented on NIFI-360:
-------------------------------------
GitHub user apiri opened a pull request:
https://github.com/apache/incubator-nifi/pull/28
NIFI-360 Create Processors to work against JSON data
I have created two Processors, EvaluateJsonPath and SplitJson which are
analogs of the functionality provided through EvaluateXPath and SpiltXML.
Both are powered primarily around the usage of JsonPath by Jayway.
Their capability descriptions are provided below:
EvaluateJsonPath
Evaluates one or more JsonPath expressions against the content of a
FlowFile. The results of those expressions are assigned to FlowFile Attributes
or are written to the content of the FlowFile itself, depending on
configuration of the Processor. JsonPaths are entered by adding user-defined
properties; the name of the property maps to the Attribute Name into which the
result will be placed (if the Destination is flowfile-attribute; otherwise, the
property name is ignored).
The value of the property must be a valid JsonPath expression. If the
JsonPath evaluates to a JSON array or JSON object and the Return Type is set to
'scalar' the FlowFile will be unmodified and will be routed to failure. A
Return Type of JSON can return scalar values if the provided JsonPath evaluates
to the specified value and will be routed as a match. If Destination is
'flowfile-content' and the JsonPath does not evaluate to a defined path, the
FlowFile will be routed to 'unmatched' without having its contents modified. If
Destination is flowfile-attribute and the expression matches nothing,
attributes will be created with empty strings as the value, and the FlowFile
will always be routed to 'matched.'
SplitJson
Splits a JSON File into multiple, separate FlowFiles for an array element
specified by a JsonPath expression. Each generated FlowFile is comprised of an
element of the specified array and transferred to relationship 'split, with the
original file transferred to the 'original' relationship. If the specified
JsonPath is not found or does not evaluate to an array element, the original
file is routed to 'failure' and no files are generated.
One item of note is the transitive dependency of ASM through Json-Smart
through JsonPath.
I have included, what I believe is needed to appropriately make use of this
item in the LICENSE. Review of its correctness would is requested.
Any feedback is appreciated. Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apiri/incubator-nifi json-processors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-nifi/pull/28.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #28
----
commit 446fc2ef8099faff786f11c7d640232191dac2b0
Author: Aldrin Piri <[email protected]>
Date: 2015-02-14T04:42:40Z
Adding JSONPath as a managed dependency and including it in the standard
processor pom
commit 5f03fb11d93791ea9cdde43ffb97bceaf22c0143
Author: Aldrin Piri <[email protected]>
Date: 2015-02-14T17:56:19Z
Creating stub for an EvaluateJSONPath processor and providing configuration
akin to EvaluateXPath
commit ec669e5b42e58335ab511dfa8dfd52bd0848f088
Author: Aldrin Piri <[email protected]>
Date: 2015-02-14T18:26:08Z
Stubbing out validator, setting up relationships and properties.
commit e75213eead936baa7046d9c9ba5b86a4508f532a
Author: Aldrin Piri <[email protected]>
Date: 2015-02-14T18:35:53Z
Adding EvaluateJsonPath to the Processor services file
commit da6b55f34ced99e3364aa382732a71d2d24b33f5
Author: Aldrin Piri <[email protected]>
Date: 2015-02-14T19:13:49Z
Adding an implementation of validation for JsonPath, providing a sample
JSON file, and creating an associated test class.
commit 40da65f1937089ea3935f21da384f4bbbc36d20d
Author: Aldrin Piri <[email protected]>
Date: 2015-02-15T20:25:22Z
Adjusting filename of EvaluateJsonPath
commit c3c4d36944fa8a773d042edefa8cfa0f58edcd07
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T15:56:06Z
Providing validation of the input FlowFile as JSON
commit b3328490c65d0b08bed4f711b6990d10262bbef2
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:03:24Z
Completing initial functionality of EvaluateJsonPath and associated tests.
commit 974617d44ee41bd5cac9f238eca43cb9f81684a6
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:09:22Z
Adding a test to verify the placement of a JSON path expression with a
content destination.
commit 78ad0a3147cf1f3573f94f79685ebb868b5d190d
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:20:31Z
Adding a test for multiple attribute paths
commit d4a94c37eea9c4a545e9f764c113a3d863257e57
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:24:57Z
Adding a test for multiple attributes where neither evaluates to a found
path.
commit 6897090771434fc1553b722b835c3b67e8dd14a9
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:26:35Z
Adding a test for a destination of attributes and only one JsonPath
expression evaluates to a found result.
commit 7e581307b7d65da79464914b09fe8a90dd5046e9
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:33:48Z
Adding tests for indefinite results with and without the usage of operators.
commit 5b145e10e8508bec5293aab7318978fc3c9e2b57
Author: Aldrin Piri <[email protected]>
Date: 2015-02-16T23:34:23Z
Adjusting error where the return type of string was omitted from the
possible selections
commit bcebba6632165b6d69eb88ad12895e7678079267
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T03:44:06Z
Refining logic of how errors are handled on a per destination basis.
Adding supporting tests to ensure contract is met.
commit 0a19ada0a3229b1c8a72fe638f286db33b6f7ba6
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T04:26:45Z
Adjusting comments and providing a test to ensure a non-scalar value for a
specified scalar return type is routed to failure.
commit 5a81f19b25bc073ec1089c115b6e6fcf01056f91
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T14:44:30Z
Moving the JsonPath Validator to the JsonPathUtil class so that it can be
reused by other processors.
commit 59ad1948519fa1f8acccfb6eafbda04684edbc96
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T15:26:18Z
Refactoring common JSON/JsonPath functionality into JsonUtils
commit 2e05dcbbfdb37843456dff121d9878d8679d7067
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T20:15:53Z
Providing a SplitJson processor which will break JSON Arrays into their
individual elements. Refactored supporting JsonUtils code and EvaluateJsonPath
to reuse common functionality.
commit 7a74dd942008bf2178634746e70266f46657d4ef
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T20:50:42Z
Adding a test case for SplitJson where the specified path cannot be found.
Adjusting documentation for additional detail.
commit 1a746212bf2e13680d71dd12dfb60df3a20784ab
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T21:04:21Z
Removing unnessecary local variable
commit 627bd91fa719748b850b6c610a89b44be3e6f35e
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T21:15:33Z
Adding an auto return type for EvaluateJsonPath to match the semantics of
the EvaluateXPath processor.
commit a058fd0460dd5ca91560b1239dd733da37cfa782
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T21:58:04Z
Adding license terms for asm dependency introduced via JsonPath ->
Json-Smart
commit 408f636361cbad025752978e10775651b7daddf4
Author: Aldrin Piri <[email protected]>
Date: 2015-02-17T22:01:06Z
Preferring the constant empty string for clarity instead of quotes.
----
> Create Processors to work against JSON data
> -------------------------------------------
>
> Key: NIFI-360
> URL: https://issues.apache.org/jira/browse/NIFI-360
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Aldrin Piri
> Priority: Minor
> Labels: processor
>
> I have created two Processors, EvaluateJsonPath and SplitJson which are
> analogs of the functionality provided through EvaluateXPath and SpiltXML.
> Both are powered primarily around the usage of [JsonPath by
> Jayway|https://github.com/jayway/JsonPath].
> Their capability descriptions are provided below:
> {panel:title= EvaluateJsonPath}
> Evaluates one or more JsonPath expressions against the content of a FlowFile.
> The results of those expressions are assigned to FlowFile Attributes or are
> written to the content of the FlowFile itself, depending on configuration of
> the Processor. JsonPaths are entered by adding user-defined properties; the
> name of the property maps to the Attribute Name into which the result will be
> placed (if the Destination is flowfile-attribute; otherwise, the property
> name is ignored).
> The value of the property must be a valid JsonPath expression. If the
> JsonPath evaluates to a JSON array or JSON object and the Return Type is set
> to 'scalar' the FlowFile will be unmodified and will be routed to failure. A
> Return Type of JSON can return scalar values if the provided JsonPath
> evaluates to the specified value and will be routed as a match. If
> Destination is 'flowfile-content' and the JsonPath does not evaluate to a
> defined path, the FlowFile will be routed to 'unmatched' without having its
> contents modified. If Destination is flowfile-attribute and the expression
> matches nothing, attributes will be created with empty strings as the value,
> and the FlowFile will always be routed to 'matched.'
> {panel}
> {panel:title=SplitJson}
> Splits a JSON File into multiple, separate FlowFiles for an array element
> specified by a JsonPath expression. Each generated FlowFile is comprised of
> an element of the specified array and transferred to relationship 'split,
> with the original file transferred to the 'original' relationship. If the
> specified JsonPath is not found or does not evaluate to an array element,
> the original file is routed to 'failure' and no files are generated.
> {panel}
> One item of note is the transitive dependency of ASM through Json-Smart
> through JsonPath.
> I have included, what I believe is needed to appropriately make use of this
> item in the LICENSE. Review of its correctness would is requested.
> Any feedback is appreciated. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
