A little further investigation/tinkering:
I suspect the ReplaceText issue is that the JSON is being parsed and the
duplicate metadata field is causing a problem, then one of them is being thrown
away
adding a teplaceFirst function call the remove the ,"metadata":{} from the
attribute first has fixed the issue… but that’s not particularly great (and
caused by the original metadata duplication caused by jsonPathPut)
additionally jsonPathPut can only add scalars, so I can’t create the metadata
object as part of UpdateAttribtue process, it needs to exist beforehand (which
I thankfully have the opportunity to do, but it’s frustrating)
would be helpful if jsonPathPut could either put objects as well as scalars or,
given a non-existent nested path, e.g. $.metadata.a_key, created the
intermediate objects in order to add the key
if I create & initialise the metadata object prior to the UpdateAttribute
processor, e.g. $.metadata.is_initialised: true, then the jsonPathPut for
a_key: a_value works as expected (i.e. uses the existing object, not by
creating a new one); but then trying to tidy up the unwanted field with a
jsonPathDelete('$.metadata.is_initialised') call in UpdateAttribute results in
all fields of metadata being deleted (i.e. a_key is also deleted)! Although
this only seems to happen when the object from which I’m trying to delete data
using jsonPathDelete has previously had data added to it with jsonPathPut -
objects that hadn’t previously been modified can have individual fields deleted
without issue.
So I’m having to fudge things a fair amount in order to manipulate the JSON
within the attribute (create/add fields to an object, delete fields from the
object
Cheers,
--
Chris Sampson
t: +44 7867 843675
e: [email protected]
> On 6 Mar 2020, at 12:45, Chris Sampson <[email protected]> wrote:
>
> Hi,
>
> (Copied form Slack channel)
>
> I’m seeing some confusing behaviours with the Expression Language json path
> functions and ReplaceText for adding JSON into a file (1.11.3):
> given input of (read into a FlowFile attribute named record_json via the
> EvaluateJsonPath processor):
> {
> "person": {
> "addresses": [
> {"current": true, "postcode": "test"}
> ]
> },
> "metadata": {}
> }
> UpdateAttribute to set address_1 as
> ${record_json:jsonPath('$.person.addresses[?(@.current)].address_1')} results
> in a value of []. yet I’d have expected an empty String (as the json path
> doesn’t exist) - doing the same but for postcode correctly returns a value of
> test
> UpdateAttribute to try and update record_json with
> ${record_json:jsonPathPut('$.metadata', 'a_key', 'a_value')} results in the
> following output:
> {
> "person": {
> "addresses": [
> {"current": true, "postcode": "test"}
> ]
> },
> "metadata": {},
> "metadata": {"a_key": "a_value"}
> }
> i.e. there’s a duplicated metadata object, where I’d have expected the new
> key/value pair to be added to the existing object instead (but can’t use
> jsonPathAdd as that’s for arrays, not objects
> Also, when I try to then ReplaceText to put the attribute back into the file
> content (for various reasons I need it as both file content and an attribute
> later in the Flow), the above example output seems to lose the additional
> metadata object and just the blank object remains, i.e. a ReplaceText
> processor with settings Search Value = (?s)$.*$, Replacement Value =
> ${record_json}, Replacement Strategy = Regex Replace, Evaluation Mode =
> Entire text and the above example output gives file content of:
> {
> "person": {
> "addresses": [
> {"current": true, "postcode": "test"}
> ]
> },
> "metadata": {}
> }
>
>
>
> Cheers,
>
> --
> Chris Sampson
>
> t: +44 7867 843675
> e: [email protected] <mailto:[email protected]>
