You should not need to explicitly set the additional module directory to cover that location. Is there a reason you can’t use the native Groovy JSON [1] parsing? That way you don’t have to download any additional libraries.
[1] http://groovy-lang.org/json.html# <http://groovy-lang.org/json.html#> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 29, 2020, at 7:41 AM, Myklebust, Bjørn Magnar > <bjorn.mykleb...@skatteetaten.no> wrote: > > Andy, just a quick followup on this. > > I wanted to test a groovy-script with this code (not finished by far yet), > and the script is placed in the Script Body part of an > ExecuteGroovyScript-process in NiFi: > > > import org.json.JSONObject > import org.json.XML > import org.apache.commons.io.IOUtils > import java.nio.charset.* > > def flowFile = session.get() > if (!flowFile) return > > flowFile = session.write(flowFile, > {inputStream, outputStream -> > def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) > def xmlJSONObj = XML.toJSONObject(text); > def json = xmlJSONObj.toString(); > outputStream.write(json.getBytes(StandardCharsets.UTF_8)) > } as StreamCallback) > > session.transfer(flowFile, ExecuteScript.REL_SUCCESS) > > But when trying to run this I get the message «unable to resolve class > org.json.JSONObject @ line 1» > I have downloaded the jar file from this site: > https://repo1.maven.org/maven2/org/json/json/20200518/json-20200518.jar > <https://repo1.maven.org/maven2/org/json/json/20200518/json-20200518.jar> > And placed it in my nifi/lib-directory. > And the content of this jar you can see in the enclosed png-picture. > > Do I need to set a value for the property Additional Classpath when the > jar-file is stored in the lib-directory? > > Thanks, > Bjørn > > > Fra: Andy LoPresto <alopre...@apache.org> > Sendt: torsdag 25. juni 2020 19:20 > Til: users@nifi.apache.org > Emne: Re: Replacing a base64-encoded field in a JSON-document with its > decoded/converted value > > Hi Bjørn, > > No, XML to JSON conversion is not an Expression Language feature. You’ll need > to either get this data into a flowfile as the complete content to perform > the conversion with existing built-in tools, or add that step to your Groovy > script. > > With that additional requirement, I think using the Groovy script to perform > those steps in tandem is probably the most performant and logical approach > here. > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 24, 2020, at 11:25 PM, Myklebust, Bjørn Magnar > <bjorn.mykleb...@skatteetaten.no <mailto:bjorn.mykleb...@skatteetaten.no>> > wrote: > > Thanks Andy. > The XML-content is around 5 kB-ish. But I also need to convert the XML to > JSON before replacing it back into the original JSON-file. Can this be done > with e.g a ConvertAttribute before the ReplaceText? > > Thanks, > Bjørn > > > > Fra: Andy LoPresto <alopre...@apache.org <mailto:alopre...@apache.org>> > Sendt: onsdag 24. juni 2020 17:24 > Til: users@nifi.apache.org <mailto:users@nifi.apache.org> > Emne: Re: Replacing a base64-encoded field in a JSON-document with its > decoded/converted value > > Hello Bjørn, > > If the size of the encoded XML document is small (under ~1 KB), you can > extract the Base64-encoded value to a flowfile attribute using > EvaluateJSONPath, perform the decoding using the base64Decode Expression > Language function [1], and then replace it into the flowfile JSON content > using ReplaceText (using some regex like "content": ".*" -> “content": > ”${decodedXML}” where decodedXML is the name of the attribute you are using). > > If the XML content could be very large, this will negatively affect your > performance, as attributes are stored directly in memory and handling large > amounts of data will impact the heap. In this case, I would recommend writing > a Groovy script in ExecuteScript processor to leverage Groovy’s very friendly > JSON handling and extract the value, Base64 decode it, and replace it in a > couple lines. > > Hope this helps. > > > [1] > https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64decode > > <https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#base64decode> > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Jun 24, 2020, at 4:24 AM, Myklebust, Bjørn Magnar > <bjorn.mykleb...@skatteetaten.no <mailto:bjorn.mykleb...@skatteetaten.no>> > wrote: > > > Hi. > I have a set of Json-files which contain a base64-coded field (Jsonpath to > this field is $.data.content), and this field contains a XML-document. > Decoding the field works as expected, so does the conversion from xml to > json, and I'm able to write the content from this field to a file in a > bucket in S3. But what I would like to do is to be able to replace the coded > value for this field in the original file with the decoded/converted value in > stead of writing the decoded/converted value to file. And after replacing the > json-value then I can write the updated Json-file to a new S3 bucket. > My process look like this at the moment, and works fine for getting the data > to file, but it's missing the last part of replacing $.data.content with the > decoded/converted data. > > So how can I do the last part? > > <pastedImage.png> > > The EvaluedJsonPath looks like this: > > <EvaluateJsonPath.png> > > > The ReplaceText looks like this: > > <ReplaceText.png> > > The Base64EncodeContent looks like this: > > <Base64Decode.png> > > and finally, the CovertRecord looks like this: > > <ConvertRecord.png> > > > > This is a testfile for that I'm working with: > > { > "header": { > "dokumentidentifikator": null, > "dokumentidentifikatorV2": "dcff985b-c652-4085-b8f1-45a2f4b6d150", > "revisjonsnummer": 1, > "dokumentnavn": > "Engangsavgiftfastsettelse:55TEST661122334455:44BIL1:2017-10-20", > "dokumenttype": "SKATTEMELDING_ENGANGSAVGIFT", > "dokumenttilstand": "OPPRETTET", > "gyldig": true, > "gjelderInntektsaar": 2017, > "gjelderPeriode": "2017_10", > "gjelderPart": { > "partsnummer": 5544332211, > "identifiseringstype": "MASKINELL", > "identifikator": null > }, > "opphavspart": { > "partsnummer": 5544332211, > "identifikator": null > }, > "kildereferanse": { > "kildesystem": "ENGANGSAVGIFTFASTSETTELSE", > "gruppe": "", > "referanse": "aef147fb-8ce8-43ef-833b-7aa3bac1ece0", > "tidspunkt": "2018-01-16T13:28:02.49+01:00" > } > }, > "data": { > "metadata": { > "format": "ske:fastsetting:motorvogn:motorvognavgift:v1", > "bytes": 4420, > "mimeType": "application/xml", > "sha1": "c0AowOsTdNdo6VufeSsZqTphc0Y=" > }, > "content": > "PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9InllcyI/Pgo8bW90b3J2b2duYXZnaWZ0IHhtbG5zPSJza2U6ZmFzdHNldHRpbmc6bW90b3J2b2duOm1vdG9ydm9nbmF2Z2lmdDp2MSI+CiAgICA8YXZnaWZ0c2xpbmplPgogICAgICAgIDxhdmdpZnRzYmVsb2VwPjU0Mjg5Ni4wMDwvYXZnaWZ0c2JlbG9lcD4KICAgICAgICA8YXZnaWZ0c29wcGx5c25pbmc+CiAgICAgICAgICAgIDxzYWVyYXZnaWZ0VHlwZWtvZGU+QkI8L3NhZXJhdmdpZnRUeXBla29kZT4KICAgICAgICAgICAgPHNhZXJhdmdpZnRHcnVwcGVrb2RlPlg8L3NhZXJhdmdpZnRHcnVwcGVrb2RlPgogICAgICAgIDwvYXZnaWZ0c29wcGx5c25pbmc+CiAgICAgICAgPGF2Z2lmdHNkYXRvPjIwMTctMTAtMjA8L2F2Z2lmdHNkYXRvPgogICAgPC9hdmdpZnRzbGluamU+CiAgICA8YmV0YWxpbmdzaW5mb3JtYXNqb24+CiAgICAgICAgPGtpZG51bW1lcj4xMDEwMTAxMDEwMTA8L2tpZG51bW1lcj4KICAgICAgICA8Zm9yZmFsbHNkYXRvPjIwMTctMTAtMjA8L2ZvcmZhbGxzZGF0bz4KICAgICAgICA8ZmFrdHVyYWRhdG8+MjAxNy0xMC0yMDwvZmFrdHVyYWRhdG8+CiAgICAgICAgPHRvdGFsQXZnaWZ0c2JlbG9lcD41NDI4OTYuMDA8L3RvdGFsQXZnaWZ0c2JlbG9lcD4KICAgIDwvYmV0YWxpbmdzaW5mb3JtYXNqb24+CiAgICA8bW90b3J2b2duYXZnaWZ0c3R5cGU+ZW5nYW5nc2F2Z2lmdDwvbW90b3J2b2duYXZnaWZ0c3R5cGU+CiAgICA8dGlkc3N0ZW1wZWw+MjAxOC0wMS0xNiswMTowMDwvdGlkc3N0ZW1wZWw+CiAgICA8Z3J1bm5sYWdGb3JNb3RvcnZvZ25hdmdpZnQ+CiAgICAgICAgPGtqb2VyaW5nZW5zQXJ0PjEwPC9ram9lcmluZ2Vuc0FydD4KICAgICAgICA8a2pvZXJldG9leT4KICAgICAgICAgICAgPGVpZXJza2FwUmVnaXN0cmVydD4yMDE3LTEwLTIwPC9laWVyc2thcFJlZ2lzdHJlcnQ+CiAgICAgICAgICAgIDxmb2Vyc3RlUmVnaXN0cmVyaW5nc2Fhcj4yMDE3PC9mb2Vyc3RlUmVnaXN0cmVyaW5nc2Fhcj4KICAgICAgICAgICAgPGZvZXJzdGVSZWdpc3RyZXJpbmdzZGF0b0lOb3JnZT4yMDE3LTEwLTIwPC9mb2Vyc3RlUmVnaXN0cmVyaW5nc2RhdG9JTm9yZ2U+CiAgICAgICAgICAgIDxram9lcmV0b2V5Z3J1cHBlPjEwMTwva2pvZXJldG9leWdydXBwZT4KICAgICAgICAgICAgPGxlbmdkZT4zOTY0PC9sZW5nZGU+CiAgICAgICAgICAgIDxtb3RvcmVmZmVrdD45NjwvbW90b3JlZmZla3Q+CiAgICAgICAgICAgIDxzbGFndm9sdW0+MTQzPC9zbGFndm9sdW0+CiAgICAgICAgICAgIDxkcml2c3RvZmY+QkVOU0lOPC9kcml2c3RvZmY+CiAgICAgICAgICAgIDxlZ2VudmVrdD4xNTE5PC9lZ2VudmVrdD4KICAgICAgICAgICAgPGVpZXI+CiAgICAgICAgICAgICAgICA8Zm9lZHNlbHNFbGxlckRudW1tZXI+MTEyMjMzNDQ1NTwvZm9lZHNlbHNFbGxlckRudW1tZXI+CiAgICAgICAgICAgICAgICA8cGFydHNudW1tZXI+NTU0NDMzMjIxMTwvcGFydHNudW1tZXI+CiAgICAgICAgICAgICAgICA8bmF2bj5LTEFSQSBLVTwvbmF2bj4KICAgICAgICAgICAgPC9laWVyPgogICAgICAgICAgICA8dGlsbGF0dFRvdGFsdmVrdD4yMTY0PC90aWxsYXR0VG90YWx2ZWt0PgogICAgICAgICAgICA8aHlicmlkPm5laTwvaHlicmlkPgogICAgICAgICAgICA8Y28ydXRzbGlwcD4yNjg8L2NvMnV0c2xpcHA+CiAgICAgICAgICAgIDxub3h1dHNsaXBwPjU5LjQ8L25veHV0c2xpcHA+CiAgICAgICAgICAgIDxram9lcmV0b2V5aWRlbnRpZmlrYXRvcj4KICAgICAgICAgICAgICAgIDxram9lcmV0b2V5VW5pa0lkZW50aWZpa2F0b3I+QUJDREVGR0hJSjwva2pvZXJldG9leVVuaWtJZGVudGlmaWthdG9yPgogICAgICAgICAgICAgICAgPGtqZW5uZW1lcmtlPjQ0QklMMTwva2plbm5lbWVya2U+CiAgICAgICAgICAgICAgICA8dW5kZXJzdGVsbHNudW1tZXI+VU5ERVJTVEVMTDQ0PC91bmRlcnN0ZWxsc251bW1lcj4KICAgICAgICAgICAgPC9ram9lcmV0b2V5aWRlbnRpZmlrYXRvcj4KICAgICAgICA8L2tqb2VyZXRvZXk+CiAgICA8L2dydW5ubGFnRm9yTW90b3J2b2duYXZnaWZ0PgogICAgPGF2Z2lmdHNwbGlrdGlnPgogICAgICAgIDxmb2Vkc2Vsc0VsbGVyRG51bW1lcj4xMTIyMzM0NDU1PC9mb2Vkc2Vsc0VsbGVyRG51bW1lcj4KICAgICAgICA8cGFydHNudW1tZXI+NTU0NDMzMjIxMTwvcGFydHNudW1tZXI+CiAgICA8L2F2Z2lmdHNwbGlrdGlnPgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5DbzI8L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjQ3NTMzNy4yMDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5FZ2VudmVrdDwva29tcG9uZW50PgogICAgICAgIDxiZWxvZXA+NjA5NDUuNjQ8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+TW90b3JlZmZla3Q8L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjAuMDA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+Tk94PC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD40MjEzLjI0PC9iZWxvZXA+CiAgICA8L2F2Z2lmdHNrb21wb25lbnQ+CiAgICA8YXZnaWZ0c2tvbXBvbmVudD4KICAgICAgICA8a29tcG9uZW50PlNsYWd2b2x1bTwva29tcG9uZW50PgogICAgICAgIDxiZWxvZXA+MC4wMDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5DbzIgRnJhdHJla2s8L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjAuMDA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+RWdlbnZla3QgRnJhdHJla2s8L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjAuMDA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+TW90b3JlZmZla3QgRnJhdHJla2s8L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjAuMDA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+Tk94IEZyYXRyZWtrPC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD4wLjAwPC9iZWxvZXA+CiAgICA8L2F2Z2lmdHNrb21wb25lbnQ+CiAgICA8YXZnaWZ0c2tvbXBvbmVudD4KICAgICAgICA8a29tcG9uZW50PlNsYWd2b2x1bSBGcmF0cmVrazwva29tcG9uZW50PgogICAgICAgIDxiZWxvZXA+MC4wMDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5DbzIgU3VtPC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD40NzUzMzcuMjA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+RWdlbnZla3QgU3VtPC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD42MDk0NS42NDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5Nb3RvcmVmZmVrdCBTdW08L2tvbXBvbmVudD4KICAgICAgICA8YmVsb2VwPjAuMDA8L2JlbG9lcD4KICAgIDwvYXZnaWZ0c2tvbXBvbmVudD4KICAgIDxhdmdpZnRza29tcG9uZW50PgogICAgICAgIDxrb21wb25lbnQ+Tk94IFN1bTwva29tcG9uZW50PgogICAgICAgIDxiZWxvZXA+NDIxMy4yNDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGF2Z2lmdHNrb21wb25lbnQ+CiAgICAgICAgPGtvbXBvbmVudD5TbGFndm9sdW0gU3VtPC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD4wLjAwPC9iZWxvZXA+CiAgICA8L2F2Z2lmdHNrb21wb25lbnQ+CiAgICA8YXZnaWZ0c2tvbXBvbmVudD4KICAgICAgICA8a29tcG9uZW50PlZyYWtwYW50PC9rb21wb25lbnQ+CiAgICAgICAgPGJlbG9lcD4yNDAwPC9iZWxvZXA+CiAgICA8L2F2Z2lmdHNrb21wb25lbnQ+CiAgICA8YXZnaWZ0c2tvbXBvbmVudD4KICAgICAgICA8a29tcG9uZW50PkJydWtzZnJhZHJhZyAwJTwva29tcG9uZW50PgogICAgICAgIDxiZWxvZXA+MC4wMDwvYmVsb2VwPgogICAgPC9hdmdpZnRza29tcG9uZW50PgogICAgPGZvcmhhYW5kc2JlcmVnbmluZz5mYWxzZTwvZm9yaGFhbmRzYmVyZWduaW5nPgo8L21vdG9ydm9nbmF2Z2lmdD4K" > }, > "extension": null, > "skjemaversjon": "v3_0" > } > > > > > Cheers, > Bjørn > > > Denne e-posten og eventuelle vedlegg er beregnet utelukkende for den > institusjon eller person den er rettet til og kan være belagt med lovbestemt > taushetsplikt. Dersom e-posten er feilsendt, vennligst slett den og kontakt > Skatteetaten. > The contents of this email message and any attachments are intended solely > for the addressee(s) and may contain confidential information and may be > legally protected from disclosure. If you are not the intended recipient of > this message, please immediately delete the message and alert the Norwegian > Tax Administration. > > <org.json.JSONObject.png><executegroovyscript.png>
