Re: Groovy script error, JsonSlurper
I believe it's your "json.each { jsonObj ->" line, with a JSON object
it's going to return a key/value pair so try "json.each { k,v ->"
instead and use the key k and value v in your script.
On Mon, May 8, 2023 at 9:32 PM James McMahon wrote:
>
> Hello. I have incoming data that is json. An example of one case looks like
> this:
>
> {"id": "20230508215236_4447cd0a-9dca-47cb-90b1-6562cf34155a_Timer-Driven
> Process Thread-9",
> "te": "0.9494",
> "diskusage": "0.2776125422110003.3 MB",
> "memory": 77,
> "cpu": 0.58,
> "host": "172.31.73.197/ip-172-31-73-197.ec2.internal",
> "temperature": "97",
> "macaddress": "f417ead3-4fa9-4cee-a14b-7172e9ecd3ea",
> "end": "61448816405795",
> "systemtime": "05/08/2023 16:52:36"}
>
>
> I wrote a Groovy script running in a nifi ExecuteScript processor to tally
> the keys from the json, reporting the results as flowfile attributes. I am
> getting this error:
>
> ExecuteScript[id=028b1d40-33a5-1766-b659-31b3faaf13f5] Error processing json
> fields: groovy.lang.MissingMethodException: No signature of method:
> Script9$_run_closure1$_closure2$_closure4.doCall() is applicable for argument
> types: (Entry) values:
> [id=20230508215236_4447cd0a-9dca-47cb-90b1-6562cf34155a_Timer-Driven Process
> Thread-9]
> Possible solutions: doCall(java.lang.Object, java.lang.Object), findAll(),
> findAll(), isCase(java.lang.Object), isCase(java.lang.Object)
>
>
>
> What is causing this error? Am I reading in the data incorrectly using the
> JsonSlurper?
>
>
>
> Here is my code:
>
> import groovy.json.JsonSlurper
> import groovy.json.JsonOutput
> import org.apache.commons.io.IOUtils
> import java.nio.charset.StandardCharsets
>
> def originalFile = ''
> def keys = []
> def tallyMap = [:]
> def topValuesMap = [:]
> def lineCount = 0
>
> def ff = session.get()
> if (!ff) return
> try {
> session.read(ff, { inputStream ->
> def json = new JsonSlurper().parseText(IOUtils.toString(inputStream,
> StandardCharsets.UTF_8))
>
>
> def keySet = new HashSet()
>
> if (json instanceof List) {
> keySet.addAll(((Map)json[0]).keySet())
> } else {
> keySet.addAll(json.keySet())
> }
>
> keys = keySet.toList()
> originalFile = ff.getAttribute('filename')
>
>
> json.each { jsonObj ->
> jsonObj.each { key, value ->
> if (value != null && !value.toString().trim().isEmpty()) {
> tallyMap[key] = tallyMap.containsKey(key) ? tallyMap[key]
> + 1 : 1
> if (topValuesMap.containsKey(key)) {
> def valuesMap = topValuesMap[key]
> valuesMap[value] = valuesMap.containsKey(value) ?
> valuesMap[value] + 1 : 1
> topValuesMap[key] = valuesMap
> } else {
> topValuesMap[key] = [:].withDefault{ 0 }.plus([value:
> 1])
> }
> // Remove the "value: 1" entry from the topValuesMap
> topValuesMap[key].remove("value")
> }
> }
> }
>
> // Sort the topValuesMap for each key based on the frequency of values
> topValuesMap.each { key, valuesMap ->
> topValuesMap[key] = valuesMap.sort{ -it.value }.take(10)
> }
>
>
> // Count the number of JSON records
> lineCount += json.size()
> } as InputStreamCallback)
>
> def tallyMapString = JsonOutput.toJson(tallyMap)
> def topValuesMapString = JsonOutput.toJson(topValuesMap)
>
> ff = session.putAttribute(ff, 'triage.json.file', originalFile)
> ff = session.putAttribute(ff, 'triage.json.fields', keys.join(","))
> ff = session.putAttribute(ff, 'triage.json.tallyMap', tallyMapString)
> ff = session.putAttribute(ff, 'triage.json.topValuesMap',
> topValuesMapString)
> ff = session.putAttribute(ff, 'triage.json.lineCount',
> lineCount.toString())
> session.transfer(ff, REL_SUCCESS)
>
> } catch (Exception e) {
> log.error('Error processing json fields', e)
> session.transfer(ff, REL_FAILURE)
> }
>
>
>
Groovy script error, JsonSlurper
Hello. I have incoming data that is json. An example of one case looks like
this:
{"id": "20230508215236_4447cd0a-9dca-47cb-90b1-6562cf34155a_Timer-Driven
Process Thread-9",
"te": "0.9494",
"diskusage": "0.2776125422110003.3 MB",
"memory": 77,
"cpu": 0.58,
"host": "172.31.73.197/ip-172-31-73-197.ec2.internal",
"temperature": "97",
"macaddress": "f417ead3-4fa9-4cee-a14b-7172e9ecd3ea",
"end": "61448816405795",
"systemtime": "05/08/2023 16:52:36"}
I wrote a Groovy script running in a nifi ExecuteScript processor to tally
the keys from the json, reporting the results as flowfile attributes. I am
getting this error:
ExecuteScript[id=028b1d40-33a5-1766-b659-31b3faaf13f5] Error
processing json fields: groovy.lang.MissingMethodException: No
signature of method:
Script9$_run_closure1$_closure2$_closure4.doCall() is applicable for
argument types: (Entry) values:
[id=20230508215236_4447cd0a-9dca-47cb-90b1-6562cf34155a_Timer-Driven
Process Thread-9]
Possible solutions: doCall(java.lang.Object, java.lang.Object),
findAll(), findAll(), isCase(java.lang.Object),
isCase(java.lang.Object)
What is causing this error? Am I reading in the data incorrectly using
the JsonSlurper?
Here is my code:
import groovy.json.JsonSlurper
import groovy.json.JsonOutput
import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
def originalFile = ''
def keys = []
def tallyMap = [:]
def topValuesMap = [:]
def lineCount = 0
def ff = session.get()
if (!ff) return
try {
session.read(ff, { inputStream ->
def json = new
JsonSlurper().parseText(IOUtils.toString(inputStream,
StandardCharsets.UTF_8))
def keySet = new HashSet()
if (json instanceof List) {
keySet.addAll(((Map)json[0]).keySet())
} else {
keySet.addAll(json.keySet())
}
keys = keySet.toList()
originalFile = ff.getAttribute('filename')
json.each { jsonObj ->
jsonObj.each { key, value ->
if (value != null && !value.toString().trim().isEmpty()) {
tallyMap[key] = tallyMap.containsKey(key) ?
tallyMap[key] + 1 : 1
if (topValuesMap.containsKey(key)) {
def valuesMap = topValuesMap[key]
valuesMap[value] =
valuesMap.containsKey(value) ? valuesMap[value] + 1 : 1
topValuesMap[key] = valuesMap
} else {
topValuesMap[key] = [:].withDefault{ 0
}.plus([value: 1])
}
// Remove the "value: 1" entry from the topValuesMap
topValuesMap[key].remove("value")
}
}
}
// Sort the topValuesMap for each key based on the frequency of values
topValuesMap.each { key, valuesMap ->
topValuesMap[key] = valuesMap.sort{ -it.value }.take(10)
}
// Count the number of JSON records
lineCount += json.size()
} as InputStreamCallback)
def tallyMapString = JsonOutput.toJson(tallyMap)
def topValuesMapString = JsonOutput.toJson(topValuesMap)
ff = session.putAttribute(ff, 'triage.json.file', originalFile)
ff = session.putAttribute(ff, 'triage.json.fields', keys.join(","))
ff = session.putAttribute(ff, 'triage.json.tallyMap', tallyMapString)
ff = session.putAttribute(ff, 'triage.json.topValuesMap',
topValuesMapString)
ff = session.putAttribute(ff, 'triage.json.lineCount', lineCount.toString())
session.transfer(ff, REL_SUCCESS)
} catch (Exception e) {
log.error('Error processing json fields', e)
session.transfer(ff, REL_FAILURE)
}
[ANNOUNCE] Groovy 4.0.12 Windows installer released
Windows installer for Groovy 4.0.12 is now available. https://groovy.jfrog.io/ui/native/dist-release-local/groovy-windows-installer/groovy-4.0.12/groovy-4.0.12.msi -Keegan
[ANNOUNCE] Apache Groovy 4.0.12 Released
Dear community, The Apache Groovy team is pleased to announce version 4.0.12 of Apache Groovy. Apache Groovy is a multi-faceted programming language for the JVM. Further details can be found at the https://groovy.apache.org website. This release is a maintenance release of the GROOVY_4_0_X branch. It is strongly encouraged that all users using prior versions on this branch upgrade to this version. This release includes 31 bug fixes/improvements as outlined in the changelog: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318123=12353079 Sources, convenience binaries, downloadable documentation and an SDK bundle can be found at: https://groovy.apache.org/download.html We recommend you verify your installation using the information on that page. Jars are also available within the major binary repositories. We welcome your help and feedback and in particular want to thank everyone who contributed to this release. For more information on how to report problems, and to get involved, visit the project website at https://groovy.apache.org/ Best regards, The Apache Groovy team.
