markap14 commented on pull request #4948:
URL: https://github.com/apache/nifi/pull/4948#issuecomment-907546294
@simonbence I got a chance to start testing these processors this evening.
The first thing that I did was to look at the Usage to understand how to use
the processors. Unfortunately, they don't really provide much of any
documentation at all. I think each of these needs to have their own
additionalDetails.html. This is really important for any & all scripted
components because otherwise users are just kind of taking a shot in the dark,
guessing how to use it. Would recommend looking at the additionalDetails of
ScriptedTransformRecord. Specifically, it will be important to explain how the
return values are to be returned. For example, Jython won't allow you to use a
`return true` syntax - you have to set `_ = true`, etc. This is important to
call out. It's also crucial to call out all of the variable bindings so that
users know what they have access to, and to ensure that we provide examples - I
would say at least 1 example in Groovy and 1 in Jython would be ideal, as these
are by
far the most commonly used.
I did also notice that there was an additionalDetails from the
ScriptedRouteRecord, which is the one we removed. So we need to make sure that
its additionalDetails.html is removed. Additionally, the other Scripted
components list ScriptedRouteRecord in their @SeeAlso annotation, so we'll want
to remove that also.
In testing, I wanted to see how it handled an Exception being thrown, so I
created a simple Groovy script:
```
throw new RuntimeException("Oops")
```
The processor (ScriptedValidateRecord in this case) did properly route the
incoming FlowFile to failure. However, the error message that I got was not
accurate:
```
2021-08-27 21:23:42,129 ERROR [Timer-Driven Process Thread-8]
o.a.n.p.script.ScriptedValidateRecord
ScriptedValidateRecord[id=8a4da405-017b-1000-54c8-e807155bed3b] Failed to route
records for
StandardFlowFileRecord[uuid=71d42d80-8e63-4d13-815c-ae0106eeed3b,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1630113179238-1, container=default,
section=1], offset=243,
length=100],offset=0,name=71d42d80-8e63-4d13-815c-ae0106eeed3b,size=100]:
org.apache.nifi.processor.exception.ProcessException: Failed to parse incoming
FlowFile
org.apache.nifi.processor.exception.ProcessException: Failed to parse
incoming FlowFile
at
org.apache.nifi.processors.script.ScriptedRouterProcessor$1.process(ScriptedRouterProcessor.java:194)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2466)
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2434)
at
org.apache.nifi.processors.script.ScriptedRouterProcessor.route(ScriptedRouterProcessor.java:126)
at
org.apache.nifi.processors.script.ScriptedRouterProcessor.onTrigger(ScriptedRouterProcessor.java:107)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1202)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.script.ScriptException: javax.script.ScriptException:
java.lang.RuntimeException: Oops
at
org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:162)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at
org.apache.nifi.processors.script.InterpretedScriptEvaluator.evaluate(InterpretedScriptEvaluator.java:47)
at
org.apache.nifi.processors.script.ScriptedRouterProcessor$1.process(ScriptedRouterProcessor.java:143)
... 16 common frames omitted
Caused by: javax.script.ScriptException: java.lang.RuntimeException: Oops
at
org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:324)
at
org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:159)
... 19 common frames omitted
Caused by: java.lang.RuntimeException: Oops
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
at
org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:237)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:249)
at Script7.run(Script7.groovy:1)
at
org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:321)
... 20 common frames omitted
```
There was no error parsing the incoming FlowFile - it parsed just fine. The
error was in evaluating the script against the record. It will be important
that we give an accurate error message. And because a FlowFile may have many
(often thousands or more) records, it will be much easier to understand what
the problem is if the error message indicates which Record was being processed
when it failed. For example: "Failed to process Record 529 in FlowFile[...]"
Otherwise, I think all is looking good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
