Many thanks to you guys for your suggestions. With your help I was able to
overcome this challenge. Here is the section of code that I modified:
flowFiles = session.get(20)
for flowFile in flowFiles :
if (flowFile != None) :
flowFile = session.write(flowFile,PyStreamCallback())
# only get back result with length 1 if invalid JSON...
if (len(result) == 1) :
session.transfer(flowFile, REL_FAILURE)
else :
if (
result.get('application','x') in
valid_application
and result.get('environment','x') in
valid_environment
and result.get('fromComponent','x') in
valid_from
and result.get('toComponent','x') in
valid_to
and result.get('messageComponent','x') in
valid_messageComponent
and result.get('messageTypeState','x') in
valid_messageTypeState
and result.get('messageType','x') in
valid_messageType
) :
flowFile = session.putAttribute(flowFile,
'isValidMessage', 'True')
else :
flowFile = session.putAttribute(flowFile,
'isValidMessage', 'False')
Attribute isValidMessage is later used by RouteAttribute to dictate the
course to an HandleHttpResponse - to a 202 if the incoming message is
valid, to a 400 if the incoming json message lacks required key-value pairs.
Thanks again.
Jim
On Wed, Jul 12, 2017 at 1:04 PM, James McMahon <[email protected]> wrote:
> Thanks very much Scott. I intend to try the logic improvements you, Andy,
> and Matt have discussed and will circle back with the results tomorrow. -Jim
>
> On Wed, Jul 12, 2017 at 12:46 PM, Scott Wagner <[email protected]>
> wrote:
>
>> Python/Jython dictionaries also support a get() method that allows you to
>> provide a value to use if the key is not in the dictionary.
>>
>> The question is what do you want to have happen if your result dictionary
>> doesn't contain the value? Do you want it to continue on, or do you want
>> it to fail?
>>
>> if result.get('application', '') in valid_application and
>> result.get('environment', '') in valid_environment ...
>>
>> If you want it to continue on if they are missing, then add an empty
>> string to each of the sets you are comparing. Also, if an empty string is
>> a possible input and you want to have the missing value be different, you
>> can specify a different default as the second argument to the get() method.
>>
>> The KeyError is thrown whenever you try to reference a key that isn't in
>> the dictionary. This page[1] might provide you some more helpful details.
>>
>> - Scott
>>
>> [1] https://wiki.python.org/moin/KeyError
>>
>> James McMahon <[email protected]>
>> Wednesday, July 12, 2017 4:43 AM
>> Thank you very much, Matt and Andy. I will explore these suggested
>> improvements to my logic this evening and post the results. Cheers, Jim
>>
>>
>> Matt Burgess <[email protected]>
>> Tuesday, July 11, 2017 8:25 PM
>> Jim,
>>
>> You can check first if the key is in the results dictionary and then
>> if the value is in the valid values dictionary:
>>
>> ...
>> and 'environment' in result and result['environment'] in valid_environment
>> ...
>>
>> if a missing value is "valid", then you can use "or" instead of "and"
>> and put the clause in parens:
>>
>> ...
>> and ('environment' in result or result['environment'] in
>> valid_environment)
>> ...
>>
>>
>> If I misunderstood what you were asking please let me know and I'll try
>> again :)
>>
>> Regards,
>> Matt
>>
>> James McMahon <[email protected]>
>> Tuesday, July 11, 2017 8:12 PM
>> Good evening. I am in a Jython 2.7.0 environment, using NiFi 1.3.0.
>>
>> In my script I check for validity in dictionary key values, like so:
>>
>> if (
>> result['application'] in valid_application
>> and result['environment'] in
>> valid_environment
>> and result['fromComponent'] in valid_from
>> and result['toComponent'] in valid_to
>> and result['messageComponent'] in
>> valid_messageComponent
>> and result['messageTypeState'] in
>> valid_messageTypeState
>> and result['messageType'] in
>> valid_messageType
>> ) :
>> flowFile = session.putAttribute(flowFile,
>> 'isValidMessage', 'True')
>> else :
>> flowFile = session.putAttribute(flowFile,
>> 'isValidMessage', 'False')
>>
>> I curl an Http POST that populates all the key values. When they are all
>> there, the script works wonderfully. But when any one has no value in the
>> incoming json submitted via POST, it throws an error. For example, like so
>> , "messageComponent":""
>>
>> The error thrown and logged by NiFi to nifi-app.log is this:
>>
>> 2017-07-12 01:03:55,578 ERROR [Timer-Driven Process Thread-3]
>> o.a.nifi.processors.script.ExecuteScript
>> ExecuteScript[id=61381fb9-1012-115d-2a56-9f9e7fe9f382]
>> Failed to process session due to
>> org.apache.nifi.processor.exception.ProcessException:
>> javax.script.ScriptException: KeyError: 'environment' in <script> at line
>> number 254: {}
>> org.apache.nifi.processor.exception.ProcessException:
>> javax.script.ScriptException: KeyError: 'environment' in <script> at line
>> number 254
>> at org.apache.nifi.processors.script.ExecuteScript.onTrigger(
>> ExecuteScript.java:230)
>> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(S
>> tandardProcessorNode.java:1120)
>> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>> .call(ContinuallyRunProcessorTask.java:147)
>> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>> .call(ContinuallyRunProcessorTask.java:47)
>> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingA
>> gent$1.run(TimerDrivenSchedulingAgent.java:132)
>> at java.util.concurrent.Executors$RunnableAdapter.call(
>> Executors.java:511)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:
>> 308)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.run(ScheduledThreadPoolExecutor.java:294)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: javax.script.ScriptException: KeyError: 'environment' in
>> <script> at line number 254
>> at org.python.jsr223.PyScriptEngine.scriptException(PyScriptEng
>> ine.java:202)
>> at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:42)
>> at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)
>> at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.
>> java:264)
>> at org.apache.nifi.script.impl.JythonScriptEngineConfigurator.
>> eval(JythonScriptEngineConfigurator.java:59)
>> at org.apache.nifi.processors.script.ExecuteScript.onTrigger(
>> ExecuteScript.java:220)
>> ... 11 common frames omitted
>> Caused by: org.python.core.PyException: null
>> at org.python.core.Py.KeyError(Py.java:249)
>> at org.python.core.PyObject.__getitem__(PyObject.java:738)
>> at org.python.pycode._pyx15333737.f$0(<script>:245)
>> at org.python.pycode._pyx15333737.call_function(<script>)
>> at org.python.core.PyTableCode.call(PyTableCode.java:167)
>> at org.python.core.PyCode.call(PyCode.java:18)
>> at org.python.core.Py.runCode(Py.java:1386)
>> at org.python.core.__builtin__.eval(__builtin__.java:497)
>> at org.python.core.__builtin__.eval(__builtin__.java:501)
>> at org.python.util.PythonInterpreter.eval(PythonInterpreter.
>> java:259)
>> at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:40)
>> ... 15 common frames omitted
>>
>> What can I do to avoid this error, since I am unable to determine why the
>> empty incoming value is causing it to occur?
>>
>> Thanks in advance for your help.
>>
>> Cheers,
>>
>> Jim
>>
>>
>>
>
