Hi Matt, Do you have an example on how to use ExecuteScript on flowContent?
I have the following url encoded string as flow content, where i would like use python parse it to get flow artibutes based on key values pairs. rt.start=navigation&rt.tstart=1458797018682&rt.bstart=1458797019033&rt.end=1458797019075&t_resp=21&t_page=372&t_done=393&t_other=t_domloaded%7C364&r=http%3A%2F%2Flocalhost%3A63342%2FBeacon%2Ftest.html&r2=&u=http%3A%2F%2Flocalhost%3A63342%2FBeacon%2Ftest.html&v=0.9& vis.st=visible -Madhu On Thu, Mar 24, 2016 at 12:34 AM, Madhukar Thota <[email protected]> wrote: > Hi Matt, > > Thank you for the input. I updated my config as you suggested and it > worked like charm and also big thankyou for nice article. i used your > article as reference when i am started Exploring ExecuteScript. > > > Thanks > Madhu > > > > On Thu, Mar 24, 2016 at 12:18 AM, Matt Burgess <[email protected]> > wrote: > >> Madhukar, >> >> Glad to hear you found a solution, I was just replying when your email >> came in. >> >> Although in ExecuteScript you have chosen "python" as the script engine, >> it is actually Jython that is being used to interpret the scripts, not your >> installed version of Python. The first line (shebang) is ignored as it is >> a comment in Python/Jython. >> >> Modules installed with pip are not automatically available to the Jython >> engine, but if the modules are pure Python code (rather than native C / >> CPython), like user_agents is, you can import them one of two equivalent >> ways: >> >> 1) The way you have done, using sys.path.append. I should mention that >> "import sys" is done for you so you can safely leave that out if you wish. >> 2) Add the path to the packages ('/usr/local/lib/python2.7/site-packages') >> to the Module Path property of the ExecuteScript processor. In this case >> the processor effectively does Option #1 for you. >> >> I was able to get your script to work but had to force the result of >> parse (a UserAgent object) into a string, so I wrapped it in str: >> >> str(parse(flowFile.getAttribute('http.headers.User-Agent')).browser) >> >> You're definitely on the right track :) For another Jython example with >> ExecuteScript, check out this post on my blog: >> http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html >> >> I am new to Python as well, but am happy to help if I can with any issues >> you run into, as it will help me learn more as well :) >> >> Regards, >> Matt >> >> >> On Thu, Mar 24, 2016 at 12:10 AM, Madhukar Thota < >> [email protected]> wrote: >> >>> I was able to solve the python modules issues by adding the following >>> lines: >>> >>> import sys >>> sys.path.append('/usr/local/lib/python2.7/site-packages') # Path where >>> my modules are installed. >>> >>> Now the issue i have is , how do i parse the incoming attributes using >>> this libarary correctly and get the new fields. I am kind of new to python >>> and also this my first attempt of using python with nifi. >>> >>> Any help is appreciated. >>> >>> >>> >>> On Wed, Mar 23, 2016 at 11:31 PM, Madhukar Thota < >>> [email protected]> wrote: >>> >>>> Hi >>>> >>>> I am trying to use the following script to parse http.headers.useragent >>>> with python useragent module using ExecuteScript Processor. >>>> >>>> Script: >>>> >>>> #!/usr/bin/env python2.7 >>>> from user_agents import parse >>>> >>>> flowFile = session.get() >>>> if (flowFile != None): >>>> flowFile = session.putAttribute(flowFile, "browser", >>>> parse(flowFile.getAttribute('http.headers.User-Agent')).browser) >>>> session.transfer(flowFile, REL_SUCCESS) >>>> >>>> >>>> But ExecuteProcessor, complaining about missing python module but >>>> modules are already installed using pip and tested outside nifi. How can i >>>> add or reference this modules to nifi? >>>> >>>> Error: >>>> >>>> 23:28:03 EDT >>>> ERROR >>>> af354413-9866-4557-808a-7f3a84353597 >>>> ExecuteScript[id=af354413-9866-4557-808a-7f3a84353597] Failed to >>>> process session due to >>>> org.apache.nifi.processor.exception.ProcessException: >>>> javax.script.ScriptException: ImportError: No module named user_agents in >>>> <script> at line number 2: >>>> org.apache.nifi.processor.exception.ProcessException: >>>> javax.script.ScriptException: ImportError: No module named user_agents in >>>> <script> at line number 2 >>>> >>>> >>>> >>>> >>>> >>> >> >
