Evan, That was definitely it. Changing from .toString() to .readLines() gets the data appropriately for CSV parsing.
Thanks, John -------------------------------------------- On Mon, 6/17/19, Evan Graham <[email protected]> wrote: Subject: Re: ExecuteScript with python and CSV parsing... To: [email protected], "John McGinn" <[email protected]> Date: Monday, June 17, 2019, 4:13 PM Hi John, I haven't tested this but I believe the the issue is that the CSV reader want to iterate on lines, instead it's being given a single string.Try this instead: Text = IOUtils.readLines(inputStream, StandardCharsets.UTF_8) Thanks On Mon, Jun 17, 2019 at 8:00 PM John McGinn <[email protected]> wrote: First off, I am far from a Python expert, so this may be simple to others, but just not me. The incoming FlowFile is a CSV file of many lines, we are trying to parse them into arrays to then search for previous records or future records based on the date which is 1 of the CSV fields. The python code appears the same as what I've seen online elsewhere, but the problem is that each call to csv.reader() is returning a single letter. I've gone through the CookBooks by Matt Burgess, and searched for things which talk about the .read() method having an issue, but all I have is the inputStream in the PyCallback. I attempted to convert the inputStream into a BufferedStreamReader to see if that would work, but the csv.reader() docs say the first param has to be an iterator, or iterator-able. class PyStreamCallback(StreamCallback): def __init__(self): pass def process(self, inputStream, outputStream): try: Text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) reader = csv.reader(Text,delimiter=',') for row in reader: outputStream.write("~".join(row)) outputStream.write("->") This is a scaled down part of the code, but basically, each time through the for loop, I get a single letter printed out before the arrow hits, rather than the entire comma-seperated field. Currently using NiFi 1.7.0, so I'm not sure if this is something that is fixed, or if a new version of jython with the newer NiFi's fixes this. Data is as follows: 2019-06-05 00:00:28.008,f_Some.Data.Value,0,0.0 2019-06-05 00:01:28.553,f_Some.Other.Data.Value,0,0.0 2019-06-05 00:02:29.098,f_Last01.Data.Value,0,0.0 Thoughts? Thanks, John
