First off, I am far from a Python expert, so this may be simple to others, but
just not me.
The incoming FlowFile is a CSV file of many lines, we are trying to parse them
into arrays to then search for previous records or future records based on the
date which is 1 of the CSV fields. The python code appears the same as what
I've seen online elsewhere, but the problem is that each call to csv.reader()
is returning a single letter. I've gone through the CookBooks by Matt Burgess,
and searched for things which talk about the .read() method having an issue,
but all I have is the inputStream in the PyCallback. I attempted to convert the
inputStream into a BufferedStreamReader to see if that would work, but the
csv.reader() docs say the first param has to be an iterator, or iterator-able.
class PyStreamCallback(StreamCallback):
def __init__(self):
pass
def process(self, inputStream, outputStream):
try:
Text = IOUtils.toString(inputStream,
StandardCharsets.UTF_8)
reader = csv.reader(Text,delimiter=',')
for row in reader:
outputStream.write("~".join(row))
outputStream.write("->")
This is a scaled down part of the code, but basically, each time through the
for loop, I get a single letter printed out before the arrow hits, rather than
the entire comma-seperated field.
Currently using NiFi 1.7.0, so I'm not sure if this is something that is fixed,
or if a new version of jython with the newer NiFi's fixes this.
Data is as follows:
2019-06-05 00:00:28.008,f_Some.Data.Value,0,0.0
2019-06-05 00:01:28.553,f_Some.Other.Data.Value,0,0.0
2019-06-05 00:02:29.098,f_Last01.Data.Value,0,0.0
Thoughts?
Thanks,
John
