Please do not remove the Python scripting facilities.
I believe most people experience it as very slow because (ok, Python is slow)
they only get a flow file at a time.
I think Matt pointed out in this ML once that you can use
flowfiles = session.get(1000)
for flowfile in filter(None, flowfiles):
# do things
In that case Jython will be kept alive much longer. Or am I missing something?
Giovanni
> -----Original Message-----
> From: Joe Witt [mailto:[email protected]]
> Sent: Saturday, May 6, 2017 1:12 AM
> To: [email protected]
> Subject: Re: Is it possible to reference python requests module in
> ExecuteScript?
>
> It is worth discussing whether there is sufficient value to warrant keeping
> jython/python support in the processors or whether we should pull it. It is
> certainly something we can document as being highly limited but we don't
> really know how limited. Frankly given the performance I've seen with it I'd
> be
> ok removing it entirely. One is better off calling the script via a system
> call.
> Groovy is one that I've seen perform very well and be fully featured.
>
> On Fri, May 5, 2017 at 6:38 PM, Russell Bateman <[email protected]>
> wrote:
> > We really want to use ExecuteScript because our end users are Pythonistas.
> > They tend to punctuate their flows with the equivalent of PutFile and
> > GetFile with Python scripts doing stuff on flowfiles that pass out of
> > NiFi before returning into NiFi.
> >
> > However, we find it nearly impossible to replace even the tamest of
> > undertakings. If there were a good set of NiFi/Python shims that, from
> > PyCharm, etc., gave us the ability to prototype, test and debug before
> > copying and pasting into ExecuteScript, that would be wonderful. It
> > hasn't worked out that way. Most of our experience is copying, pasting
> > into the processor property, only to find something wrong, sometimes
> > syntax, sometimes something runtime.
> >
> > On their behalf, I played with this processor a few hours a while back.
> > Another colleague too. Googling this underused tool hasn't been
> > helpful, so the overall experience is negative so far. I can get most
> > of the examples out there to work, but as soon as I try to do "real"
> > work from my point of view, my plans sort of cave in.
> >
> > Likely the Groovy and/or Ruby options are better? But, we're not
> > Groovy or Ruby guys here. I understand the problems with this tool and
> > so I understand what the obstacles are to it growing stronger. The
> > problems won't yield to a few hours one Saturday afternoon. Better
> > problem-logging underneath and
> > better- and more lenient Python support on top. The second one is
> > tough, though.
> >
> > My approach is to minimize those black holes these guys put into their
> > flows by creating custom processors for what I can't solve using
> > standard processors.
> >
> > Trying not to be too negative here...
> >
> >
> > On 05/05/2017 04:09 PM, Andre wrote:
> >
> > Mike,
> >
> > I believe it is possible to use requests under jython, however the
> > process isn't very intuitive.
> >
> > I know one folk that if I recall correctly has used it. Happy to try
> > to find out how it is done.
> >
> > Cheers
> >
> > On Sat, May 6, 2017 at 4:57 AM, Mike Harding <[email protected]>
> wrote:
> >>
> >> Hi All, I'm now looking at using ExecuteScript and python engine to
> >> execute HTTP requests using the requests module. I've tried
> >> referencing requests the module but when I try to import requests I
> >> get a module reference error.
> >> I downloaded the module from here >
> >> https://pypi.python.org/pypi/requests
> >> Not sure why it isnt picking it up. Ive tried referencing the
> >> directory and the .py directly with no success.
> >> Any ideas where im going wrong?
> >> Cheers,
> >> Mike
