In general, I would recommend moving the Python code to Flask-based
rest services running on something like Kubernetes. That way you can
access them with InvokeHttp which has a much lower overhead than
ExecuteStreamCommand (no CPython startup per run). Creating
containerized Python services is not
Hi,
As James mentioned in your example the "system" memory would be used by the
process/script in this case.
I wanted to add that if you use the "ExecuteScript" processor that the NiFi
memory would be used. This is because NiFi uses Jython for processing
Python files. So where a native Cython proc
Hi,
The memory you allocate to NiFi will typically be the maximum heap size for the
Java process. If you spawn a separate Python process then it will not be bound
by the JVM heap size, so in your example it would come from the 8GB allocated
for the “system”.
I would recommend if you see high re
Hello!
Can you help me with one question - can't find any information :(
We have a nifi on my server (16 CPU, 32 RAM), in config our system adm.
set ram usage for nifi = 24 Gb, rest 8 Gb - for the system.
In Nifi we execute many Python scripts by using ExecuteStreamCommand (just
set path of scrip
