David Handermann created NIFI-13042:
---------------------------------------
Summary: Support Python 3.12 for Python Processors
Key: NIFI-13042
URL: https://issues.apache.org/jira/browse/NIFI-13042
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: David Handermann
Assignee: David Handermann
Fix For: 2.0.0-M3
NiFi 2.0.0-M1 and M2 support Python Processors with Python 3.11, but attempting
to start NiFi with Python 3.12 results in the following error:
{noformat}
ERROR [python-log-29812] py4j.java_gateway Error while waiting for a connection.
Traceback (most recent call last):
File "nifi-2.0.0-SNAPSHOT/python/framework/py4j/java_gateway.py", line 2304,
in run
connection.start()
File "python3.12/threading.py", line 992, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
{noformat}
This issue follows the pattern of [CPython Issue
115533|https://github.com/python/cpython/issues/115533], which describes
changes in Python 3.12 to prevent spawning new threads when the main Python
process has finalized.
In the context of the NiFi Python Framework, this applies to the Controller.py
main method, which is responsible for starting the Py4J gateway to communicate
between Java and Python processes.
Following the workaround proposed in the CPython issue discussion, the main
function in Controller.py can be adjusted to join to non-daemon threads. This
maintains the association between the Python main function and threads spawn
for Py4J socket communication.
In addition to this thread issue, the Python FileFinder
[find_module|https://docs.python.org/3.10/library/importlib.html#importlib.abc.PathEntryFinder.find_module]
has been deprecated since Python 3.4 and was removed in Python 3.12. Usage of
{{find_module}} in ExtensionsManager.py should be replaced with {{find_spec}}
which provide equivalent functionality.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
