[
https://issues.apache.org/jira/browse/NIFI-14368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Ledoux updated NIFI-14368:
---------------------------------
Affects Version/s: 2.6.0
2.5.0
2.4.0
> Python Processor can not use parent class properties as PropertyDependency
> --------------------------------------------------------------------------
>
> Key: NIFI-14368
> URL: https://issues.apache.org/jira/browse/NIFI-14368
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0
> Environment: Official Docker Images NiFi 2.x running on Ubuntu
> 22.04.4 LTS and Docker v27.0.3
> Reporter: Julien Ledoux
> Priority: Blocker
> Labels: extensions, python
> Attachments: debug_inheritance.zip
>
>
> When using a Python parent class (multiple inheritance) to store common
> properties and methods for a collection processors, it is not possible to use
> parent class properties as a PropertyDependency in its child classes.
> This approach worked correctly in NiFi versions 2.0.0-M4 and 2.0.0. However,
> starting with NiFi 2.1.0, referencing a parent class property as a
> PropertyDependency causes a crash of the processors using this pattern.
> Here is the error message from 2.3.0 version:
> {code:java}
> 2025-03-14 14:06:52,561 ERROR [python-log-344] py4j.java_gateway There was an
> exception while executing the Python Proxy on the Python Side.Traceback (most
> recent call last): File
> "/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line 2506, in
> _call_proxy return_value = getattr(self.pool[obj_id], method)(*params)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/./python/framework/Controller.py", line 77, in
> reloadProcessor self.extensionManager.reload_processor(processorType,
> version, workDirectory) File
> "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 135, in
> reload_processor self.__gather_extension_details(module_file,
> extension_home, dependencies_bundled, work_dir) File
> "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 239, in
> __gather_extension_details classes_and_details =
> self.__get_processor_classes_and_details(module_file, extension_home,
> dependencies_bundled)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line
> 269, in __get_processor_classes_and_details details =
> ProcessorInspection.get_processor_details(class_node, module_file,
> extension_home, dependencies_bundled=dependencies_bundled)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line
> 154, in get_processor_details property_descriptions =
> get_property_descriptions(class_node, module_string_constants)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line
> 288, in get_property_descriptions visitor.visit(class_node) File
> "/usr/lib/python3.11/ast.py", line 418, in visit return visitor(node)
> ^^^^^^^^^^^^^ File "/usr/lib/python3.11/ast.py", line 426, in
> generic_visit self.visit(item) File "/usr/lib/python3.11/ast.py", line
> 418, in visit return visitor(node) ^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line 94, in
> visit_Assign value = self.resolve_dependencies(keyword.value)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line 61, in
> resolve_dependencies variable_name = dependency.args[0].id
> ^^^^^^^^^^^^^^^^^^^^^AttributeError: 'Attribute' object has no attribute
> 'id'2025-03-14 14:06:52,564 WARN [Validate Components Thread-3]
> o.a.n.p.processor.FlowFileSourceProxy
> PythonProcessor[type=AnotherFlowFileSourceProcessor,
> id=9453f6c2-0195-1000-0d8a-2de7d7b09bca] Failed to perform validation for
> Python Processor PythonProcessor[type=AnotherFlowFileSourceProcessor,
> id=9453f6c2-0195-1000-0d8a-2de7d7b09bca]; assuming invalidpy4j.Py4JException:
> An exception was raised by the Python Proxy. Return Message: Traceback (most
> recent call last): File
> "/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line 2506, in
> _call_proxy return_value = getattr(self.pool[obj_id], method)(*params)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/./python/framework/Controller.py", line 77, in
> reloadProcessor self.extensionManager.reload_processor(processorType,
> version, workDirectory) File
> "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 135, in
> reload_processor self.__gather_extension_details(module_file,
> extension_home, dependencies_bundled, work_dir) File
> "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 239, in
> __gather_extension_details classes_and_details =
> self.__get_processor_classes_and_details(module_file, extension_home,
> dependencies_bundled)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line
> 269, in __get_processor_classes_and_details details =
> ProcessorInspection.get_processor_details(class_node, module_file,
> extension_home, dependencies_bundled=dependencies_bundled)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line
> 154, in get_processor_details property_descriptions =
> get_property_descriptions(class_node, module_string_constants)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line
> 288, in get_property_descriptions visitor.visit(class_node) File
> "/usr/lib/python3.11/ast.py", line 418, in visit return visitor(node)
> ^^^^^^^^^^^^^ File "/usr/lib/python3.11/ast.py", line 426, in
> generic_visit self.visit(item) File "/usr/lib/python3.11/ast.py", line
> 418, in visit return visitor(node) ^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line 94, in
> visit_Assign value = self.resolve_dependencies(keyword.value)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File
> "/opt/nifi/nifi-current/python/framework/ProcessorInspection.py", line 61, in
> resolve_dependencies variable_name = dependency.args[0].id
> ^^^^^^^^^^^^^^^^^^^^^AttributeError: 'Attribute' object has no attribute
> 'id'
> at py4j.Protocol.getReturnValue(Protocol.java:476) at
> org.apache.nifi.py4j.client.PythonProxyInvocationHandler.invoke(PythonProxyInvocationHandler.java:87)
> at jdk.proxy6/jdk.proxy6.$Proxy109.reloadProcessor(Unknown Source)
> at
> org.apache.nifi.py4j.StandardPythonProcessorBridge.reload(StandardPythonProcessorBridge.java:186)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.reload(PythonProcessorProxy.java:357)
> at
> org.apache.nifi.python.processor.PythonProcessorProxy.customValidate(PythonProcessorProxy.java:199)
> at
> org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:130)
> at
> org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:912)
> at
> org.apache.nifi.controller.StandardProcessorNode.computeValidationErrors(StandardProcessorNode.java:1054)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:847)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:861)
> at
> org.apache.nifi.components.validation.StandardValidationTrigger.trigger(StandardValidationTrigger.java:52)
> at
> org.apache.nifi.components.validation.TriggerValidationTask.run(TriggerValidationTask.java:58)
> at
> org.apache.nifi.engine.FlowEngine.lambda$wrap$1(FlowEngine.java:105) at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583) {code}
> Because of this issue, upgrades beyond NiFi 2.0.0 are blocked for anyone
> relying on this inheritance pattern.
> I've attached a simple processor structure that uses such pattern :
> [^debug_inheritance.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
