[
https://issues.apache.org/jira/browse/NIFI-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17956133#comment-17956133
]
Fabian Reiber edited comment on NIFI-14621 at 6/4/25 3:07 PM:
--------------------------------------------------------------
Additional information: It only happens with third party dependencies. Now I
tried it with {{import sys}} and I could use the processor by downloading it
from nifi registry. It was stored with the colons in the name of nar file in
the filesystem.
was (Author: JIRAUSER282296):
Additional information: It only happens with third party dependencies. Now I
tried it with {{{{import sys}}}} and I could use the processor by downloading
it from nifi registry. It was stored with the colons in the name of nar file in
the filesystem.
> Failed to import external python dependencies in nar extensions managed by
> nifi registry
> ----------------------------------------------------------------------------------------
>
> Key: NIFI-14621
> URL: https://issues.apache.org/jira/browse/NIFI-14621
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 2.3.0
> Environment: Podman 5.5.0
> Debian 12
> Nifi 2.3.0
> Nifi-Registry 1.26.0
> Reporter: Fabian Reiber
> Priority: Major
>
> I wrote a simple Python processor with an external dependency, created a nar
> file with hatch, uploaded the file to a Nifi Registry and configured the
> [Nifi nar registry
> property|https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#nifi-registry-nar-provider]
> in {{nifi.properties}} to use this registry. After a restart of Nifi, it
> logs an error that the dependency can't be found:
> {code:python}
> 2025-06-02 12:35:18,000 ERROR [python-log-212] py4j.java_gateway There was an
> exception while executing the Python Proxy on the Python Side.
> Traceback (most recent call last):
> File "/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line
> 2506, in _call_proxy
> return_value = getattr(self.pool[obj_id], method)(*params)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/./python/framework/Controller.py", line 71, in
> createProcessor
> processorClass = self.extensionManager.getProcessorClass(processorType,
> version, work_dir)
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line
> 105, in getProcessorClass
> processor_class = self.__load_extension_module(module_file,
> details.local_dependencies)
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line
> 365, in __load_extension_module
> module_spec.loader.exec_module(module)
> File "<frozen importlib._bootstrap_external>", line 940, in exec_module
> File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
> File
> "/opt/nifi/nifi-current/work/nar/extensions/processors::processors-nar::0.0.1::444c6774-eb67-4bf2-abab-25beb1631736.nar-unpacked/processors/hello_world.py",
> line 1, in <module>
> import redis
> ModuleNotFoundError: No module named 'redis'
> {code}
> Regardless of whether it makes sense to use redis in a separate python
> module, it does not work with pandas or any other third party dependency as
> well.
> But if I replace the colons of the filename of the automatically downloaded
> nar file
> {{processors::processors-nar::0.0.1::444c6774-eb67-4bf2-abab-25beb1631736.nar}}
> with dashes
> {{processors-processors-nar-0.0.1-444c6774-eb67-4bf2-abab-25beb1631736.nar}}
> it works.
> It does not explain why the python environment does not find the external
> dependencies. I would suggest, that the loader fails parsing the nar file due
> to the colons. I tested it with Nifi 2.3.0 but would expect, that 2.4.0 is
> affected as well, because I did not find any release note which indicates a
> fix.
> The workaround is not suitable. For any new version or new processor, I have
> to rename it, after all nar files were downloaded.
> I found out that the
> [NiFiRegistryExtensionBundleMetadata|https://github.com/apache/nifi/blob/rel/nifi-2.3.0/nifi-framework-bundle/nifi-framework/nifi-framework-components/src/main/java/org/apache/nifi/registry/extension/NiFiRegistryExtensionBundleMetadata.java#L27]
> class uses the colons as a filename separator.
> I used [this how
> to|https://datavolo.io/2024/05/how-to-package-and-deploy-python-processors-for-apache-nifi/]
> to deploy a nar file with hatch. After the nar file was created with {{hatch
> build --target nar}}, I uploaded it to the registry with {{curl -F
> file=@dist/processors-0.0.1.nar
> http://localhost/nifi-registry-api/buckets/4a7579d3-c41e-4a9b-a3db-9f57340c9e29/bundles/nifi-nar}}.
>
> If you need more information, please let me know :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
