[
https://issues.apache.org/jira/browse/MINIFICPP-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-2440:
--------------------------------------
Description:
When installing dependencies for Python processors, we can get into trouble if,
for example, processor A depends on pyfoo>=1.0.0 but processor B depends on
pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A
first, it will install pyfoo==2.0.1, which will cause an error when we try to
load processor B.
With the current setup, we need to manually resolve the conflict, and (in the
example above) add "pyfoo == 1.9.5" to the list of
ProcessorDetails.dependencies in processor A if we know we will use it together
with processor B. Then, if another processor C comes along with the
requirement pyfoo<1.8.0, we will need to change it again, and also update
ProcessorDetails.dependencies in processor B. This is not maintainable.
Pip is quite smart in resolving such conflicts, but it can only do so if we
give it all the requirements at once. So we should collect the dependency
requirements from all requirements.txt files and the
ProcessorDetails.dependencies fields in all processors, and call pip install
with this full list once, instead of calling it multiple times.
was:
When installing dependencies for Python processors, we can get into trouble if,
for example, processor A depends on pyfoo>=1.0.0 but processor B depends on
pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load processor A
first, it will install pyfoo=2.0.1, which will cause an error when we try to
load processor B.
With the current setup, we need to manually resolve the conflict, and (in the
example above) add "pyfoo == 1.9.5" to the list of
ProcessorDetails.dependencies in processor A if we know we will use it together
with processor B. Then, if another processor C comes along with the
requirement pyfoo<1.8.0, we will need to change it again, and also update
ProcessorDetails.dependencies in processor B. This is not maintainable.
Pip is quite smart in resolving such conflicts, but it can only do so if we
give it all the requirements at once. So we should collect the dependency
requirements from all requirements.txt files and all
ProcessorDetails.dependencies fields in all processors, and call pip install
with this full list once, instead of calling it multiple times.
> Install all Python dependencies at once
> ---------------------------------------
>
> Key: MINIFICPP-2440
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2440
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
> Reporter: Ferenc Gerlits
> Priority: Major
>
> When installing dependencies for Python processors, we can get into trouble
> if, for example, processor A depends on pyfoo>=1.0.0 but processor B depends
> on pyfoo<2.0.0. If the latest version of pyfoo is 2.0.1, and we load
> processor A first, it will install pyfoo==2.0.1, which will cause an error
> when we try to load processor B.
> With the current setup, we need to manually resolve the conflict, and (in the
> example above) add "pyfoo == 1.9.5" to the list of
> ProcessorDetails.dependencies in processor A if we know we will use it
> together with processor B. Then, if another processor C comes along with the
> requirement pyfoo<1.8.0, we will need to change it again, and also update
> ProcessorDetails.dependencies in processor B. This is not maintainable.
> Pip is quite smart in resolving such conflicts, but it can only do so if we
> give it all the requirements at once. So we should collect the dependency
> requirements from all requirements.txt files and the
> ProcessorDetails.dependencies fields in all processors, and call pip install
> with this full list once, instead of calling it multiple times.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
