Christos Hadjinikolis created FLINK-33382:
---------------------------------------------
Summary: Flink Python Environment Manager Fails with Pip
--install-option in Recent Pip Versions
Key: FLINK-33382
URL: https://issues.apache.org/jira/browse/FLINK-33382
Project: Flink
Issue Type: Bug
Affects Versions: 1.17.1
Environment: * Flink version: 1.17.1
* Python version: 3.10
* Pip version: >= 21
* Operating System: [MacOS]
Reporter: Christos Hadjinikolis
I encountered an issue when running Flink jobs that use Python dependencies.
The underlying problem seems to stem from Flink's use of the
{{--install-option}} argument when calling {{pip install}} to manage Python
dependencies. This argument has been deprecated in newer versions of pip,
resulting in job failures.
This issue forces users to downgrade their {{pip}} version as a temporary
workaround, which is not ideal due to potential security vulnerabilities and
missing features in older {{pip}} versions.
{*}Steps to Reproduce{*}:
# Setup a Flink job with Python dependencies.
# Use a Python environment with a {{pip}} version >= 21.
# Run the Flink job.
# Observe the error: {{{}no such option: --install-option{}}}.
{*}Error Logs{*}:
pythonCopy code
{{no such option: --install-option...}}
(You can add more logs here if needed)
{*}Expected Behavior{*}: Flink should handle Python dependency management in a
way that's compatible with newer versions of {{{}pip{}}}.
{*}Possible Solutions{*}:
# Update Flink's Python dependency management code to remove or replace the
{{--install-option}} argument.
# Provide Flink configuration options to customize the {{pip install}} command
or to skip certain options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
