[
https://issues.apache.org/jira/browse/FLINK-32034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732518#comment-17732518
]
Dian Fu commented on FLINK-32034:
---------------------------------
[~coltenp] Thanks for reporting this issue and the finding. Would you like to
create a PR for this issue?
> Python's DistUtils is deprecated as of 3.10
> -------------------------------------------
>
> Key: FLINK-32034
> URL: https://issues.apache.org/jira/browse/FLINK-32034
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 1.17.0
> Environment: Kubernetes
> Java 11
> Python 3.10.9
> Reporter: Colten Pilgreen
> Priority: Minor
> Attachments: get_site_packages_path_script.py,
> get_site_packages_path_script_shortened.py
>
>
> I have recent just went through an upgrade from 1.13 to 1.17, along with that
> I upgraded the python version on our Flink Session server. Most everything
> that is part of our workflow works, except for Python Dependency Management.
> After doing some digging, I found the reason is due to the DeprecationWarning
> that is printed when trying to get the site packages path. The script is
> GET_SITE_PACKAGES_PATH_SCRIPT and it is executed in the getSitePackagesPath
> method in the PythonEnvironmentManagerUtils class. The issue is that the
> DeprecationWarning is included into the PYTHONPATH environment variable which
> is passed to the beam runner. The deprecation warning breaks Python's ability
> to find the site packages due to characters that are not allowed in
> filesystem paths.
>
> Example of the PYTHONPATH environment variable:
> PYTHONPATH == <string>:1: DeprecationWarning: The distutils package is
> deprecated and slated for removal in Python 3.12. Use setuptools or check PEP
> 632 for potential
> alternatives:/tmp/python-dist-c63e1464-925c-4289-bb71-c6f50e83186f/python-requirements/lib/python3.10/site-packages
> HADOOP_CONF_DIR == /opt/flink/conf
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
