[
https://issues.apache.org/jira/browse/FLINK-32034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colten Pilgreen updated FLINK-32034:
------------------------------------
Description:
I have recent just went through an upgrade from 1.13 to 1.17, along with that I
upgraded the python version on our Flink Session server. Most everything that
is part of our workflow works, except for Python Dependency Management.
After doing some digging, I found the reason is due to the DeprecationWarning
that is printed when trying to get the site packages path. The script is
GET_SITE_PACKAGES_PATH_SCRIPT and it is executed in the getSitePackagesPath
method in the PythonEnvironmentManagerUtils class. The issue is that the
DeprecationWarning is included into the PYTHONPATH environment variable which
is passed to the beam runner. The deprecation warning breaks Python's ability
to find the site packages due to characters that are not allowed in filesystem
paths.
was:
I have recent just went through an upgrade from 1.13 to 1.17, along with that I
upgraded the python version on our Flink Session server. Most everything that
is part of our workflow works, except for Python Dependency Management.
After doing some digging, I found the reason is due to the DeprecationWarning
that is printed when trying to get the site packages path. The script is `
GET_SITE_PACKAGES_PATH_SCRIPT` and it is executed in the `getSitePackagesPath`
method in the `PythonEnvironmentManagerUtils` class. The issue is that the
DeprecationWarning is included into the `PYTHONPATH` environment variable which
is passed to the beam runner. The deprecation warning breaks Python's ability
to find the site packages due to characters that are not allowed in filesystem
paths.
> Python's DistUtils is deprecated as of 3.10
> -------------------------------------------
>
> Key: FLINK-32034
> URL: https://issues.apache.org/jira/browse/FLINK-32034
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 1.17.0
> Environment: Kubernetes
> Java 11
> Python 3.10.9
> Reporter: Colten Pilgreen
> Priority: Minor
>
> I have recent just went through an upgrade from 1.13 to 1.17, along with that
> I upgraded the python version on our Flink Session server. Most everything
> that is part of our workflow works, except for Python Dependency Management.
> After doing some digging, I found the reason is due to the DeprecationWarning
> that is printed when trying to get the site packages path. The script is
> GET_SITE_PACKAGES_PATH_SCRIPT and it is executed in the getSitePackagesPath
> method in the PythonEnvironmentManagerUtils class. The issue is that the
> DeprecationWarning is included into the PYTHONPATH environment variable which
> is passed to the beam runner. The deprecation warning breaks Python's ability
> to find the site packages due to characters that are not allowed in
> filesystem paths.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
