[
https://issues.apache.org/jira/browse/FLINK-22871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357219#comment-17357219
]
Dian Fu commented on FLINK-22871:
---------------------------------
[~wukong] Thanks for sharing this information~. This is definitely a feature we
should support.
For the solution you proposed, I have some questions, e.g. how to configure
python.client.executable for users?
I have thought about this problem and had a solution in my mind:
- Users still configure venv via "python.archives". This is also how to
configure to execute Python UDFs using venv. In this way, users could configure
venv via "python.archives" and use it both at client side (when compiling job)
and server side (when executing Python UDF).
- Extract the zip files specified in "python.archives" into a working directory
in method PythonEnvUtils.preparePythonEnvironment.
- Users could configure to use the venv at client side via the existing
configuration "python.client.executable", just like how to configure
"python.execute"
- Set the working directory for the Python Process in
PythonEnvUtils.startPythonProcess
What's your thought?
> pyflink deploy support yarn application mode
> --------------------------------------------
>
> Key: FLINK-22871
> URL: https://issues.apache.org/jira/browse/FLINK-22871
> Project: Flink
> Issue Type: New Feature
> Components: API / Python, Deployment / YARN
> Affects Versions: 1.12.1
> Reporter: konwu
> Priority: Minor
>
> for now pyflink is not support hadoop yarn application mode, cause of yarn
> nodemanager may not have suitable python version
> after test for use venv(python virtual environment) that uploaded by
> 'python.files' properties, then change 'env.pythonExec' path, it also works
> so,is there any possiable to support this in a suitable way
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
