Pyflink提交

Fri, 25 Nov 2022 00:02:09 -0800 

在使用pyflink提交任务时，部署模式onyarn 
1 在不使用Map等算子下如下参数 能够提交成功 并且运行
.flink run  -ynm pytest   -m yarn-cluster  -pyclientexec ***/python3   -pyexec 
***/python3  -pyarch *** /python3.6.8.zip   -py  demo.py 


2 在使用到map算子时 提交没有问题，但是运行报错，报错日志如下：
.flink run  -ynm pytest   -m yarn-cluster  -pyclientexec ***/python3   -pyexec 
***/python3  -pyarch *** /python3.6.8.zip   -py  demo.py 
报错内容：
Caused by: java.io.IOException: Failed to execute the command: 
./python3.6.8.zip/bin/python3 -c import pyflink;import 
os;print(os.path.join(os.path.abspath(os.path.dirname(pyflink.__file__)), 
'bin'))
output: Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pyflink'
        at 
org.apache.flink.python.util.PythonEnvironmentManagerUtils.execute(PythonEnvironmentManagerUtils.java:211)
        at 
org.apache.flink.python.util.PythonEnvironmentManagerUtils.getPythonUdfRunnerScript(PythonEnvironmentManagerUtils.java:154)
        at 
org.apache.flink.python.env.beam.ProcessPythonEnvironmentManager.createEnvironment(ProcessPythonEnvironmentManager.java:156)
        at 
org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.createPythonExecutionEnvironment(BeamPythonFunctionRunner.java:398)
        at 
org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.lambda$open$0(BeamPythonFunctionRunner.java:246)
        at 
org.apache.flink.runtime.memory.MemoryManager.lambda$getSharedMemoryResourceForManagedMemory$5(MemoryManager.java:539)


请问环境变量如何设置才能正常运行

回复