flink使用在docker环境中部署出现的两个问题

Sun, 27 Sep 2020 04:57:03 -0700 

根据官网[1]使用docker部署flink,session cluster模式
环境win10+docker+flink1.11.2
cmd命令
docker run ^
-d^
    --rm ^
    --name=jobmanager ^
    --hostname=jobmanager ^
    --network flink-network ^
--env FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager" ^
    -p 28081:8081 ^
    flink:1.11.2-scala_2.11 jobmanager
docker run ^
-d^
    --rm ^
    --name=taskmanager ^
    --hostname=taskmanager ^
    --network flink-network ^
--env FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager" ^
    flink:1.11.2-scala_2.11 taskmanager


问题一:
在webui查看任务输出stdout提示找不到输出文件
java.util.concurrent.CompletionException: org.apache.flink.util.FlinkException: 
The file STDOUT does not exist on the TaskExecutor.
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$requestFileUploadByFilePath$25(TaskExecutor.java:1742)
 ~[flink-dist_2.11-1.11.2.jar:1.11.2]
at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
 ~[?:1.8.0_265]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
~[?:1.8.0_265]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
~[?:1.8.0_265]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_265]
Caused by: org.apache.flink.util.FlinkException: The file STDOUT does not exist 
on the TaskExecutor.
... 5 more
2020-09-27 09:04:33,370 ERROR 
org.apache.flink.runtime.rest.handler.taskmanager.TaskManagerStdoutFileHandler 
[] - Unhandled exception.
org.apache.flink.util.FlinkException: The file STDOUT does not exist on the 
TaskExecutor.
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$requestFileUploadByFilePath$25(TaskExecutor.java:1742)
 ~[flink-dist_2.11-1.11.2.jar:1.11.2]
at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
 ~[?:1.8.0_265]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
~[?:1.8.0_265]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
~[?:1.8.0_265]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_265]




问题二:
mount的src的配置文件要多份吗
因为我把env替换成mount,如下
docker run ^
-d^
    --rm ^
    --name=jobmanager ^
    --network flink-network ^
--mount type=bind,src=D:/cxy/soft/flink-1.11.2/conf,target=/opt/flink/conf ^
    -p 28081:8081 ^
    flink:1.11.2-scala_2.11 jobmanager


docker run ^
-d^
    --rm ^
    --name=taskmanager ^
    --network flink-network ^
--mount type=bind,src=D:/cxy/soft/flink-1.11.2/conf,target=/opt/flink/conf ^
    flink:1.11.2-scala_2.11 taskmanager


结果发现webui上的可用Task Managers为0
每次执行命令的时候都会把mount配置的src下flink-conf.yaml中的jobmanager.rpc.address替换成了新的容器ip
我猜应该是这个原因导致启动taskmanager的时候jobmanager.rpc.address替换成了taskmanager的ip.所以没有Task可用
想问下大佬们,是我哪一步出现问题了吗


[1]:https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/docker.html

回复