[
https://issues.apache.org/jira/browse/FLINK-23548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17390543#comment-17390543
]
zx10r commented on FLINK-23548:
-------------------------------
[~chesnay] your comment helped tremendously. Mounting a PVC into job-manager
deployment and using the --output for arguments pointing to that PVC solves
this issue. The log is found on the volume and the REST error disappears. It
might be worth it updating the Kubernetes guide
[https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/standalone/kubernetes/]
and adding notes on mounting volumes and passing the correct arguments if
submitting jobs from the UI. I can definitely do it if everyone thinks it's of
value.
> Flink Session Cluster (Standalone) in Kubernetes: Exception occurred in REST
> handler: Could not execute application.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-23548
> URL: https://issues.apache.org/jira/browse/FLINK-23548
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.13.0, 1.13.1
> Reporter: zx10r
> Priority: Major
> Labels: Kubernetes
>
> Following the guide located here:
> [https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/standalone/kubernetes/]
> to deploy Session Cluster in standalone mode on Kubernetes seems to produce
> weird REST errors in the UI. When running any of the examples through the UI
> and submitting jobs the Job Manager will show: *Exception occurred in REST
> handler: Could not execute application* while the UI produces the bellow
> stack trace.
> I will mention the the job completes successfully and submitting the job
> through command line also works correctly but we are attempting to figure out
> why the UI/REST errors are appearing and if there is a possible fix in the
> kubernetes yaml definitions that are provided in the guide above. This is
> painful for us due to many of our users submitting jobs through the UI.
>
> *Job Manager Pod Error:*
> {code:java}
> 2021-07-29 21:03:01,982 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
> Reduce (SUM(1), at main(WordCount.java:87) (1/1) (attempt #0) with attempt id
> e626567e699fcfafd5902
> 2021-07-29 21:03:01,983 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - DataSink
> (collect()) (1/1) (2461dbbb123ea836d88d3bba293c5b2c) switched from SCHEDULED
> to DEPLOYING.
> 2021-07-29 21:03:01,985 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
> DataSink (collect()) (1/1) (attempt #0) with attempt id
> 2461dbbb123ea836d88d3bba293c5b2c to 10.30.4
> 2021-07-29 21:03:02,002 WARN
> org.apache.flink.client.deployment.application.DetachedApplicationRunner [] -
> Could not execute application:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: Job was submitted in detached mode. Results of job
> execution, such as accumulators, runtime, etc. are not availa
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
> at
> org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:102)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> [?:1.8.0_292]
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_292]
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_292]
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0_292]
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [?:1.8.0_292]
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_292]
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_292]
>
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292]
>
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Job was
> submitted in detached mode. Results of job execution, such as accumulators,
> runtime, etc. are not available. Please make sure your pr
> at
> org.apache.flink.core.execution.DetachedJobExecutionResult.getAccumulatorResult(DetachedJobExecutionResult.java:56)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:419)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at org.apache.flink.api.java.DataSet.print(DataSet.java:1748)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> at
> org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:96)
> ~[?:?]
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_292]
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_292]
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_292]
>
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_292]
>
>
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
> ~[flink-dist_2.11-1.13.0.jar:1.13.0]
>
> ... 13 more
>
>
> 2021-07-29 21:03:02,004 ERROR
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler [] - Exception
> occurred in REST handler: Could not execute application.
> {code}
>
> *UI Stack Error:*
> {code:java}
> org.apache.flink.runtime.rest.handler.RestHandlerException: Could not execute
> application. at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(JarRunHandler.java:108)
> at
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
> at
> java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) Caused by:
> java.util.concurrent.CompletionException:
> org.apache.flink.util.FlinkRuntimeException: Could not execute application.
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
> ... 7 more Caused by: org.apache.flink.util.FlinkRuntimeException: Could not
> execute application. at
> org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:88)
> at
> org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:102)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> ... 7 more Caused by:
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: Job was submitted in detached mode. Results of job
> execution, such as accumulators, runtime, etc. are not available. Please make
> sure your program doesn't call an eager execution function [collect, print,
> printToErr, count]. at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
> at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
> at
> org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
> ... 10 more Caused by: org.apache.flink.api.common.InvalidProgramException:
> Job was submitted in detached mode. Results of job execution, such as
> accumulators, runtime, etc. are not available. Please make sure your program
> doesn't call an eager execution function [collect, print, printToErr, count].
> at
> org.apache.flink.core.execution.DetachedJobExecutionResult.getAccumulatorResult(DetachedJobExecutionResult.java:56)
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:419) at
> org.apache.flink.api.java.DataSet.print(DataSet.java:1748) at
> org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:96) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
> ... 13 more
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
