A jar upload shouldn't take minutes. There are two possibilities that
likely co-occured:
- your jar is much bigger than needed. Did you make sure that you don't put
Flink into the fatjar? That's counterproductive on many levels. Please
check the jar size.
- your connection to the JM is bad. Where is your Flink cluster running? If
connection is superslow, it's often better to build the jar on the
respective server and submit on command line or through REST.
On Wed, Nov 4, 2020 at 7:15 AM forideal <fszw...@163.com> wrote:
> Hello my friend：
> My line of code runs very slowly. What are the possibilities?
> code：
>
> CompletableFuture<JobGraph> jarUploadFuture =
> jobGraphFuture.thenCombine(blobServerPortFuture, (jobGraph, blobServerPort)
> -> {
> final InetSocketAddress address = new
> InetSocketAddress(gateway.getHostname(), blobServerPort);
> try {
> ClientUtils.extractAndUploadJobGraphFiles(jobGraph, () -> new
> BlobClient(address, configuration));
> } catch (FlinkException e) {
> throw new CompletionException(e);
> }
> return jobGraph;
> });
>
> code link：
> https://github.com/apache/flink/blob/release-1.10.0/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/JarRunHandler.java#L93
>
> Through jstack, we can also find that the thread
> Flink-DispatcherRestEndpoint-thread-1 has been running for 5 minutes.
> "Flink-DispatcherRestEndpoint-thread-1" #82 daemon prio=5 os_prio=0
> tid=0x00007f590c03c800 nid=0x179 runnable [0x00007f5a34165000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> at
> org.apache.flink.runtime.blob.BlobOutputStream.write(BlobOutputStream.java:88)
> at org.apache.flink.util.IOUtils.copyBytes(IOUtils.java:65)
> at
> org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:368)
> at org.apache.flink.runtime.blob.BlobClient.uploadFile(BlobClient.java:428)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadUserJars(ClientUtils.java:102)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadAndSetUserJars(ClientUtils.java:95)
> at
> org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:77)
> at
> org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:57)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$1(RedJarRunHandler.java:92)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler$$Lambda$770/941040494.apply(Unknown
> Source)
> at
> java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1105)
> at
> java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1070)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Thank you very much for your reply, forideal
>
>
>
>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng
