Hi, Smith It seems that the log file(blob_t-274d3c2d5acd78ced877d898b1877b10b62a64df- 590b54325d599a6782a77413691e0a7b) is deleted for some reason. But AFAIK there are no other guys reporting this exception.(Maybe other guys know what would happen). 1. I think if you could refresh the page and you would see the correct result because this would trigger another file retrieving from TM. 2. And It might be more safe that setting an dedicated blob directory path(other than /tmp) `blob.storage.directory`[1]
[1] https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#blob-storage-directory Best, Guowei On Wed, Nov 3, 2021 at 7:50 AM John Smith <java.dev....@gmail.com> wrote: > Hi running Flink 1.10.0 With 3 zookeepers, 3 job nodes and 3 task nodes. > and I saw this exception on the job node logs... > 2021-11-02 23:20:22,703 ERROR > org.apache.flink.runtime.rest.handler.taskmanager. > TaskManagerLogFileHandler - Unhandled exception. > org.apache.flink.util.FlinkException: Could not retrieve file from > transient blob store. > at org.apache.flink.runtime.rest.handler.taskmanager. > AbstractTaskManagerFileHandler.lambda$respondToRequest$0( > AbstractTaskManagerFileHandler.java:135) > at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture > .java:670) > at java.util.concurrent.CompletableFuture$UniAccept.tryFire( > CompletableFuture.java:646) > at java.util.concurrent.CompletableFuture$Completion.run( > CompletableFuture.java:456) > at org.apache.flink.shaded.netty4.io.netty.util.concurrent. > AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) > at org.apache.flink.shaded.netty4.io.netty.util.concurrent. > SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:416) > at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop > .run(NioEventLoop.java:515) > at org.apache.flink.shaded.netty4.io.netty.util.concurrent. > SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > at org.apache.flink.shaded.netty4.io.netty.util.internal. > ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: Local file /tmp/blobStore-9 > cb73f27-11db-4c42-a3fc-9b77f558e722/no_job/blob_t-274d3 > c2d5acd78ced877d898b1877b10b62a64df-590b54325d599a6782a77413691e0a7b does > not exist and failed to copy from blob store. > at org.apache.flink.runtime.blob.BlobServer.getFileInternal(BlobServer > .java:516) > at org.apache.flink.runtime.blob.BlobServer.getFileInternal(BlobServer > .java:444) > at org.apache.flink.runtime.blob.BlobServer.getFile(BlobServer.java: > 369) > at org.apache.flink.runtime.rest.handler.taskmanager. > AbstractTaskManagerFileHandler.lambda$respondToRequest$0( > AbstractTaskManagerFileHandler.java:133) > ... 9 more >
