Hi Sidhant, Have you seen this discussion [1]? If you want to use S3, then you need to make sure that you start your Flink processes with the appropriate FileSystemProvider for S3 [2]. So the problem you are seeing is most likely caused by the JVM not knowing a S3 file system implementation.
Be aware that the filesystem used by the FileUploadHandler is java.nio.file.FileSystem and not Flink's org.apache.flink.core.fs.FileSystem for which we provide different FileSystem implementations. My best recommendation would be to not use the web submission for Flink jobs when running Flink behind a load balancer. Instead use the CLI which will retrieve the leader address from ZooKeeper. [1] https://lists.apache.org/thread.html/1dbaf68bc86ad3a1acc87e4bbc2dd5fa8c7560b3f59eb256f81bc855%40%3Cuser.flink.apache.org%3E [2] https://stackoverflow.com/questions/41113119/java-nio-file-implementation-for-aws Cheers, Till On Wed, Dec 2, 2020 at 11:31 AM sidhant gupta <sidhan...@gmail.com> wrote: > Hi All, > > I have 2 job managers in flink HA mode cluster setup. I have a load > balancer forwarding request to both (leader and stand by) the job managers > in default round-robin fashion. While uploading the job jar the Web UI is > fluctuating between the leader and standby page. Its difficult to upload > the job jar and also getting errors after submitting the job as below: > > Caused by: org.apache.flink.runtime.rest.handler.RestHandlerException: Jar > file /tmp/flink-web-4be8d72f-6a7d-4ec9-b71e-516e58 > c629d5/flink-web-upload/ce291201-31e4-4e55 > -a68b-eae1a770518d_flinkfhirmapperjob-1.2.jar does not exist > > > I tried using *web.upload.dir* property to use a folder location in S3 > but getting below error: > > 2020-12-02 08:26:06,242 WARN > org.apache.flink.runtime.rest.FileUploadHandler [] - File > upload failed. > > java.lang.IllegalArgumentException: UploadDirectory is not absolute. > > at > org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:139) > ~[flink-dist_2.11-1.11.2.jar:1.11.2] > > at > org.apache.flink.runtime.rest.handler.FileUploads.<init>(FileUploads.java:58) > ~[flink-dist_2.11-1.11.2.jar:1.11.2] > > at > org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:159) > [flink-dist_2.11-1.11.2.jar:1.11.2] > > at > org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:68) > [flink-dist_2.11-1.11.2.jar:1.11.2] > > at > org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > [flink-dist_2.11-1.11.2.jar:1.11.2] > > > Looks like web.upload.dir only supports local path. > > > Any suggestions on how to upload and submit job jar in flink HA cluster > setup via web UI and also it from the CLI ? > > > Thanks and regards > > Sidhant Gupta > > > >
