huangkaiyan10 opened a new issue, #1895: URL: https://github.com/apache/incubator-streampark/issues/1895
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/streampark/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported -Dstate.backend=rocksdb -Dcontainerized.master.env.ENABLE_BUILT_IN_PLUGINS=flink-oss-fs-hadoop-1.14.5.jar -Dcontainerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS=flink-oss-fs-hadoop-1.14.5.jar -Dfs.oss.endpoint=http://172.18.17.235:30900 -Dfs.oss.accessKeyId=admin -Dfs.oss.accessKeySecret=admin123456 -Dstate.checkpoints.dir=oss://realtime-xxx/streamx/dev/checkpoints/ -Dstate.savepoints.dir=oss://realtime-xxx/streamx/dev/savepoints/ ### StreamPark Version 1.2.3-2.12 ### Java Version jdk 8 ### Flink Version 1.14.5-2.12 ### Scala Version of Flink 2.12 ### Error Exception ```log org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported Caused by: java.util.concurrent.CompletionException: org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. [ErrorCode]: InvalidRequest [RequestId]: [HostId]: 6d957ce9-671d-4759-aa0d-0cfe6aa34c32 [ResponseError]: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><Resource>/realtime-xxx/</Resource><RequestId></RequestId><HostId>6d957ce9-671d-4759-aa0d-0cfe6aa34c32</HostId></Error> at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_345] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345] Caused by: org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. [ErrorCode]: InvalidRequest [RequestId]: [HostId]: 6d957ce9-671d-4759-aa0d-0cfe6aa34c32 [ResponseError]: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><Resource>/realtime-xxx/</Resource><RequestId></RequestId><HostId>6d957ce9-671d-4759-aa0d-0cfe6aa34c32</HostId></Error> at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.common.utils.ExceptionFactory.createOSSException(ExceptionFactory.java:100) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.internal.OSSErrorResponseHandler.handle(OSSErrorResponseHandler.java:70) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.common.comm.ServiceClient.handleResponse(ServiceClient.java:257) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.common.comm.ServiceClient.sendRequestImpl(ServiceClient.java:140) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.common.comm.ServiceClient.sendRequest(ServiceClient.java:70) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.internal.OSSOperation.send(OSSOperation.java:83) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:145) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:102) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.internal.OSSBucketOperation.listObjects(OSSBucketOperation.java:411) ~[?:?] at org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSClient.listObjects(OSSClient.java:443) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.listObjects(AliyunOSSFileSystemStore.java:437) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.getFileStatus(AliyunOSSFileSystem.java:274) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.mkdirs(AliyunOSSFileSystem.java:549) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2304) ~[?:?] at org.apache.flink.fs.osshadoop.common.HadoopFileSystem.mkdirs(HadoopFileSystem.java:183) ~[?:?] at org.apache.flink.core.fs.PluginFileSystemFactory$ClassLoaderFixingFileSystem.mkdirs(PluginFileSystemFactory.java:162) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.initializeBaseLocations(FsCheckpointStorageAccess.java:113) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:322) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:240) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.enableCheckpointing(DefaultExecutionGraph.java:452) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:315) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:107) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:335) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:191) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:140) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:134) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:110) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:346) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:323) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:106) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:94) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_345] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345] 2022-10-24 09:20:19,454 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Fatal error occurred in the cluster entrypoint. java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_345] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:287) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:224) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_345] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_345] at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:171) ~[flink-rpc-akka_86669a35-0502-497b-90fe-e6b2729c9f38.jar:1.14.5] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_86669a35-0502-497b-90fe-e6b2729c9f38.jar:1.14.5] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41) ~[flink-rpc-akka_86669a35-0502-497b-90fe-e6b2729c9f38.jar:1.14.5] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) [flink-rpc-akka_86669a35-0502-497b-90fe-e6b2729c9f38.jar:1.14.5] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) [flink-rpc-akka_86669a35-0502-497b-90fe-e6b2729c9f38.jar:1.14.5] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_345] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_345] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_345] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) [?:1.8.0_345] Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 13 more Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Failed to execute sql at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:261) ~[flink-dist_2.12-1.14.5.jar:1.14.5] ... 12 more Caused by: org.apache.flink.table.api.TableException: Failed to execute sql at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:791) ~[flink-table_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:754) ~[flink-table_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.api.internal.StatementSetImpl.execute(StatementSetImpl.java:124) ~[flink-table_2.12-1.14.5.jar:1.14.5] at com.streamxhub.streamx.flink.core.FlinkSqlExecutor$.executeSql(FlinkSqlExecutor.scala:182) ~[?:?] at com.streamxhub.streamx.flink.core.FlinkStreamTableTrait.sql(FlinkStreamTableTrait.scala:94) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient$.handle(SqlClient.scala:29) ~[?:?] at com.streamxhub.streamx.flink.core.scala.FlinkStreamTable.main(FlinkStreamTable.scala:49) ~[?:?] at com.streamxhub.streamx.flink.core.scala.FlinkStreamTable.main$(FlinkStreamTable.scala:46) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient$.main(SqlClient.scala:27) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient.main(SqlClient.scala) ~[?:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_345] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_345] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_345] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_345] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:261) ~[flink-dist_2.12-1.14.5.jar:1.14.5] ... 12 more Caused by: org.apache.flink.util.FlinkException: Failed to execute job 'kafka-2-doris-test-2'. at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2055) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:137) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.planner.delegation.DefaultExecutor.executeAsync(DefaultExecutor.java:95) ~[flink-table_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:773) ~[flink-table_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:754) ~[flink-table_2.12-1.14.5.jar:1.14.5] at org.apache.flink.table.api.internal.StatementSetImpl.execute(StatementSetImpl.java:124) ~[flink-table_2.12-1.14.5.jar:1.14.5] at com.streamxhub.streamx.flink.core.FlinkSqlExecutor$.executeSql(FlinkSqlExecutor.scala:182) ~[?:?] at com.streamxhub.streamx.flink.core.FlinkStreamTableTrait.sql(FlinkStreamTableTrait.scala:94) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient$.handle(SqlClient.scala:29) ~[?:?] at com.streamxhub.streamx.flink.core.scala.FlinkStreamTable.main(FlinkStreamTable.scala:49) ~[?:?] at com.streamxhub.streamx.flink.core.scala.FlinkStreamTable.main$(FlinkStreamTable.scala:46) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient$.main(SqlClient.scala:27) ~[?:?] at com.streamxhub.streamx.flink.cli.SqlClient.main(SqlClient.scala) ~[?:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_345] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_345] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_345] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_345] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:261) ~[flink-dist_2.12-1.14.5.jar:1.14.5] ... 12 more Caused by: java.lang.RuntimeException: org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster. at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:316) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456) ~[?:1.8.0_345] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345] Caused by: org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster. at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_345] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345] Caused by: java.util.concurrent.CompletionException: org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. [ErrorCode]: InvalidRequest [RequestId]: [HostId]: 6d957ce9-671d-4759-aa0d-0cfe6aa34c32 [ResponseError]: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message><Resource>/realtime-xxx/</Resource><RequestId></RequestId><HostId>6d957ce9-671d-4759-aa0d-0cfe6aa34c32</HostId></Error> at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_345] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_345] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345] Caused by: org.apache.flink.fs.osshadoop.shaded.com.aliyun.oss.OSSException: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. [ErrorCode]: InvalidRequest ``` ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
