[
https://issues.apache.org/jira/browse/FLINK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser closed FLINK-25603.
----------------------------------
Resolution: Duplicate
> make the BlobServer use aws s3
> ------------------------------
>
> Key: FLINK-25603
> URL: https://issues.apache.org/jira/browse/FLINK-25603
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Affects Versions: 1.13.3
> Reporter: lupan
> Priority: Major
>
> Currently, blob.storage.directory does not support using aws s3 as
> storage。details as follows：
> When I use the following configuration:
> {code:java}
> blob.storage.directory: s3://iceberg-bucket/flink/blob {code}
> I get the following error：
> {code:java}
> taskmanager | 2022-01-11 02:41:11,460 ERROR
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Terminating
> TaskManagerRunner with exit code 1.
> taskmanager | org.apache.flink.util.FlinkException: Failed to start the
> TaskManagerRunner.
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManager(TaskManagerRunner.java:374)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.lambda$runTaskManagerProcessSecurely$3(TaskManagerRunner.java:413)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManagerProcessSecurely(TaskManagerRunner.java:413)
> [flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManagerProcessSecurely(TaskManagerRunner.java:396)
> [flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:354)
> [flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | Caused by: java.io.IOException: Could not create storage
> directory for BLOB store in 's3:/iceberg-bucket/flink/blob'.
> taskmanager | at
> org.apache.flink.runtime.blob.BlobUtils.initLocalStorageDirectory(BlobUtils.java:139)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.blob.AbstractBlobCache.<init>(AbstractBlobCache.java:89)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.blob.PermanentBlobCache.<init>(PermanentBlobCache.java:93)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.blob.BlobCacheService.<init>(BlobCacheService.java:55)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.<init>(TaskManagerRunner.java:169)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManager(TaskManagerRunner.java:367)
> ~[flink-dist_2.12-1.13.3.jar:1.13.3]
> taskmanager | ... 5 more{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
