zhaijack commented on a change in pull request #2065: GCS offload support(2):
replace `s3client` api with `jclouds` related api
URL: https://github.com/apache/incubator-pulsar/pull/2065#discussion_r202986600
##########
File path:
pulsar-broker/src/main/java/org/apache/pulsar/broker/offload/impl/ManagedLedgerOffloader.java
##########
@@ -96,23 +122,66 @@ public static S3ManagedLedgerOffloader
create(ServiceConfiguration conf,
throw new
PulsarServerException("s3ManagedLedgerOffloadMaxBlockSizeInBytes cannot be less
than 5MB");
}
- AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard();
+ return new ManagedLedgerOffloader(driver, bucket, scheduler,
maxBlockSize, readBufferSize, endpoint, region);
+ }
+
+ // build context for jclouds BlobStoreContext
+ ManagedLedgerOffloader(String driver, String container, OrderedScheduler
scheduler,
+ int maxBlockSize, int readBufferSize, String
endpoint, String region) {
+ this.scheduler = scheduler;
+ this.readBufferSize = readBufferSize;
+
+ this.bucket = container;
+ this.maxBlockSize = maxBlockSize;
+
+ Properties overrides = new Properties();
+ // This property controls the number of parts being uploaded in
parallel.
+ overrides.setProperty("jclouds.mpu.parallel.degree", "1");
+ overrides.setProperty("jclouds.mpu.parts.size",
Integer.toString(maxBlockSize));
+ overrides.setProperty(Constants.PROPERTY_SO_TIMEOUT, "25000");
+ overrides.setProperty(Constants.PROPERTY_MAX_RETRIES,
Integer.toString(100));
+
+ ContextBuilder contextBuilder = ContextBuilder.newBuilder(driver);
+
+ AWSCredentials credentials = null;
+ try {
+ DefaultAWSCredentialsProviderChain creds =
DefaultAWSCredentialsProviderChain.getInstance();
+ credentials = creds.getCredentials();
+ } catch (Exception e) {
+ log.error("Exception when get credentials for s3 ", e);
+ }
+
+ String id = "accesskey";
Review comment:
Thanks. will keep this default value. empty string will meet error in
jclouds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
