wcmolin commented on issue #18267: URL: https://github.com/apache/dolphinscheduler/issues/18267#issuecomment-4483923633
Thanks, that makes sense. I agree we should not assume MinIO supports all AWS S3 features. I also agree that compatibility `if` branches should have a clear boundary; otherwise they can make the code harder to maintain. To clarify, this change does not rely on MinIO supporting the full S3 feature set. MinIO’s own S3 compatibility document lists both supported and unsupported APIs, so I agree we should not treat it as a complete AWS S3 replacement: [MinIO S3 API Compatibility](https://minio.community/community/minio-object-store/reference/s3-api-compatibility.html). But DolphinScheduler currently uses only a small subset of basic S3 object/bucket APIs in resource storage and remote logging, such as `PutObject`, `GetObject`, `CopyObject`, `DeleteObject`, `ListObjectsV2`, `HeadObject` / `doesObjectExist`, and `HeadBucket` / `doesBucketExistV2`. These APIs are within MinIO’s documented compatibility scope. So the current change does not require MinIO to include all S3 features; it only depends on the subset that DolphinScheduler actually uses. Also, to avoid making this look like a random compatibility `if`, we can make the boundary explicit in the code/comment. This is not a MinIO-specific branch. It is the SDK endpoint configuration path: - custom endpoint: use `EndpointConfiguration(endpoint, region)` and pass the configured region as the signing region string; - no custom endpoint: keep the existing AWS S3 path and continue validating the region with `Regions.fromName()`. AWS SDK documents `EndpointConfiguration` as the configuration for service endpoint and signing region, and the region selection doc also describes using explicit endpoint configuration when choosing a specific endpoint: [AWS SDK v1 EndpointConfiguration Javadoc](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/client/builder/AwsClientBuilder.EndpointConfiguration.html), [AWS SDK for Java 1.x Region Selection](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-region-selection.html). If DolphinScheduler later needs APIs or behavior that MinIO does not support, then splitting a dedicated plugin would be a better option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
