Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2519498911 @sunxiaojian , merged to main, thanks for your contribution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 merged PR #5701: URL: https://github.com/apache/gravitino/pull/5701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870578074
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2518995553 @FANNG1 @yuqi1129 thanks for your review, all comments have been fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870556475
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
+ implementation(libs.aliyun.kms.sdk)
Review Comment:
removed kms sdk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870540905
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
Review Comment:
got it, thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701: URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870523543 ## docs/iceberg-rest-service.md: ## @@ -128,16 +128,20 @@ To configure the JDBC catalog backend, set the `gravitino.iceberg-rest.warehouse OSS configuration -Gravitino Iceberg REST service supports using static access-key-id and secret-access-key to access OSS data. +Gravitino Iceberg REST service supports using static access-key-id and secret-access-key or generating temporary token to access OSS data. -| Configuration item | Description | Default value | Required | Since Version| -||---|---|--|--| -| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none)| No | 0.6.0-incubating | -| `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access key used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none)| No | 0.7.0-incubating | +| Configuration item| Description | Default value | Required | Since Version| +|---|--|-|--|--| +| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none) | No | 0.6.0-incubating | +| `gravitino.iceberg-rest.oss-access-key-id`| The static access key ID used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-secret-access-key`| The static secret access key used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-region` | The region of the OSS service, like `oss-cn-hangzhou`, only used when `credential-provider-type` is `oss-token`. | (none) | No | 0.8.0-incubating | +| `gravitino.iceberg-rest.oss-role-arn` | The ARN of the role to access the OSS data, only used when `credential-provider-type` is `oss-token`. | (none) | No | 0.8.0-incubating | +| `gravitino.iceberg-rest.oss-external-id` | The OSS external id to generate token, only used when `credential-provider-type` is `oss-token`. | (none) | No | 0.8.0-incubating | +| `gravitino.iceberg-rest.oss-token-expire-in-secs` | The OSS security token expire time in secs, only used when `credential-provider-type` is `oss-token`. | 3600| No | 0.8.0-incubating | -For other Iceberg OSS properties not managed by Gravitino like `client.security-token`, you could config it directly by `gravitino.iceberg-rest.client.security-token`. Review Comment: reverted -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870515802
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
Review Comment:
Credentials Java SDK depends on this package, and JDK>9 requires manual add
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1870515802
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
Review Comment:
aliyun oss SDK depends on this package, and JDK>9 requires manual add
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1869393921
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2517199656 > @FANNG1 rebuild and try it Cool! I could pass the test in deploy mode, LGTM except minor comments, could you fix it? @yuqi1129 do you have time to view it again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1869368928
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
Review Comment:
please keep alphabetical order
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
you could use `@SuppressWarnings("unused")`
##
bundles/aliyun-bundle/build.gradle.kts:
##
@@ -25,8 +25,15 @@ plugins {
}
dependencies {
+ compileOnly(project(":api"))
+ compileOnly(project(":core"))
+ compileOnly(project(":catalogs:catalog-common"))
compileOnly(project(":catalogs:catalog-hadoop"))
compileOnly(libs.hadoop3.common)
+
+ implementation(libs.aliyun.credentials.sdk)
+ implementation(libs.sun.activation)
+ implementation(libs.aliyun.kms.sdk)
Review Comment:
is it necessary to add this dependence?
##
docs/iceberg-rest-service.md:
##
@@ -128,16 +128,20 @@ To configure the JDBC catalog backend, set the
`gravitino.iceberg-rest.warehouse
OSS configuration
-Gravitino Iceberg REST service supports using static access-key-id and
secret-access-key to access OSS data.
+Gravitino Iceberg REST service supports using static access-key-id and
secret-access-key or generating temporary token to access OSS data.
-| Configuration item | Description
|
Default value | Required | Since Version|
-||---|---|--|--|
-| `gravitino.iceberg-rest.io-impl` | The IO implementation for
`FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. |
(none)| No | 0.6.0-incubating |
-| `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID
used to access OSS data. |
(none)| No | 0.7.0-incubating |
-| `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access
key used to access OSS data. |
(none)| No | 0.7.0-incubating |
-| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS
service. |
(none)| No | 0.7.0-incubating |
+| Configuration item| Description
| Default value | Required | Since Version|
+|---|--|-|--|--|
+| `gravi
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2516581242 > I couldn't run pass the test in deploy mode, ` ./gradlew :iceberg:iceberg-rest-server:test --tests "org.apache.gravitino.iceberg.integration.test.IcebergRESTOSSIT.testSnapshot" -PtestMode=deploy` with following errors, seems there are class conflict reduced by the oss-bundle jar, could you fix it? You could refer to https://gravitino.apache.org/docs/0.7.0-incubating/how-to-test/ about how to run integration test in deploy mode. > > ``` > Exception in thread "main" java.lang.NoClassDefFoundError: javax/activation/DataSource > at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl.(RuntimeBuiltinLeafInfoImpl.java:478) > at com.sun.xml.bind.v2.model.impl.RuntimeTypeInfoSetImpl.(RuntimeTypeInfoSetImpl.java:63) > at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:128) > at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:84) > at com.sun.xml.bind.v2.model.impl.ModelBuilder.(ModelBuilder.java:162) > at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.(RuntimeModelBuilder.java:92) > ``` @FANNG1 rebuild and try it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867827508
##
core/src/main/java/org/apache/gravitino/credential/config/OSSCredentialConfig.java:
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential.config;
+
+import java.util.Map;
+import javax.validation.constraints.NotNull;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.Config;
+import org.apache.gravitino.config.ConfigBuilder;
+import org.apache.gravitino.config.ConfigConstants;
+import org.apache.gravitino.config.ConfigEntry;
+import org.apache.gravitino.credential.CredentialConstants;
+import org.apache.gravitino.storage.OSSProperties;
+
+public class OSSCredentialConfig extends Config {
+
+ public static final ConfigEntry OSS_REGION =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_REGION)
+ .doc("The region of the OSS service")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .create();
+
+ public static final ConfigEntry OSS_ACCESS_KEY_ID =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ACCESS_KEY_ID)
+ .doc("The static access key ID used to access OSS data")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_SECRET_ACCESS_KEY =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ACCESS_KEY_SECRET)
+ .doc("The static secret access key used to access OSS data")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_ROLE_ARN =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ROLE_ARN)
+ .doc("OSS role arn")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_EXTERNAL_ID =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_EXTERNAL_ID)
+ .doc("OSS external ID")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .create();
+
+ public static final ConfigEntry OSS_TOKEN_EXPIRE_IN_SECS =
+ new ConfigBuilder(CredentialConstants.OSS_TOKEN_EXPIRE_IN_SECS)
+ .doc("OSS token expire in seconds")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .intConf()
+ .createWithDefault(3700);
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2514392294 I couldn't run pass the test in deploy mode, ` ./gradlew :iceberg:iceberg-rest-server:test --tests "org.apache.gravitino.iceberg.integration.test.IcebergRESTOSSIT.testSnapshot" -PtestMode=deploy` with following errors, seems there are class conflict reduced by the oss-bundle jar, could you fix it? ``` Exception in thread "main" java.lang.NoClassDefFoundError: javax/activation/DataSource at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl.(RuntimeBuiltinLeafInfoImpl.java:478) at com.sun.xml.bind.v2.model.impl.RuntimeTypeInfoSetImpl.(RuntimeTypeInfoSetImpl.java:63) at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:128) at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.createTypeInfoSet(RuntimeModelBuilder.java:84) at com.sun.xml.bind.v2.model.impl.ModelBuilder.(ModelBuilder.java:162) at com.sun.xml.bind.v2.model.impl.RuntimeModelBuilder.(RuntimeModelBuilder.java:92) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867587781
##
core/src/main/java/org/apache/gravitino/credential/config/OSSCredentialConfig.java:
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential.config;
+
+import java.util.Map;
+import javax.validation.constraints.NotNull;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.Config;
+import org.apache.gravitino.config.ConfigBuilder;
+import org.apache.gravitino.config.ConfigConstants;
+import org.apache.gravitino.config.ConfigEntry;
+import org.apache.gravitino.credential.CredentialConstants;
+import org.apache.gravitino.storage.OSSProperties;
+
+public class OSSCredentialConfig extends Config {
+
+ public static final ConfigEntry OSS_REGION =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_REGION)
+ .doc("The region of the OSS service")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .create();
+
+ public static final ConfigEntry OSS_ACCESS_KEY_ID =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ACCESS_KEY_ID)
+ .doc("The static access key ID used to access OSS data")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_SECRET_ACCESS_KEY =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ACCESS_KEY_SECRET)
+ .doc("The static secret access key used to access OSS data")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_ROLE_ARN =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_ROLE_ARN)
+ .doc("OSS role arn")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .checkValue(StringUtils::isNotBlank,
ConfigConstants.NOT_BLANK_ERROR_MSG)
+ .create();
+
+ public static final ConfigEntry OSS_EXTERNAL_ID =
+ new ConfigBuilder(OSSProperties.GRAVITINO_OSS_EXTERNAL_ID)
+ .doc("OSS external ID")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .stringConf()
+ .create();
+
+ public static final ConfigEntry OSS_TOKEN_EXPIRE_IN_SECS =
+ new ConfigBuilder(CredentialConstants.OSS_TOKEN_EXPIRE_IN_SECS)
+ .doc("OSS token expire in seconds")
+ .version(ConfigConstants.VERSION_0_8_0)
+ .intConf()
+ .createWithDefault(3700);
Review Comment:
`3700` seems exceed the max expire time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867579548
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867579548
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867579548
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867579548
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867354188
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867353038
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Policy.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.ArrayList;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Policy {
+
+ @JsonProperty("Version")
+ private String version;
+
+ @JsonProperty("Statement")
+ private List statements;
+
+ private Policy(Builder builder) {
+this.version = builder.version;
+this.statements = builder.statements;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private String version;
+private List statements = new ArrayList<>();
+
+public Builder version(String version) {
+ this.version = version;
+ return this;
+}
+
+public Builder addStatement(Statement statement) {
+ this.statements.add(statement);
+ return this;
+}
+
+public Policy build() {
+ return new Policy(this);
+}
+ }
+
+ public String getVersion() {
+return version;
+ }
+
+ public void setVersion(String version) {
+this.version = version;
+ }
+
+ public List getStatements() {
Review Comment:
ditto , retain the get method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867351399
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867341338
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701: URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867330110 ## docs/iceberg-rest-service.md: ## @@ -130,14 +130,18 @@ To configure the JDBC catalog backend, set the `gravitino.iceberg-rest.warehouse Gravitino Iceberg REST service supports using static access-key-id and secret-access-key to access OSS data. Review Comment: fixed ## docs/iceberg-rest-service.md: ## @@ -130,14 +130,18 @@ To configure the JDBC catalog backend, set the `gravitino.iceberg-rest.warehouse Gravitino Iceberg REST service supports using static access-key-id and secret-access-key to access OSS data. -| Configuration item | Description | Default value | Required | Since Version| -||---|---|--|--| -| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none)| No | 0.6.0-incubating | -| `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access key used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none)| No | 0.7.0-incubating | - -For other Iceberg OSS properties not managed by Gravitino like `client.security-token`, you could config it directly by `gravitino.iceberg-rest.client.security-token`. +| Configuration item| Description | Default value | Required | Since Version| +|---||-|--|--| +| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none) | No | 0.6.0-incubating | +| `gravitino.iceberg-rest.oss-access-key-id`| The static access key ID used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-secret-access-key`| The static secret access key used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-region` | The region of the OSS service, like `oss-cn-hangzhou`. | (none) | No | 0.8.0-incubating | +| `gravitino.iceberg-rest.oss-role-arn` | The ARN of the role to access the OSS data. | (none) | No | 0.8.0-incubating | Review Comment: fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867315483
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701: URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867314346 ## docs/iceberg-rest-service.md: ## @@ -130,14 +130,18 @@ To configure the JDBC catalog backend, set the `gravitino.iceberg-rest.warehouse Gravitino Iceberg REST service supports using static access-key-id and secret-access-key to access OSS data. -| Configuration item | Description | Default value | Required | Since Version| -||---|---|--|--| -| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none)| No | 0.6.0-incubating | -| `gravitino.iceberg-rest.oss-access-key-id` | The static access key ID used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-secret-access-key` | The static secret access key used to access OSS data. | (none)| No | 0.7.0-incubating | -| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none)| No | 0.7.0-incubating | - -For other Iceberg OSS properties not managed by Gravitino like `client.security-token`, you could config it directly by `gravitino.iceberg-rest.client.security-token`. +| Configuration item| Description | Default value | Required | Since Version| +|---||-|--|--| +| `gravitino.iceberg-rest.io-impl` | The IO implementation for `FileIO` in Iceberg, use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS. | (none) | No | 0.6.0-incubating | +| `gravitino.iceberg-rest.oss-access-key-id`| The static access key ID used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-secret-access-key`| The static secret access key used to access OSS data. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-endpoint` | The endpoint of Aliyun OSS service. | (none) | No | 0.7.0-incubating | +| `gravitino.iceberg-rest.oss-region` | The region of the OSS service, like `oss-cn-hangzhou`. | (none) | No | 0.8.0-incubating | +| `gravitino.iceberg-rest.oss-role-arn` | The ARN of the role to access the OSS data. | (none) | No | 0.8.0-incubating | Review Comment: could you add only used when `credential-provider-type` is `oss-token`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701: URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867313462 ## docs/iceberg-rest-service.md: ## @@ -130,14 +130,18 @@ To configure the JDBC catalog backend, set the `gravitino.iceberg-rest.warehouse Gravitino Iceberg REST service supports using static access-key-id and secret-access-key to access OSS data. Review Comment: this line should update too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867308410
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,268 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867271472
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867249973
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
Could you figure out the reason and add comments to the methods ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867220956
##
api/src/main/java/org/apache/gravitino/credential/OSSTokenCredential.java:
##
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+
+/** OSS token credential. */
+public class OSSTokenCredential implements Credential {
+
+ /** OSS token credential type. */
+ public static final String OSS_TOKEN_CREDENTIAL_TYPE = "oss-token";
+ /** OSS session access key ID used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_ACCESS_KEY_ID =
"oss-access-key-id";
+ /** OSS session secret access key used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_SECRET_ACCESS_KEY =
"oss-secret-access-key";
+ /** OSS session token. */
+ public static final String GRAVITINO_OSS_TOKEN = "oss-security-token";
+
+ private final String accessKeyId;
+ private final String secretAccessKey;
+ private final String securityToken;
+ private final long expireTimeInMS;
Review Comment:
timestamp value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867192017
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
Review Comment:
AMRoleARN supports automatic token refresh, but I haven't used it yet. The
underlying imp
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867184580
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2513756606 > Could you add related document in https://github.com/apache/gravitino/blob/main/docs/iceberg-rest-service.md ? added -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867178958
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
After removal, ci cannot pass，retain the get method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867164813
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Policy.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.ArrayList;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Policy {
+
+ @JsonProperty("Version")
+ private String version;
+
+ @JsonProperty("Statement")
+ private List statements;
+
+ private Policy(Builder builder) {
+this.version = builder.version;
+this.statements = builder.statements;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private String version;
+private List statements = new ArrayList<>();
+
+public Builder version(String version) {
+ this.version = version;
+ return this;
+}
+
+public Builder addStatement(Statement statement) {
+ this.statements.add(statement);
+ return this;
+}
+
+public Policy build() {
+ return new Policy(this);
+}
+ }
+
+ public String getVersion() {
+return version;
+ }
+
+ public void setVersion(String version) {
+this.version = version;
+ }
+
+ public List getStatements() {
Review Comment:
removed
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867172527
##
common/src/main/java/org/apache/gravitino/credential/CredentialPropertyUtils.java:
##
@@ -62,6 +72,9 @@ public static Map
toIcebergProperties(Credential credential) {
if (credential instanceof S3TokenCredential || credential instanceof
S3SecretKeyCredential) {
return transformProperties(credential.credentialInfo(),
icebergCredentialPropertyMap);
}
+if (credential instanceof OSSTokenCredential) {
Review Comment:
added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867172325
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867169972
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867170449
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867167981
##
api/src/main/java/org/apache/gravitino/credential/OSSTokenCredential.java:
##
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+
+/** OSS token credential. */
+public class OSSTokenCredential implements Credential {
+
+ /** OSS token credential type. */
+ public static final String OSS_TOKEN_CREDENTIAL_TYPE = "oss-token";
+ /** OSS session access key ID used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_ACCESS_KEY_ID =
"oss-access-key-id";
+ /** OSS session secret access key used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_SECRET_ACCESS_KEY =
"oss-secret-access-key";
+ /** OSS session token. */
+ public static final String GRAVITINO_OSS_TOKEN = "oss-security-token";
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867165017
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Condition.java:
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Condition {
+
+ @JsonProperty("StringLike")
+ private StringLike stringLike;
+
+ private Condition(Builder builder) {
+this.stringLike = builder.stringLike;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private StringLike stringLike;
+
+public Builder stringLike(StringLike stringLike) {
+ this.stringLike = stringLike;
+ return this;
+}
+
+public Condition build() {
+ return new Condition(this);
+}
+ }
+
+ public StringLike getStringLike() {
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867164813
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/policy/Policy.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.oss.credential.policy;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.ArrayList;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+public class Policy {
+
+ @JsonProperty("Version")
+ private String version;
+
+ @JsonProperty("Statement")
+ private List statements;
+
+ private Policy(Builder builder) {
+this.version = builder.version;
+this.statements = builder.statements;
+ }
+
+ public static Builder builder() {
+return new Builder();
+ }
+
+ public static class Builder {
+private String version;
+private List statements = new ArrayList<>();
+
+public Builder version(String version) {
+ this.version = version;
+ return this;
+}
+
+public Builder addStatement(Statement statement) {
+ this.statements.add(statement);
+ return this;
+}
+
+public Policy build() {
+ return new Policy(this);
+}
+ }
+
+ public String getVersion() {
+return version;
+ }
+
+ public void setVersion(String version) {
+this.version = version;
+ }
+
+ public List getStatements() {
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1867003334
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1866974275
##
iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTOSSIT.java:
##
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.integration.test;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
+import org.apache.gravitino.credential.CredentialConstants;
+import org.apache.gravitino.iceberg.common.IcebergConfig;
+import org.apache.gravitino.integration.test.util.BaseIT;
+import org.apache.gravitino.integration.test.util.DownloaderUtils;
+import org.apache.gravitino.integration.test.util.ITUtils;
+import org.apache.gravitino.storage.OSSProperties;
+import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
+import org.junit.platform.commons.util.StringUtils;
+
+@EnabledIfEnvironmentVariable(named = "GRAVITINO_TEST_CLOUD_IT", matches =
"true")
+public class IcebergRESTOSSIT extends IcebergRESTJdbcCatalogIT {
+
+ private String warehouse;
+ private String accessKey;
+ private String secretKey;
+ private String endpoint;
+ private String roleArn;
+ private String externalId;
+ private String region;
+
+ @Override
+ void initEnv() {
+this.warehouse =
+String.format(
+"oss://%s/gravitino-test",
+getFromEnvOrDefault("GRAVITINO_OSS_BUCKET", "{BUCKET_NAME}"));
+this.accessKey = getFromEnvOrDefault("GRAVITINO_OSS_ACCESS_KEY",
"{ACCESS_KEY}");
+this.secretKey = getFromEnvOrDefault("GRAVITINO_OSS_SECRET_KEY",
"{SECRET_KEY}");
+this.endpoint = getFromEnvOrDefault("GRAVITINO_OSS_ENDPOINT",
"{GRAVITINO_OSS_ENDPOINT}");
+this.region = getFromEnvOrDefault("GRAVITINO_OSS_REGION",
"{GRAVITINO_OSS_REGION}");
+this.roleArn = getFromEnvOrDefault("GRAVITINO_OSS_ROLE_ARN", "{ROLE_ARN}");
+this.externalId = getFromEnvOrDefault("GRAVITINO_OSS_EXTERNAL_ID", "");
+
+if (ITUtils.isEmbedded()) {
+ return;
+}
+try {
+ downloadIcebergForAliyunJar();
+} catch (IOException e) {
+ LOG.warn("Download Iceberg AWS bundle jar failed,", e);
+ throw new RuntimeException(e);
+}
+copyAliyunOSSJar();
+ }
+
+ @Override
+ public Map getCatalogConfig() {
+HashMap m = new HashMap();
+m.putAll(getCatalogJdbcConfig());
+m.putAll(getOSSConfig());
+return m;
+ }
+
+ public boolean supportsCredentialVending() {
+return true;
+ }
+
+ private Map getOSSConfig() {
+Map configMap = new HashMap();
+
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX +
CredentialConstants.CREDENTIAL_PROVIDER_TYPE,
+CredentialConstants.OSS_TOKEN_CREDENTIAL_PROVIDER);
+configMap.put(IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_REGION, region);
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_ENDPOINT, endpoint);
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_ACCESS_KEY_ID, accessKey);
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_ACCESS_KEY_SECRET,
+secretKey);
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_ROLE_ARN, roleArn);
+if (StringUtils.isNotBlank(externalId)) {
+ configMap.put(
+ IcebergConfig.ICEBERG_CONFIG_PREFIX +
OSSProperties.GRAVITINO_OSS_EXTERNAL_ID,
+ externalId);
+}
+
+configMap.put(
+IcebergConfig.ICEBERG_CONFIG_PREFIX + IcebergConstants.IO_IMPL,
+"org.apache.iceberg.aliyun.oss.OSSFileIO");
+configMap.put(IcebergConfig.ICEBERG_CONFIG_PREFIX +
IcebergConstants.WAREHOUSE, warehouse);
+
+return configMap;
+ }
+
+ private void downloadIcebergForAliyunJar() throws IOException {
+String icebergBundleJarName = "iceberg-aliyun-1.5.2.jar";
+String icebergBundleJarUri =
+"https://repo1.maven.org/maven2/org/apache/iceberg/";
++ "iceberg-aliyun/1.5.2
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1866973937
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
yuqi1129 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1862320351
##
api/src/main/java/org/apache/gravitino/credential/OSSTokenCredential.java:
##
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+
+/** OSS token credential. */
+public class OSSTokenCredential implements Credential {
+
+ /** OSS token credential type. */
+ public static final String OSS_TOKEN_CREDENTIAL_TYPE = "oss-token";
+ /** OSS session access key ID used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_ACCESS_KEY_ID =
"oss-access-key-id";
+ /** OSS session secret access key used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_SECRET_ACCESS_KEY =
"oss-secret-access-key";
+ /** OSS session token. */
+ public static final String GRAVITINO_OSS_TOKEN = "oss-security-token";
+
+ private final String accessKeyId;
+ private final String secretAccessKey;
+ private final String securityToken;
+ private final long expireTimeInMS;
Review Comment:
is `expireTimeInMS` a timestamp value?
##
iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/integration/test/IcebergRESTOSSIT.java:
##
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.integration.test;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergConstants;
+import org.apache.gravitino.credential.CredentialConstants;
+import org.apache.gravitino.iceberg.common.IcebergConfig;
+import org.apache.gravitino.integration.test.util.BaseIT;
+import org.apache.gravitino.integration.test.util.DownloaderUtils;
+import org.apache.gravitino.integration.test.util.ITUtils;
+import org.apache.gravitino.storage.OSSProperties;
+import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
+import org.junit.platform.commons.util.StringUtils;
+
+@EnabledIfEnvironmentVariable(named = "GRAVITINO_TEST_CLOUD_IT", matches =
"true")
+public class IcebergRESTOSSIT extends IcebergRESTJdbcCatalogIT {
+
+ private String warehouse;
+ private String accessKey;
+ private String secretKey;
+ private String endpoint;
+ private String roleArn;
+ private String externalId;
+ private String region;
+
+ @Override
+ void initEnv() {
+this.warehouse =
+String.format(
+"oss://%s/gravitino-test",
+getFromEnvOrDefault("GRAVITINO_OSS_BUCKET", "{BUCKET_NAME}"));
+this.accessKey = getFromEnvOrDefault("GRAVITINO_OSS_ACCESS_KEY",
"{ACCESS_KEY}");
+this.secretKey = getFromEnvOrDefault("GRAVITINO_OSS_SECRET_KEY",
"{SECRET_KEY}");
+this.endpoint = getFromEnvOrDefault("GRAVITINO_OSS_ENDPOINT",
"{GRAVITINO_OSS_ENDPOINT}");
+this.region = getFromEnvOrDefault("GRAVITINO_OSS_REGION",
"{GRAVITINO_OSS_REGION}");
+this.roleArn = getFromEnvOrDefault("GRAVITINO_OSS_ROLE_ARN", "{ROLE_ARN}");
+this.externalId = getFromEnvOrDefault("GRAVITINO_OSS_EXTERNAL_ID", "");
+
+if (ITUtils.isEmbedded()) {
+ return;
+}
+try {
+ downloadIcebergForAliyun
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1866802311
##
api/src/main/java/org/apache/gravitino/credential/OSSTokenCredential.java:
##
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+
+/** OSS token credential. */
+public class OSSTokenCredential implements Credential {
+
+ /** OSS token credential type. */
+ public static final String OSS_TOKEN_CREDENTIAL_TYPE = "oss-token";
+ /** OSS session access key ID used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_ACCESS_KEY_ID =
"oss-access-key-id";
+ /** OSS session secret access key used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_SECRET_ACCESS_KEY =
"oss-secret-access-key";
+ /** OSS session token. */
+ public static final String GRAVITINO_OSS_TOKEN = "oss-security-token";
Review Comment:
ok, please keep consistent with the comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1866801857
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1865704686
##
api/src/main/java/org/apache/gravitino/credential/OSSTokenCredential.java:
##
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.credential;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import java.util.Map;
+import org.apache.commons.lang3.StringUtils;
+
+/** OSS token credential. */
+public class OSSTokenCredential implements Credential {
+
+ /** OSS token credential type. */
+ public static final String OSS_TOKEN_CREDENTIAL_TYPE = "oss-token";
+ /** OSS session access key ID used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_ACCESS_KEY_ID =
"oss-access-key-id";
+ /** OSS session secret access key used to access OSS data. */
+ public static final String GRAVITINO_OSS_SESSION_SECRET_ACCESS_KEY =
"oss-secret-access-key";
+ /** OSS session token. */
+ public static final String GRAVITINO_OSS_TOKEN = "oss-security-token";
Review Comment:
I think it should be consistent with the OSS document
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1865701180
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Con
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2511259317 Could you add related document in https://github.com/apache/gravitino/blob/main/docs/iceberg-rest-service.md ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1865515898
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
+ String roleArn, Set readLocations, Set writeLocations,
String userName) {
+Config config = new Config()
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on code in PR #5701:
URL: https://github.com/apache/gravitino/pull/5701#discussion_r1865514938
##
bundles/aliyun-bundle/src/main/java/org/apache/gravitino/oss/credential/OSSTokenProvider.java:
##
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.oss.credential;
+
+import com.aliyun.credentials.Client;
+import com.aliyun.credentials.models.Config;
+import com.aliyun.credentials.models.CredentialModel;
+import com.aliyun.credentials.utils.AuthConstant;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.io.IOException;
+import java.net.URI;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.annotation.Nullable;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.credential.Credential;
+import org.apache.gravitino.credential.CredentialContext;
+import org.apache.gravitino.credential.CredentialProvider;
+import org.apache.gravitino.credential.OSSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.credential.config.OSSCredentialConfig;
+import org.apache.gravitino.oss.credential.policy.Condition;
+import org.apache.gravitino.oss.credential.policy.Effect;
+import org.apache.gravitino.oss.credential.policy.Policy;
+import org.apache.gravitino.oss.credential.policy.Statement;
+import org.apache.gravitino.oss.credential.policy.StringLike;
+
+/** Generates OSS token to access OSS data. */
+public class OSSTokenProvider implements CredentialProvider {
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private String accessKeyId;
+ private String secretAccessKey;
+ private String roleArn;
+ private String externalID;
+ private int tokenExpireSecs;
+ private String region;
+
+ /**
+ * Initializes the credential provider with catalog properties.
+ *
+ * @param properties catalog properties that can be used to configure the
provider. The specific
+ * properties required vary by implementation.
+ */
+ @Override
+ public void initialize(Map properties) {
+OSSCredentialConfig credentialConfig = new OSSCredentialConfig(properties);
+this.roleArn = credentialConfig.ossRoleArn();
+this.externalID = credentialConfig.externalID();
+this.tokenExpireSecs = credentialConfig.tokenExpireInSecs();
+this.accessKeyId = credentialConfig.accessKeyID();
+this.secretAccessKey = credentialConfig.secretAccessKey();
+this.region = credentialConfig.region();
+ }
+
+ /**
+ * Returns the type of credential, it should be identical in Gravitino.
+ *
+ * @return A string identifying the type of credentials.
+ */
+ @Override
+ public String credentialType() {
+return OSSTokenCredential.OSS_TOKEN_CREDENTIAL_TYPE;
+ }
+
+ /**
+ * Obtains a credential based on the provided context information.
+ *
+ * @param context A context object providing necessary information for
retrieving credentials.
+ * @return A Credential object containing the authentication information
needed to access a system
+ * or resource. Null will be returned if no credential is available.
+ */
+ @Nullable
+ @Override
+ public Credential getCredential(CredentialContext context) {
+if (!(context instanceof PathBasedCredentialContext)) {
+ return null;
+}
+PathBasedCredentialContext pathBasedCredentialContext =
(PathBasedCredentialContext) context;
+CredentialModel credentialModel =
+createOSSCredentialModel(
+roleArn,
+pathBasedCredentialContext.getReadPaths(),
+pathBasedCredentialContext.getWritePaths(),
+pathBasedCredentialContext.getUserName());
+return new OSSTokenCredential(
+credentialModel.accessKeyId,
+credentialModel.accessKeySecret,
+credentialModel.securityToken,
+credentialModel.expiration);
+ }
+
+ private CredentialModel createOSSCredentialModel(
Review Comment:
Is there any reason to use `RAMRoleARN` to get credentials not `assumeRole`
from STS directly?
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
FANNG1 commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2506980367 @sunxiaojian , great work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ #5625] feat(bundles) support Aliyun oss credential provider [gravitino]
sunxiaojian commented on PR #5701: URL: https://github.com/apache/gravitino/pull/5701#issuecomment-2506442889 @FANNG1 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
