yuqi1129 commented on code in PR #5040:
URL: https://github.com/apache/gravitino/pull/5040#discussion_r1796624990
##########
docs/spark-connector/spark-authentication-with-gravitino.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Spark authentication with Gravitino server"
+slug: /spark-connector/spark-authentication
+keyword: spark connector authentication oauth2 kerberos
+license: "This software is licensed under the Apache License version 2."
+---
+
+## Overview
+
+Spark connector supports `simple` `oauth2` and `kerberos` authentication when
accessing Gravitino server.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|------------------------------|--------|---------------|---------------------------------------------------------------------------------------------------------------------|----------|---------------|
+| spark.sql.gravitino.authType | string | `simple` | The authentication
mechanisms when communicating with Gravitino server, supports `simple`,
`oauth2` and `kerberos`. | No | 0.7.0 |
+
+## Simple mode
+
+In the simple mode, you could specify the username by configuration.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|-------------------------------------|--------|---------------|---------------------------------------|----------|---------------|
+| spark.sql.gravitino.simple.userName | string | None | The user name
to access the Gravitino | No | 0.7.0 |
+
+If you doesn't set `spark.sql.gravitino.simple.userName` explicitly. It will
use the value of environment variable `GRAVITINO_USER` as the user. If the
environment variable `GRAVITINO_USER` isn't set, the client uses the user
logging in the machine.
Review Comment:
So the default value of `spark.sql.gravitino.simple.userName` is not `None`?
##########
spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/plugin/GravitinoDriverPlugin.java:
##########
@@ -155,4 +162,45 @@ private void registerSqlExtensions(SparkConf conf) {
conf.set(StaticSQLConf.SPARK_SESSION_EXTENSIONS().key(),
extensionString);
}
}
+
+ private static GravitinoClient createGravitinoClient(
+ String uri, String metalake, SparkConf sparkConf) {
+ ClientBuilder builder =
GravitinoClient.builder(uri).withMetalake(metalake);
+ String authType =
+ sparkConf.get(GravitinoSparkConfig.GRAVITINO_AUTH_TYPE,
AuthProperties.SIMPLE_AUTH_TYPE);
+ if (AuthProperties.isSimple(authType)) {
+ String username =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_SIMPLE_USER_NAME, null);
+ if (StringUtils.isNotBlank(username)) {
+ builder.withSimpleAuth(username.trim());
+ } else {
+ builder.withSimpleAuth();
+ }
+ } else if (AuthProperties.isOAuth2(authType)) {
+ String oAuthUri =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_URI, null);
+ String credential =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_CREDENTIAL, null);
+ String path = sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_PATH,
null);
+ String scope =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_SCOPE, null);
+ DefaultOAuth2TokenProvider oAuth2TokenProvider =
+ DefaultOAuth2TokenProvider.builder()
+ .withUri(oAuthUri)
+ .withCredential(credential)
+ .withPath(path)
+ .withScope(scope)
+ .build();
+ builder.withOAuth(oAuth2TokenProvider);
+ } else if (AuthProperties.isKerberos(authType)) {
+ String principal =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_KERBEROS_PRINCIPAL, null);
Review Comment:
The default value `null` is unnecessary since null is NOT a valid value for
`principal`, could you check the validity before creating a Gravitino client?
##########
spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/plugin/GravitinoDriverPlugin.java:
##########
@@ -155,4 +162,45 @@ private void registerSqlExtensions(SparkConf conf) {
conf.set(StaticSQLConf.SPARK_SESSION_EXTENSIONS().key(),
extensionString);
}
}
+
+ private static GravitinoClient createGravitinoClient(
+ String uri, String metalake, SparkConf sparkConf) {
+ ClientBuilder builder =
GravitinoClient.builder(uri).withMetalake(metalake);
+ String authType =
+ sparkConf.get(GravitinoSparkConfig.GRAVITINO_AUTH_TYPE,
AuthProperties.SIMPLE_AUTH_TYPE);
+ if (AuthProperties.isSimple(authType)) {
+ String username =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_SIMPLE_USER_NAME, null);
+ if (StringUtils.isNotBlank(username)) {
+ builder.withSimpleAuth(username.trim());
+ } else {
+ builder.withSimpleAuth();
+ }
+ } else if (AuthProperties.isOAuth2(authType)) {
+ String oAuthUri =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_URI, null);
+ String credential =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_CREDENTIAL, null);
+ String path = sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_PATH,
null);
+ String scope =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_OAUTH2_SCOPE, null);
+ DefaultOAuth2TokenProvider oAuth2TokenProvider =
+ DefaultOAuth2TokenProvider.builder()
+ .withUri(oAuthUri)
+ .withCredential(credential)
+ .withPath(path)
+ .withScope(scope)
+ .build();
+ builder.withOAuth(oAuth2TokenProvider);
+ } else if (AuthProperties.isKerberos(authType)) {
+ String principal =
sparkConf.get(GravitinoSparkConfig.GRAVITINO_KERBEROS_PRINCIPAL, null);
+ String keyTabFile =
+
sparkConf.get(GravitinoSparkConfig.GRAVITINO_KERBEROS_KEYTAB_FILE_PATH, null);
+ KerberosTokenProvider kerberosTokenProvider =
+ KerberosTokenProvider.builder()
+ .withClientPrincipal(principal)
+ .withKeyTabFile(new File(keyTabFile))
+ .build();
+ builder.withKerberosAuth(kerberosTokenProvider);
+ } else {
+ throw new UnsupportedOperationException("Doesn't support auth: " +
authType);
Review Comment:
Unsupported auth type: xxxx
##########
docs/spark-connector/spark-authentication-with-gravitino.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Spark authentication with Gravitino server"
+slug: /spark-connector/spark-authentication
+keyword: spark connector authentication oauth2 kerberos
+license: "This software is licensed under the Apache License version 2."
+---
+
+## Overview
+
+Spark connector supports `simple` `oauth2` and `kerberos` authentication when
accessing Gravitino server.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|------------------------------|--------|---------------|---------------------------------------------------------------------------------------------------------------------|----------|---------------|
+| spark.sql.gravitino.authType | string | `simple` | The authentication
mechanisms when communicating with Gravitino server, supports `simple`,
`oauth2` and `kerberos`. | No | 0.7.0 |
+
+## Simple mode
+
+In the simple mode, you could specify the username by configuration.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|-------------------------------------|--------|---------------|---------------------------------------|----------|---------------|
+| spark.sql.gravitino.simple.userName | string | None | The user name
to access the Gravitino | No | 0.7.0 |
Review Comment:
Can we use `spark.sql.gravitino.auth.simple.userName`?
##########
docs/spark-connector/spark-authentication-with-gravitino.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Spark authentication with Gravitino server"
+slug: /spark-connector/spark-authentication
+keyword: spark connector authentication oauth2 kerberos
+license: "This software is licensed under the Apache License version 2."
+---
+
+## Overview
+
+Spark connector supports `simple` `oauth2` and `kerberos` authentication when
accessing Gravitino server.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|------------------------------|--------|---------------|---------------------------------------------------------------------------------------------------------------------|----------|---------------|
+| spark.sql.gravitino.authType | string | `simple` | The authentication
mechanisms when communicating with Gravitino server, supports `simple`,
`oauth2` and `kerberos`. | No | 0.7.0 |
+
+## Simple mode
+
+In the simple mode, you could specify the username by configuration.
+
+| Property | Type | Default Value | Description
| Required | Since Version |
+|-------------------------------------|--------|---------------|---------------------------------------|----------|---------------|
+| spark.sql.gravitino.simple.userName | string | None | The user name
to access the Gravitino | No | 0.7.0 |
+
+If you doesn't set `spark.sql.gravitino.simple.userName` explicitly. It will
use the value of environment variable `GRAVITINO_USER` as the user. If the
environment variable `GRAVITINO_USER` isn't set, the client uses the user
logging in the machine.
+
+## OAuth2 mode
+
+In the OAuth2 mode, you could use following configuration to fetch an OAuth2
token to access Gravitino server.
Review Comment:
use following -> use the following
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
