This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 50cd91b59 [KYUUBI #5683] Add `kyuubi.session.proxy.user` as an
alternative to `hive.server2.proxy.user`
50cd91b59 is described below
commit 50cd91b59ec09e374a6fef7a5f8e2759111e2ddd
Author: mrtisttt <[email protected]>
AuthorDate: Mon Nov 13 20:08:33 2023 +0800
[KYUUBI #5683] Add `kyuubi.session.proxy.user` as an alternative to
`hive.server2.proxy.user`
### _Why are the changes needed?_
We need a configuration to be an alternative to `hive.server2.proxy.user`.
`hive.server2.proxy.user` is derived from the hive server, and it's perfect for
keeping the behavior the same as the hive. But hive thrift protocol is only one
of the FrontendServices. There are also other FrontendServices not necessary to
follow `hive.server2.proxy.user`. So we need `kyuubi.proxy.user`.
In this commit, support for using the `kyuubi.proxy.user` configuration has
been implemented in scenarios where the original `hive.server2.proxy.user`
configuration was used.In the future, this feature will continue to be improved
according to other scenarios where it is used.
Close #5683
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run
test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests)
locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes #5653 from mrtisttt/add-kyuubi-proxy-user.
Closes #5683
6d60e7632 [mrtisttt] Rollback unnecessary modifications
7298e4137 [mrtisttt] fix doc violations
19f6f4084 [mrtisttt] For API side, shorten the name of
kyuubi.session.proxy.user and change it to proxyUser.
0fbcc704d [mrtisttt] Change kyuubi.proxy.user to kyuubi.session.proxy.user
and improve documentation
c2a810ce2 [mrtisttt] fix format violations
a6413e8aa [mrtisttt] Compatible with scenarios where
hive.server2.proxy.user is used
b687c21b9 [mrtisttt] Add kyuubi.proxy.user as an alternative to
hive.server2.proxy.user
Authored-by: mrtisttt <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
docs/client/rest/rest_api.md | 12 ++++++++++++
docs/configuration/settings.md | 1 +
.../scala/org/apache/kyuubi/config/KyuubiConf.scala | 10 ++++++++++
.../org/apache/kyuubi/service/TFrontendService.scala | 5 +++--
.../kyuubi/server/KyuubiRestFrontendService.scala | 20 +++++++++++---------
.../apache/kyuubi/server/api/v1/AdminResource.scala | 12 ++++++++----
.../kyuubi/server/api/v1/BatchesResource.scala | 8 +++++---
7 files changed, 50 insertions(+), 18 deletions(-)
diff --git a/docs/client/rest/rest_api.md b/docs/client/rest/rest_api.md
index 2fa2a7bf7..6214d977c 100644
--- a/docs/client/rest/rest_api.md
+++ b/docs/client/rest/rest_api.md
@@ -414,8 +414,12 @@ Kill the batch if it is still running.
| Name | Description | Type |
|:------------------------|:------------------------------|:-----------------|
+| proxyUser | the proxy user to impersonate | String(optional) |
| hive.server2.proxy.user | the proxy user to impersonate | String(optional) |
+`proxyUser` is an alternative to `hive.server2.proxy.user`, and the current
behavior is consistent with
+`hive.server2.proxy.user`. When both parameters are set, `proxyUser` takes
precedence.
+
#### Response Body
| Name | Description | Type |
@@ -468,8 +472,12 @@ Delete the specified engine.
| type | the engine type | String(optional) |
| sharelevel | the engine share level | String(optional) |
| subdomain | the engine subdomain | String(optional) |
+| proxyUser | the proxy user to impersonate | String(optional) |
| hive.server2.proxy.user | the proxy user to impersonate | String(optional) |
+`proxyUser` is an alternative to `hive.server2.proxy.user`, and the current
behavior is consistent with
+`hive.server2.proxy.user`. When both parameters are set, `proxyUser` takes
precedence.
+
### GET /admin/engine
Get a list of satisfied engines.
@@ -481,8 +489,12 @@ Get a list of satisfied engines.
| type | the engine type | String(optional) |
| sharelevel | the engine share level | String(optional) |
| subdomain | the engine subdomain | String(optional) |
+| proxyUser | the proxy user to impersonate | String(optional) |
| hive.server2.proxy.user | the proxy user to impersonate | String(optional) |
+`proxyUser` is an alternative to hive.server2.proxy.user, and the current
behavior is consistent with
+hive.server2.proxy.user. When both parameters are set, proxyUser takes
precedence.
+
#### Response Body
The [Engine](#engine) List.
diff --git a/docs/configuration/settings.md b/docs/configuration/settings.md
index 0fa6d8e5a..4c4eb8be9 100644
--- a/docs/configuration/settings.md
+++ b/docs/configuration/settings.md
@@ -453,6 +453,7 @@ You can configure the Kyuubi properties in
`$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.session.idle.timeout | PT6H
| session idle timeout, it will be closed when it's not accessed for this
duration
[...]
| kyuubi.session.local.dir.allow.list
|| The local dir list that are allowed to access by the kyuubi session
application. End-users might set some parameters such as `spark.files` and it
will upload some local files when launching the kyuubi engine, if the local
dir allow list is defined, kyuubi will check whether the path to upload is in
the allow list. Note that, if it is empty, there is no limitation for that. And
please use absolute paths. [...]
| kyuubi.session.name | <undefined>
| A human readable name of the session and we use empty string by default.
This name will be recorded in the event. Note that, we only apply this value
from session conf.
[...]
+| kyuubi.session.proxy.user | <undefined>
| An alternative to hive.server2.proxy.user. The current behavior is
consistent with hive.server2.proxy.user and now only takes effect in RESTFul
API. When both parameters are set, kyuubi.session.proxy.user takes precedence.
[...]
| kyuubi.session.timeout | PT6H
| (deprecated)session timeout, it will be closed when it's not accessed for
this duration
[...]
| kyuubi.session.user.sign.enabled | false
| Whether to verify the integrity of session user name on the engine side,
e.g. Authz plugin in Spark.
[...]
diff --git
a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
index 042a211a2..9ab392c2a 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
@@ -1932,6 +1932,16 @@ object KyuubiConf {
.stringConf
.createWithDefault("server_operation_logs")
+ val PROXY_USER: OptionalConfigEntry[String] =
+ buildConf("kyuubi.session.proxy.user")
+ .doc("An alternative to hive.server2.proxy.user. " +
+ "The current behavior is consistent with hive.server2.proxy.user " +
+ "and now only takes effect in RESTFul API. " +
+ "When both parameters are set, kyuubi.session.proxy.user takes
precedence.")
+ .version("1.9.0")
+ .stringConf
+ .createOptional
+
@deprecated("using kyuubi.engine.share.level instead", "1.2.0")
val LEGACY_ENGINE_SHARE_LEVEL: ConfigEntry[String] =
buildConf("kyuubi.session.engine.share.level")
diff --git
a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
index 7cc23779f..8882fbd23 100644
---
a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
+++
b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
@@ -31,7 +31,7 @@ import org.apache.thrift.transport.TTransport
import org.apache.kyuubi.{KyuubiSQLException, Logging, Utils}
import org.apache.kyuubi.Utils.stringifyException
-import org.apache.kyuubi.config.KyuubiConf.{FRONTEND_ADVERTISED_HOST,
FRONTEND_CONNECTION_URL_USE_HOSTNAME, SESSION_CLOSE_ON_DISCONNECT}
+import org.apache.kyuubi.config.KyuubiConf.{FRONTEND_ADVERTISED_HOST,
FRONTEND_CONNECTION_URL_USE_HOSTNAME, PROXY_USER, SESSION_CLOSE_ON_DISCONNECT}
import org.apache.kyuubi.config.KyuubiReservedKeys._
import org.apache.kyuubi.operation.{FetchOrientation, OperationHandle}
import org.apache.kyuubi.service.authentication.KyuubiAuthenticationFactory
@@ -127,7 +127,8 @@ abstract class TFrontendService(name: String)
sessionConf: java.util.Map[String, String],
ipAddress: String,
realUser: String): String = {
- val proxyUser = sessionConf.get(KyuubiAuthenticationFactory.HS2_PROXY_USER)
+ val proxyUser = Option(sessionConf.get(PROXY_USER.key))
+ .getOrElse(sessionConf.get(KyuubiAuthenticationFactory.HS2_PROXY_USER))
if (proxyUser == null) {
realUser
} else {
diff --git
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiRestFrontendService.scala
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiRestFrontendService.scala
index 5335ff61a..376c97934 100644
---
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiRestFrontendService.scala
+++
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiRestFrontendService.scala
@@ -219,9 +219,10 @@ class KyuubiRestFrontendService(override val serverable:
Serverable)
Option(AuthenticationFilter.getUserName).filter(_.nonEmpty).getOrElse("anonymous"))
}
- def getSessionUser(hs2ProxyUser: String): String = {
- val sessionConf = Option(hs2ProxyUser).filter(_.nonEmpty).map(proxyUser =>
- Map(KyuubiAuthenticationFactory.HS2_PROXY_USER ->
proxyUser)).getOrElse(Map())
+ def getSessionUser(proxyUser: String): String = {
+ // Internally, we use kyuubi.session.proxy.user to unify the key as
proxyUser
+ val sessionConf = Option(proxyUser).filter(_.nonEmpty).map(proxyUser =>
+ Map(PROXY_USER.key -> proxyUser)).getOrElse(Map())
getSessionUser(sessionConf)
}
@@ -250,12 +251,13 @@ class KyuubiRestFrontendService(override val serverable:
Serverable)
if (sessionConf == null) {
realUser
} else {
- sessionConf.get(KyuubiAuthenticationFactory.HS2_PROXY_USER).map {
proxyUser =>
- if (!isAdministrator(realUser)) {
- KyuubiAuthenticationFactory.verifyProxyAccess(realUser, proxyUser,
ipAddress, hadoopConf)
- }
- proxyUser
- }.getOrElse(realUser)
+ val proxyUser = sessionConf.getOrElse(
+ PROXY_USER.key,
+ sessionConf.getOrElse(KyuubiAuthenticationFactory.HS2_PROXY_USER,
realUser))
+ if (!proxyUser.equals(realUser) && !isAdministrator(realUser)) {
+ KyuubiAuthenticationFactory.verifyProxyAccess(realUser, proxyUser,
ipAddress, hadoopConf)
+ }
+ proxyUser
}
}
diff --git
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/AdminResource.scala
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/AdminResource.scala
index 0b017543e..ca35a1b6b 100644
---
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/AdminResource.scala
+++
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/AdminResource.scala
@@ -245,11 +245,13 @@ private[v1] class AdminResource extends ApiRequestContext
with Logging {
@QueryParam("type") engineType: String,
@QueryParam("sharelevel") shareLevel: String,
@QueryParam("subdomain") subdomain: String,
+ @QueryParam("proxyUser") kyuubiProxyUser: String,
@QueryParam("hive.server2.proxy.user") hs2ProxyUser: String): Response =
{
+ val activeProxyUser = Option(kyuubiProxyUser).getOrElse(hs2ProxyUser)
val userName = if (fe.isAdministrator(fe.getRealUser())) {
- Option(hs2ProxyUser).getOrElse(fe.getRealUser())
+ Option(activeProxyUser).getOrElse(fe.getRealUser())
} else {
- fe.getSessionUser(hs2ProxyUser)
+ fe.getSessionUser(activeProxyUser)
}
val engine = normalizeEngineInfo(userName, engineType, shareLevel,
subdomain, "default")
val engineSpace = calculateEngineSpace(engine)
@@ -283,11 +285,13 @@ private[v1] class AdminResource extends ApiRequestContext
with Logging {
@QueryParam("type") engineType: String,
@QueryParam("sharelevel") shareLevel: String,
@QueryParam("subdomain") subdomain: String,
+ @QueryParam("proxyUser") kyuubiProxyUser: String,
@QueryParam("hive.server2.proxy.user") hs2ProxyUser: String):
Seq[Engine] = {
+ val activeProxyUser = Option(kyuubiProxyUser).getOrElse(hs2ProxyUser)
val userName = if (fe.isAdministrator(fe.getRealUser())) {
- Option(hs2ProxyUser).getOrElse(fe.getRealUser())
+ Option(activeProxyUser).getOrElse(fe.getRealUser())
} else {
- fe.getSessionUser(hs2ProxyUser)
+ fe.getSessionUser(activeProxyUser)
}
val engine = normalizeEngineInfo(userName, engineType, shareLevel,
subdomain, "")
val engineSpace = calculateEngineSpace(engine)
diff --git
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
index 5e32016bf..0f2516459 100644
---
a/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
+++
b/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
@@ -450,6 +450,7 @@ private[v1] class BatchesResource extends ApiRequestContext
with Logging {
@Path("{batchId}")
def closeBatchSession(
@PathParam("batchId") batchId: String,
+ @QueryParam("proxyUser") kyuubiProxyUser: String,
@QueryParam("hive.server2.proxy.user") hs2ProxyUser: String):
CloseBatchResponse = {
def checkPermission(operator: String, owner: String): Unit = {
@@ -471,9 +472,10 @@ private[v1] class BatchesResource extends
ApiRequestContext with Logging {
(killed, message)
}
- val sessionHandle = formatSessionHandle(batchId)
- val userName = fe.getSessionUser(hs2ProxyUser)
+ val activeProxyUser = Option(kyuubiProxyUser).getOrElse(hs2ProxyUser)
+ val userName = fe.getSessionUser(activeProxyUser)
+ val sessionHandle = formatSessionHandle(batchId)
sessionManager.getBatchSession(sessionHandle).map { batchSession =>
checkPermission(userName, batchSession.user)
sessionManager.closeSession(batchSession.handle)
@@ -492,7 +494,7 @@ private[v1] class BatchesResource extends ApiRequestContext
with Logging {
} else if (batchV2Enabled(metadata.requestConf) &&
metadata.kyuubiInstance == null) {
// code goes here indicates metadata is outdated, recursively calls
itself to refresh
// the metadata
- closeBatchSession(batchId, hs2ProxyUser)
+ closeBatchSession(batchId, kyuubiProxyUser, hs2ProxyUser)
} else if (metadata.kyuubiInstance != fe.connectionUrl) {
info(s"Redirecting delete batch[$batchId] to
${metadata.kyuubiInstance}")
val internalRestClient =
getInternalRestClient(metadata.kyuubiInstance)
