[
https://issues.apache.org/jira/browse/KNOX-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802446#comment-16802446
]
Bhanu Teja commented on KNOX-1835:
----------------------------------
{code:java}
Test 1 - user1 sends KERNEL_USERNAME=user2
curl -i -k -X POST -H "Authorization: $TOKEN" -H 'Content-Type:
application/json' $JEG_URL/api/kernels -d '{"name" :
"spark_python_yarn_cluster", "env": { "KERNEL_USERNAME": "user2" ,
"KERNEL_EXTRA_SPARK_OPTS" : "--archives
hdfs:///user/dsxhi/environments/x86_glibc2_17.tar.gz --conf
spark.yarn.appMasterEnv.PYSPARK_PYTHON=x86_glibc2_17.tar.gz/x86_glibc2_17/bin/python2.7
--conf spark.executor.memory=2g"} }'
HTTP/1.1 201 Created
Date: Wed, 27 Mar 2019 03:52:22 GMT
Server: TornadoServer/5.1.1
Location: /api/kernels/bddd7d0c-1b76-4425-8c92-10d6a17fcca3
Date: Wed, 27 Mar 2019 03:52:22 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 180
{"connections": 0, "last_activity": "2019-03-27T03:52:35.467814Z",
"execution_state": "starting", "id": "bddd7d0c-1b76-4425-8c92-10d6a17fcca3",
"name": "spark_python_yarn_cluster"}
19/03/26 20:52:29 INFO Client:
client token: N/A
diagnostics: [Tue Mar 26 20:52:29 -0700 2019] Scheduler has assigned a
container for AM, waiting for AM container to be launched
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1553658748834
final status: UNDEFINED
tracking URL:
http://shad1.fyre.ibm.com:8088/proxy/application_1553550168370_0051/
user: user1
Test 2 : Empty TOKEN
curl -i -k -X POST -H "Authorization: $TOKEN1" -H 'Content-Type:
application/json' $JEG_URL/api/kernels -d '{"name" :
"spark_python_yarn_cluster", "env": { "KERNEL_USERNAME": "user1" ,
"KERNEL_EXTRA_SPARK_OPTS" : "--archives
hdfs:///user/dsxhi/environments/x86_glibc2_17.tar.gz --conf
spark.yarn.appMasterEnv.PYSPARK_PYTHON=x86_glibc2_17.tar.gz/x86_glibc2_17/bin/python2.7
--conf spark.executor.memory=2g"} }'
HTTP/1.1 401 Unauthorized
Date: Wed, 27 Mar 2019 03:55:28 GMT
Content-Type: text/html; charset=ISO-8859-1
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 334
Server: Jetty(9.2.22.v20170606)
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /gateway/jalv-wsl123-master-1/jkg/api/kernels. Reason:
<pre> Unauthorized</pre></p><hr><i><small>Powered by
Jetty://</small></i><hr/>
</body>
</html>
Test 3: - user1 sends KERNEL_USERNAME is user1
curl -i -k -X POST -H "Authorization: $TOKEN" -H 'Content-Type:
application/json' $JEG_URL/api/kernels -d '{"name" :
"spark_python_yarn_cluster", "env": { "KERNEL_USERNAME": "user1" ,
"KERNEL_EXTRA_SPARK_OPTS" : "--archives
hdfs:///user/dsxhi/environments/x86_glibc2_17.tar.gz --conf
spark.yarn.appMasterEnv.PYSPARK_PYTHON=x86_glibc2_17.tar.gz/x86_glibc2_17/bin/python2.7
--conf spark.driver.memory=2g --conf spark.executor.memory=1g"} }'
HTTP/1.1 201 Created
Date: Wed, 27 Mar 2019 04:06:46 GMT
Server: TornadoServer/5.1.1
Location: /api/kernels/11e2099c-1838-44f9-a8fd-de3734e27c74
Date: Wed, 27 Mar 2019 04:06:46 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 180
{"connections": 0, "last_activity": "2019-03-27T04:07:00.196847Z",
"execution_state": "starting", "id": "11e2099c-1838-44f9-a8fd-de3734e27c74",
"name": "spark_python_yarn_cluster"}
19/03/26 21:06:53 INFO Client:
client token: N/A
diagnostics: [Tue Mar 26 21:06:53 -0700 2019] Application is
Activated, waiting for resources to be assigned for AM. Details : AM Partition
= <DEFAULT_PARTITION> ; Partition Resource = <memory:27136, vCores:18> ;
Queue's Absolute capacity = 100.0 % ; Queue's Absolute used capacity =
33.962265 % ; Queue's Absolute max capacity = 100.0 % ;
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1553659612969
final status: UNDEFINED
tracking URL:
http://shad1.fyre.ibm.com:8088/proxy/application_1553550168370_0055/
user: user1
Test 4: - user1 sends KERNEL_USERNAME is empty
curl -i -k -X POST -H "Authorization: $TOKEN" -H 'Content-Type:
application/json' $JEG_URL/api/kernels -d '{"name" :
"spark_python_yarn_cluster", "env": { "KERNEL_EXTRA_SPARK_OPTS" : "--archives
hdfs:///user/dsxhi/environments/x86_glibc2_17.tar.gz --conf
spark.yarn.appMasterEnv.PYSPARK_PYTHON=x86_glibc2_17.tar.gz/x86_glibc2_17/bin/python2.7
--conf spark.driver.memory=2g --conf spark.executor.memory=1g"} }'
HTTP/1.1 201 Created
Date: Wed, 27 Mar 2019 04:08:01 GMT
Server: TornadoServer/5.1.1
Location: /api/kernels/5a043b45-c94e-40f2-9930-0ca48f3cd4cb
Date: Wed, 27 Mar 2019 04:08:01 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 180
{"connections": 0, "last_activity": "2019-03-27T04:08:15.195738Z",
"execution_state": "starting", "id": "5a043b45-c94e-40f2-9930-0ca48f3cd4cb",
"name": "spark_python_yarn_cluster"}
19/03/26 21:08:08 INFO Client:
client token: N/A
diagnostics: [Tue Mar 26 21:08:07 -0700 2019] Application is
Activated, waiting for resources to be assigned for AM. Details : AM Partition
= <DEFAULT_PARTITION> ; Partition Resource = <memory:27136, vCores:18> ;
Queue's Absolute capacity = 100.0 % ; Queue's Absolute used capacity = 56.60377
% ; Queue's Absolute max capacity = 100.0 % ;
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1553659687854
final status: UNDEFINED
tracking URL:
http://shad1.fyre.ibm.com:8088/proxy/application_1553550168370_0056/
user: user1
{code}
> Jupyter Enterprise Gateway Dispatch Handler
> -------------------------------------------
>
> Key: KNOX-1835
> URL: https://issues.apache.org/jira/browse/KNOX-1835
> Project: Apache Knox
> Issue Type: New Feature
> Reporter: Jesus Alvarez
> Priority: Major
> Attachments: KNOX-1835.patch
>
>
> Jupyter Enterprise Gateway
> [https://jupyter-enterprise-gateway.readthedocs.io/|https://jupyter-enterprise-gateway.readthedocs.io/en/latest/]
> is built directly on Jupyter Kernel Gateway, for which there are service
> definitions available from [https://issues.apache.org/jira/browse/KNOX-976]
>
> In the same manner that Apache Livy uses "proxyUser", JEG / JKG leverage a
> "KERNEL_USERNAME"
> [https://jupyter-enterprise-gateway.readthedocs.io/en/latest/getting-started-security.html].
>
> For requests which are Authenticated via Knox, Knox could take a similar
> approach to that in https://jira.apache.org/jira/browse/KNOX-1098 , and
> ensure this is set to the Authenticated user, whether the JEG client request
> has provided it in the payload or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
