Yes, that mostly sums it up!
As for (1), that part is solved by the current rewrite filtering we have in
place, as if there is a proxyUser defined in the request body, then the
actual value will be replaced with {$username}, which seems to be the
authenticated user in Knox.
For (2), I don't know for sure why this happens -- but one possibility is
that while the Livy response when you delete a missing session indicates
the body content is "application/json", the body itself is just a simple
string, which as per RFC-4627 is not valid for that content type, since it
has to be a serialized object or array; see:
$ curl -i -u : --negotiate http://myserver:8999/sessions/40 -X DELETE
...
HTTP/1.1 404 Not Found
Date: Fri, 01 Sep 2017 07:56:02 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 25
Connection: keep-alive
Server: Jetty(9.2.16.v20160414)
...
"Session '40' not found."
tors 31 aug. 2017 kl 17:42 skrev Jeffrey Rodriguez <[email protected]>:
> Hi Johan,
> So I think what you are asking is whether the authenticated
> “user" could be added to the json body, userProxy=user, if it is not there
> by the rewrite rules. Right? Also I think you are reporting that the
> status code from service is not returning correctly (500 instead of a 404),.
>
> Maybe a rewrite function that could return the authenticate “user” (if
> possible) could add the userProxy value.
>
> This brings another question, where today an authenticated user “foo”
> could send a userProxy “bar” so livy would impersonate “bar” instead of
> “foo”. Which doesn’t seem consistent to the identity provider which alway
> attaches doAs as the authenticated user.
>
> So I see:
>
> 1. One issue with the impersonation to livy since the mechanism is using
> the body proxyUser. So an authenticated user could potentially impersonate
> any other user.
> 2. a claim Status returned from livy not honor for some reason.
> 3. a feature to reflect the authenticated user into the “json” proxyUser
> value.
> 4. a feature to add LIVYSERVER since I don’t see the service in Knox code
> but I’ve seen it on HW blogs.
>
>
>
> Regards,
> Jeff
>
> On Aug 31, 2017, at 8:05 AM, Johan Wärlander <[email protected]> wrote:
>
> Hi, Jeffrey!
>
> Indeed we're able to get impersonation working - it's just that if a
> request is posted without information on who to impersonate, Livy will
> start the session as 'knox'.
>
> On tor 31 aug. 2017 17:02 Johan Wärlander <[email protected]> wrote:
>
>> Hmm, I see..
>>
>> On our part though, trying to get a proof of concept up and running, can
>> we deploy a rewrite function or dispatch as a separate "plugin" though, in
>> our existing Knox installation?
>>
>> We ran into another issue too, that a Livy "valid" 404 response (trying
>> to DELETE a session that's already gone) turns into a 500 from Knox, so I
>> suspect we need to handle that similarly..
>>
>> On tor 31 aug. 2017 16:31 larry mccay <[email protected]> wrote:
>>
>>> I don't believe there is any way to inject that currently it will likely
>>> require a rewrite function or specialized dispatch.
>>>
>>> Livy needs to support the proper trusted proxy pattern used by other
>>> services.
>>>
>>>
>>> On Aug 31, 2017 6:23 AM, "Johan Wärlander" <[email protected]> wrote:
>>>
>>> We've been able to set up Knox to route Livy requests, and it's working
>>> mostly as expected; when creating a new Spark session via a POST request
>>> with a JSON body, Knox has a rewrite rule that modifies the "proxyUser" in
>>> the JSON body, making sure you can only act as the user you authenticated
>>> to Knox with:
>>>
>>> From service.xml:
>>>
>>> <route path="/livy/v1/sessions">
>>> <rewrite apply="LIVYSERVER/livy/addusername/inbound"
>>> to="request.body"/>
>>> </route>
>>>
>>> From rewrite.xml:
>>>
>>> <filter name="LIVYSERVER/livy/addusername/inbound">
>>> <content type="*/json">
>>> <apply path="$.proxyUser" rule="LIVYSERVER/livy/user-name"/>
>>> </content>
>>> </filter>
>>>
>>> Example of a request:
>>>
>>> curl -u johwar -v -s --data '{"proxyUser":"foobar","kind": "pyspark"}'
>>> -H "Content-Type: application/json"
>>> https://myknoxserver/gateway/default/livy/v1/sessions
>>>
>>> This works fine, and "foobar" above gets replaced with "johwar" before
>>> the request reaches Livy.
>>>
>>> However, if you *don't* pass "proxyUser" at all in the request, this
>>> rule doesn't seem to *add* the element, so it ends up as "knox" on the Livy
>>> end; it's probably defaulting to the Kerberos-authenticated user, which is
>>> of course "knox".
>>>
>>> Is there a way to make sure that "proxyUser" is modified if it exists
>>> (as above) AND added if it's missing?
>>>
>>> NOTE: For our full config, we followed the example below:
>>>
>>>
>>> https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html
>>>
>>>
>>>
>
