Wonderful, Adam! Glad it was helpful.
On Tue, Feb 23, 2016 at 6:05 AM, Adam Davidson < [email protected]> wrote: > Hi Larry, > > that's very helpful, thank you! I think I've cracked it now; I now have my > service checking that the remote user is allowed to impersonate the doAs > user. In this case the remote user should always be Knox. > > Thanks for the help, it's saved me a lot of grief! > Best Regards, > Adam > > On Mon, 22 Feb 2016 at 19:25 larry mccay <[email protected]> wrote: > >> Yes, so you have configured HBase's trusted proxy users to include your >> service. >> When users are making calls through Knox to HBase the knox user must also >> be configured to be allowed to impersonate users via doAs. >> >> What you need to add is the same enforcement of such configuration for >> other apps to impersonate users to you via doAs. >> You must be a trusted proxy server to Knox as well as a trusted proxy >> client to HBase. >> >> Then you will add similar config for Knox to be allowed to impersonate >> users via doAs: >> >> hadoop.proxyuser.knox.groups = * >> hadoop.proxyuser.knox.hosts = * >> >> trusted trusted >> proxy proxy >> | | >> client->knox->adamservice->hbase >> >> Make sense? >> >> >> On Mon, Feb 22, 2016 at 12:59 PM, Adam Davidson < >> [email protected]> wrote: >> >>> Hi Larry, >>> >>> thanks for getting back to me so soon. We did indeed add proxy user >>> support, but that was to allow our service user (that runs the REST >>> application) to impersonate anyone using accessing HBase, identified via >>> the doAs parameter; We have the following properties in our hbase-site.xml- >>> >>> hadoop.proxyuser.<service user>.groups = * >>> hadoop.proxyuser.<service user>.hosts = * >>> >>> The above config lets our application perform 'doAs' requests on behalf >>> of any user on the other side of the Knox gateway. Is what your are >>> suggesting that we change the 'hosts' property above to only allow requests >>> from the Knox gateway hosts? >>> >>> Thanks for everything so far, >>> Adam >>> >>> On Mon, 22 Feb 2016 at 17:40 larry mccay <[email protected]> wrote: >>> >>>> Hi Adam - >>>> >>>> Your concern is exactly what the trusted proxy or proxyuser support >>>> that I was describing is for. >>>> Based on your description, it seems that you have requests going >>>> through Knox and successfully being handled by your service. >>>> I assume that you didn't add the proxyuser config to allow this to work >>>> - otherwise, you would already understand what was needed. >>>> >>>> I don't recall in detail what needs to be done in addition to the >>>> hadoop auth authentication filter in order to add the proxyuser support but >>>> you can check this JIRA for an example: >>>> https://issues.apache.org/jira/browse/HADOOP-10698 >>>> >>>> I remember that being done for KMS and it requiring some refactoring in >>>> order to make the proxyuser support more easily consumable from a common >>>> code perspective. I will try and find some time to dig into it and provide >>>> some additional guidance but that JIRA is exactly where I would start. So I >>>> suggest that you do the same. :) >>>> >>>> Hope that is helpful. >>>> >>>> --larry >>>> >>>> >>>> On Mon, Feb 22, 2016 at 10:22 AM, Adam Davidson < >>>> [email protected]> wrote: >>>> >>>>> Hi Larry, >>>>> >>>>> thank you for your response; I believe I have implemented something >>>>> that should work now; can I just check my understanding? >>>>> >>>>> When a user is sending a REST request through Knox to our service, >>>>> they are authenticated to Knox by the authentication service outside the >>>>> cluster. Knox itself is authenticated via Kerberos. Knox adds the doAs >>>>> query parameter to represent the original user making the request. Our >>>>> data >>>>> service then receives the forwarded REST request from Knox. >>>>> >>>>> At this point, the hadoop-auth 'AuthenticationFilter' has been loaded >>>>> by the service and ensures that the 'knox' user has a valid Kerberos >>>>> ticket. The filter also requires the HTTP principal for the service host >>>>> is >>>>> logged in via keytab to Kerberos (I'm not clear why this part is >>>>> necessary). The service itself also has a principal that is authenticated >>>>> in Kerberos via keytab, so that it can access Kerberised Hadoop resources >>>>> on behalf of the 'doAs' user. >>>>> >>>>> This appears to work in that a user must exist in Kerberos and >>>>> authenticate to Knox before being able to make REST calls to the service; >>>>> however as far as I can tell it still breaks down in the following case. A >>>>> user may be logged into a node of the cluster behind the gateway and have >>>>> a >>>>> Kerberos ticket. They could supply any 'doAs' user they wish and be >>>>> granted >>>>> access according to the ACL settings for that user i.e. there is no >>>>> authentication that the 'doAs' user matches the user making the original >>>>> request (inside or outside of the cluster, as Knox doesn't pass on the >>>>> Kerberos ticket). Does that make sense to you? My view then would be that >>>>> we either have to disallow any requests not made by Knox, or somehow have >>>>> a >>>>> ticket to match the 'doAs' user. >>>>> >>>>> I appreciate I'm moving a bit outside what Knox covers here but I'd be >>>>> grateful to hear your thoughts. >>>>> >>>>> Best Regards. >>>>> Adam >>>>> >>>>> On Wed, 17 Feb 2016 at 18:21 larry mccay <[email protected]> wrote: >>>>> >>>>>> Hi Adam - >>>>>> >>>>>> Let me articulate the expected pattern for such a deployment - so >>>>>> that we are both on the same page... >>>>>> >>>>>> * Users authenticate to Knox via whatever configured >>>>>> authentication/federation provider mechanism >>>>>> * Knox authenticates to the proxied services via kerberos (in secure >>>>>> clusters) as Knox and asserts the identity via doAs >>>>>> * The proxied service needs to ensure that only "trusted" proxies >>>>>> issue requests on behalf of end users via doAs >>>>>> * Trust is generally determined via kerberos authentication but could >>>>>> be done in other ways as well but would likely require work on the Knox >>>>>> side to provide something else - like mutual authentication via SSL, etc. >>>>>> >>>>>> Current support for authenticating users with kerberos via the hadoop >>>>>> auth module assumes the above trusted proxy pattern as well. >>>>>> >>>>>> There may be some way to provide a passthrough of the kerberos header >>>>>> but this may not be as straightforward as other HTTP headers. SPNEGO >>>>>> authentication is expected between Knox and the proxied service therefore >>>>>> passing the end user's ticket may cause difficulties in establishing the >>>>>> connection to the service. >>>>>> >>>>>> Generally speaking, I would encourage you to adopt the above >>>>>> described pattern rather than introduce some new pattern in a secure >>>>>> environment. >>>>>> >>>>>> HTH, >>>>>> >>>>>> --larry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Feb 17, 2016 at 7:03 AM, Adam Davidson < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> we are using Knox on a Kerberized cluster to secure access a RESTful >>>>>>> Java application. We know the user making the request is authenticated >>>>>>> against Kerberos in Knox, but is the resulting ticket then passed on >>>>>>> with >>>>>>> the request? We believe there is a security issue in our app where we do >>>>>>> not authenticate the 'doAs' user supplied by Knox. From inside the >>>>>>> cluster, >>>>>>> a malicious actor may in theory supply any value for this parameter in >>>>>>> their request and be granted the same access as that user. >>>>>>> >>>>>>> So, is the Kerberos ticket passed on that our service could then >>>>>>> authenticate? Or do we just assume that the Kerberos authentication is a >>>>>>> one-time job that happens in the Knox gateway? >>>>>>> >>>>>>> Best Regards, >>>>>>> Adam >>>>>>> >>>>>>> *We're hiring!* >>>>>>> Please check out our current positions *here* >>>>>>> <https://www.bigdatapartnership.com/careers/>*.* >>>>>>> ------------------------------ >>>>>>> >>>>>>> *NOTICE AND DISCLAIMER* >>>>>>> >>>>>>> This email (including attachments) is confidential. If you are not >>>>>>> the intended recipient, notify the sender immediately, delete this email >>>>>>> from your system and do not disclose or use for any purpose. >>>>>>> >>>>>>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. >>>>>>> United Kingdom >>>>>>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. >>>>>>> United Kingdom >>>>>>> Big Data Partnership Limited is a company registered in England & >>>>>>> Wales with Company No 7904824 >>>>>>> >>>>>> >>>>>> >>>>> *We're hiring!* >>>>> Please check out our current positions *here* >>>>> <https://www.bigdatapartnership.com/careers/>*.* >>>>> ------------------------------ >>>>> >>>>> *NOTICE AND DISCLAIMER* >>>>> >>>>> This email (including attachments) is confidential. If you are not the >>>>> intended recipient, notify the sender immediately, delete this email from >>>>> your system and do not disclose or use for any purpose. >>>>> >>>>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United >>>>> Kingdom >>>>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. >>>>> United Kingdom >>>>> Big Data Partnership Limited is a company registered in England & >>>>> Wales with Company No 7904824 >>>>> >>>> >>>> >>> *We're hiring!* >>> Please check out our current positions *here* >>> <https://www.bigdatapartnership.com/careers/>*.* >>> ------------------------------ >>> >>> *NOTICE AND DISCLAIMER* >>> >>> This email (including attachments) is confidential. If you are not the >>> intended recipient, notify the sender immediately, delete this email from >>> your system and do not disclose or use for any purpose. >>> >>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United >>> Kingdom >>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. >>> United Kingdom >>> Big Data Partnership Limited is a company registered in England & Wales >>> with Company No 7904824 >>> >> >> > *We're hiring!* > Please check out our current positions *here* > <https://www.bigdatapartnership.com/careers/>*.* > ------------------------------ > > *NOTICE AND DISCLAIMER* > > This email (including attachments) is confidential. If you are not the > intended recipient, notify the sender immediately, delete this email from > your system and do not disclose or use for any purpose. > > Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United > Kingdom > Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United > Kingdom > Big Data Partnership Limited is a company registered in England & Wales > with Company No 7904824 >
