Local development and testing w/ kerberos
Looking for suggestions on local development and testing with kerberos, i have followed below steps and able to do kinit , klist and everything seems fine., however my Spnego is not working tried all browsers, cleaning up users.xml, authorizations, archive and flow.xml.gz. i can see the headers for negotiating in my browser ( network tab). https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication Here is the post with more details. https://community.hortonworks.com/questions/243723/kerberos-spnego-authentication.html Also do i have to install KDC server on my production hosts to make kerberos work? or all i need is a keytab and krb5.conf files? Any help would be great. -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: Local development and testing w/ kerberos
Thanks. Got that working and was able to login as mthom...@nifi.apache.org. On Wed, Oct 24, 2018 at 11:24 AM Bryan Bende wrote: > I think all your kerberos/KDC stuff is fine, you just need to add > mthom...@nifi.apache.org to the user-group-provider. > > My post was old before we had separated authorizer into > user-group-provider and access-policy-provider. > On Wed, Oct 24, 2018 at 11:18 AM Mike Thomsen > wrote: > > > > Alright, I think I'm pretty close here. I followed all of those steps, > > except I changed bbende to mthomsen. > > > > * I can run kinit mthom...@nifi.apache.org and it works. > > * I can run klist and see the expected output. > > > > When I bring up NiFi, I get the following (trimmed for brevity): > > > > Caused by: > > org.apache.nifi.authorization.exception.AuthorizerCreationException: > > org.apache.nifi.authorization.exception.AuthorizerCreationException: > Unable > > to locate initial admin mthom...@nifi.apache.org to seed policies > > at > > > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > > org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54) > > at com.sun.proxy.$Proxy76.onConfigured(Unknown Source) > > at > > > org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152) > > at > > > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) > > ... 96 common frames omitted > > Caused by: > > org.apache.nifi.authorization.exception.AuthorizerCreationException: > Unable > > to locate initial admin mthom...@nifi.apache.org to seed policies > > at > > > org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598) > > at > > > org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541) > > at > > > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254) > > ... 104 common frames omitted > > > > I double-checked the paths to krb5.conf and the keytab and they're both > > pointing to /tmp/docker-kdc > > > > Any ideas? > > > > Thanks, > > > > Mike > > > > > > On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen > > wrote: > > > > > Awesome, thanks Bryan! I'm halfway through that (got klist view) and > it's > > > working great so far. > > > > > > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende wrote: > > > > > >> There is a docker-kdc project that is easy to use: > > >> > > >> > > >> > https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication > > >> > > >> It was made before docker for mac was good/popular and it previously > > >> relied on boot2docker, but I made the following modification to not > > >> use boot2docker > > >> > > >> docker-kdc$ git diff > > >> diff --git a/kdc b/kdc > > >> index 9410fc5..0a887e1 100755 > > >> --- a/kdc > > >> +++ b/kdc > > >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' > > >> GET_KDC_HOST="echo $KDC_NATHOST" > > >> > > >> # Adjust container in case of OSX. > > >> -if [[ $OSTYPE =~ darwin.+ ]]; then > > >> - CONTAINER='boot2docker' > > >> - GET_KDC_HOST='boot2docker ip' > > >> -fi > > >> +#if [[ $OSTYPE =~ darwin.+ ]]; then > > >> +# CONTAINER='boot2docker' > > >> +# GET_KDC_HOST='boot2docker ip' > > >> +#fi > > >> > > >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen > > >> wrote: > > >> > > > >> > Looking for suggestions on local development and testing with > kerberos. > > >> We > > >> > have a kerberized cluster set up in an AWS instance, but it's more > for > > >> UAT > > >> > than development. Anyone have any suggestions/experience, say, > setting > > >> up a > > >> > Mac or Linux box for developing and testing like this? > > >> > > > >> > Thanks, > > >> > > > >> > Mike > > >> > > > >
Re: Local development and testing w/ kerberos
I think all your kerberos/KDC stuff is fine, you just need to add mthom...@nifi.apache.org to the user-group-provider. My post was old before we had separated authorizer into user-group-provider and access-policy-provider. On Wed, Oct 24, 2018 at 11:18 AM Mike Thomsen wrote: > > Alright, I think I'm pretty close here. I followed all of those steps, > except I changed bbende to mthomsen. > > * I can run kinit mthom...@nifi.apache.org and it works. > * I can run klist and see the expected output. > > When I bring up NiFi, I get the following (trimmed for brevity): > > Caused by: > org.apache.nifi.authorization.exception.AuthorizerCreationException: > org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable > to locate initial admin mthom...@nifi.apache.org to seed policies > at > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54) > at com.sun.proxy.$Proxy76.onConfigured(Unknown Source) > at > org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152) > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) > ... 96 common frames omitted > Caused by: > org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable > to locate initial admin mthom...@nifi.apache.org to seed policies > at > org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598) > at > org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541) > at > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254) > ... 104 common frames omitted > > I double-checked the paths to krb5.conf and the keytab and they're both > pointing to /tmp/docker-kdc > > Any ideas? > > Thanks, > > Mike > > > On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen > wrote: > > > Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's > > working great so far. > > > > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende wrote: > > > >> There is a docker-kdc project that is easy to use: > >> > >> > >> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication > >> > >> It was made before docker for mac was good/popular and it previously > >> relied on boot2docker, but I made the following modification to not > >> use boot2docker > >> > >> docker-kdc$ git diff > >> diff --git a/kdc b/kdc > >> index 9410fc5..0a887e1 100755 > >> --- a/kdc > >> +++ b/kdc > >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' > >> GET_KDC_HOST="echo $KDC_NATHOST" > >> > >> # Adjust container in case of OSX. > >> -if [[ $OSTYPE =~ darwin.+ ]]; then > >> - CONTAINER='boot2docker' > >> - GET_KDC_HOST='boot2docker ip' > >> -fi > >> +#if [[ $OSTYPE =~ darwin.+ ]]; then > >> +# CONTAINER='boot2docker' > >> +# GET_KDC_HOST='boot2docker ip' > >> +#fi > >> > >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen > >> wrote: > >> > > >> > Looking for suggestions on local development and testing with kerberos. > >> We > >> > have a kerberized cluster set up in an AWS instance, but it's more for > >> UAT > >> > than development. Anyone have any suggestions/experience, say, setting > >> up a > >> > Mac or Linux box for developing and testing like this? > >> > > >> > Thanks, > >> > > >> > Mike > >> > >
Re: Local development and testing w/ kerberos
Can you share the authorizers.xml? I guess something wrong with the CN that’s mentioned there. - Sivaprasanna On Wed, 24 Oct 2018 at 8:48 PM, Mike Thomsen wrote: > Alright, I think I'm pretty close here. I followed all of those steps, > except I changed bbende to mthomsen. > > * I can run kinit mthom...@nifi.apache.org and it works. > * I can run klist and see the expected output. > > When I bring up NiFi, I get the following (trimmed for brevity): > > Caused by: > org.apache.nifi.authorization.exception.AuthorizerCreationException: > org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable > to locate initial admin mthom...@nifi.apache.org to seed policies > at > > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > > org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54) > at com.sun.proxy.$Proxy76.onConfigured(Unknown Source) > at > > org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152) > at > > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) > ... 96 common frames omitted > Caused by: > org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable > to locate initial admin mthom...@nifi.apache.org to seed policies > at > > org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598) > at > > org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541) > at > > org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254) > ... 104 common frames omitted > > I double-checked the paths to krb5.conf and the keytab and they're both > pointing to /tmp/docker-kdc > > Any ideas? > > Thanks, > > Mike > > > On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen > wrote: > > > Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's > > working great so far. > > > > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende wrote: > > > >> There is a docker-kdc project that is easy to use: > >> > >> > >> > https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication > >> > >> It was made before docker for mac was good/popular and it previously > >> relied on boot2docker, but I made the following modification to not > >> use boot2docker > >> > >> docker-kdc$ git diff > >> diff --git a/kdc b/kdc > >> index 9410fc5..0a887e1 100755 > >> --- a/kdc > >> +++ b/kdc > >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' > >> GET_KDC_HOST="echo $KDC_NATHOST" > >> > >> # Adjust container in case of OSX. > >> -if [[ $OSTYPE =~ darwin.+ ]]; then > >> - CONTAINER='boot2docker' > >> - GET_KDC_HOST='boot2docker ip' > >> -fi > >> +#if [[ $OSTYPE =~ darwin.+ ]]; then > >> +# CONTAINER='boot2docker' > >> +# GET_KDC_HOST='boot2docker ip' > >> +#fi > >> > >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen > >> wrote: > >> > > >> > Looking for suggestions on local development and testing with > kerberos. > >> We > >> > have a kerberized cluster set up in an AWS instance, but it's more for > >> UAT > >> > than development. Anyone have any suggestions/experience, say, setting > >> up a > >> > Mac or Linux box for developing and testing like this? > >> > > >> > Thanks, > >> > > >> > Mike > >> > > >
Re: Local development and testing w/ kerberos
Alright, I think I'm pretty close here. I followed all of those steps, except I changed bbende to mthomsen. * I can run kinit mthom...@nifi.apache.org and it works. * I can run klist and see the expected output. When I bring up NiFi, I get the following (trimmed for brevity): Caused by: org.apache.nifi.authorization.exception.AuthorizerCreationException: org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable to locate initial admin mthom...@nifi.apache.org to seed policies at org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54) at com.sun.proxy.$Proxy76.onConfigured(Unknown Source) at org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) ... 96 common frames omitted Caused by: org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable to locate initial admin mthom...@nifi.apache.org to seed policies at org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598) at org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541) at org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254) ... 104 common frames omitted I double-checked the paths to krb5.conf and the keytab and they're both pointing to /tmp/docker-kdc Any ideas? Thanks, Mike On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen wrote: > Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's > working great so far. > > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende wrote: > >> There is a docker-kdc project that is easy to use: >> >> >> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication >> >> It was made before docker for mac was good/popular and it previously >> relied on boot2docker, but I made the following modification to not >> use boot2docker >> >> docker-kdc$ git diff >> diff --git a/kdc b/kdc >> index 9410fc5..0a887e1 100755 >> --- a/kdc >> +++ b/kdc >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' >> GET_KDC_HOST="echo $KDC_NATHOST" >> >> # Adjust container in case of OSX. >> -if [[ $OSTYPE =~ darwin.+ ]]; then >> - CONTAINER='boot2docker' >> - GET_KDC_HOST='boot2docker ip' >> -fi >> +#if [[ $OSTYPE =~ darwin.+ ]]; then >> +# CONTAINER='boot2docker' >> +# GET_KDC_HOST='boot2docker ip' >> +#fi >> >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen >> wrote: >> > >> > Looking for suggestions on local development and testing with kerberos. >> We >> > have a kerberized cluster set up in an AWS instance, but it's more for >> UAT >> > than development. Anyone have any suggestions/experience, say, setting >> up a >> > Mac or Linux box for developing and testing like this? >> > >> > Thanks, >> > >> > Mike >> >
Re: Local development and testing w/ kerberos
Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's working great so far. On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende wrote: > There is a docker-kdc project that is easy to use: > > > https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication > > It was made before docker for mac was good/popular and it previously > relied on boot2docker, but I made the following modification to not > use boot2docker > > docker-kdc$ git diff > diff --git a/kdc b/kdc > index 9410fc5..0a887e1 100755 > --- a/kdc > +++ b/kdc > @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' > GET_KDC_HOST="echo $KDC_NATHOST" > > # Adjust container in case of OSX. > -if [[ $OSTYPE =~ darwin.+ ]]; then > - CONTAINER='boot2docker' > - GET_KDC_HOST='boot2docker ip' > -fi > +#if [[ $OSTYPE =~ darwin.+ ]]; then > +# CONTAINER='boot2docker' > +# GET_KDC_HOST='boot2docker ip' > +#fi > > On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen > wrote: > > > > Looking for suggestions on local development and testing with kerberos. > We > > have a kerberized cluster set up in an AWS instance, but it's more for > UAT > > than development. Anyone have any suggestions/experience, say, setting > up a > > Mac or Linux box for developing and testing like this? > > > > Thanks, > > > > Mike >
Re: Local development and testing w/ kerberos
There is a docker-kdc project that is easy to use: https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication It was made before docker for mac was good/popular and it previously relied on boot2docker, but I made the following modification to not use boot2docker docker-kdc$ git diff diff --git a/kdc b/kdc index 9410fc5..0a887e1 100755 --- a/kdc +++ b/kdc @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm' GET_KDC_HOST="echo $KDC_NATHOST" # Adjust container in case of OSX. -if [[ $OSTYPE =~ darwin.+ ]]; then - CONTAINER='boot2docker' - GET_KDC_HOST='boot2docker ip' -fi +#if [[ $OSTYPE =~ darwin.+ ]]; then +# CONTAINER='boot2docker' +# GET_KDC_HOST='boot2docker ip' +#fi On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen wrote: > > Looking for suggestions on local development and testing with kerberos. We > have a kerberized cluster set up in an AWS instance, but it's more for UAT > than development. Anyone have any suggestions/experience, say, setting up a > Mac or Linux box for developing and testing like this? > > Thanks, > > Mike
Local development and testing w/ kerberos
Looking for suggestions on local development and testing with kerberos. We have a kerberized cluster set up in an AWS instance, but it's more for UAT than development. Anyone have any suggestions/experience, say, setting up a Mac or Linux box for developing and testing like this? Thanks, Mike
