Hello again.

As a follow-up, I tried some further troubleshooting on two fresh
virtual machines. The setup process was as follows:

[both]
- install CentOS via kickstart
- change hostname
- ipa-client-install
- ipa-client-automount
- install @Network File System Client / @File and Storage Server
- useradd --system --no-create-home git
- add static mapping in /etc/idmapd.conf (see other reply)
- add service principals in ipa:
 * git/test00.client.$domain@$REALM
 * nfs/zfs0.storage.$domain@$REALM

[server zfs0.storage.$domain]
- kinit -k
- ipa-getkeytab -p nfs/$HOSTNAME -k /etc/krb5.keytab
- create a directory structure for exporting:
 • root @zfs0 ~ # ls -la /media/testenv/
 total 0
 drwxr-sr-x. 1 admin kerberos 18 Jul 19 20:56 .
 drwxr-xr-x. 1 root  root     14 Jul 19 00:06 ..
 drwx--S---. 1 git   kerberos  0 Jul 19 00:06 git
 drwxrwsrwx. 1 root  kerberos  8 Jul 19 19:02 public
- export with '/media *(rw,sec=krb5p,crossmnt,fsid=0,sync)'
- add firewalld service nfs permanently
- reboot

[client test00.client.$domain]
- kinit -k
- ipa-getkeytab -p git/$HOSTNAME -k /var/lib/gssproxy/clients/$(id -u
git).keytab
- fstab: zfs0.storage.$domain:/  /media  nfs4  defaults,proto=tcp 0 0
- reboot


Then, logging in to the client and switching users with 'su - git'
allowed me to navigate in /media, yet again I was denied access to the
folder owned by git:

• git @test00 /media/testenv $ ls git
ls: cannot open directory git: Permission denied

Gssproxy surely seems to use the client keytab, a cache is created:

• root @test00 / # klist -kt /var/lib/gssproxy/clients/995.keytab
Keytab name: FILE:/var/lib/gssproxy/clients/995.keytab
KVNO Timestamp         Principal
---- -----------------
--------------------------------------------------------
   1 18/07/17 23:32:43 git/test00.client.$domain@$REALM
   1 18/07/17 23:32:43 git/test00.client.$domain@$REALM
• root @test00 / # ll /var/lib/gssproxy/clients/
total 8
-rw-------. 1 root root  198 Jul 18 23:32 995.keytab
-rw-------. 1 root root 1815 Jul 19 19:02 krb5cc_995


After adding some verbosity flags ("-vvv" in /etc/sysconfig/nfs and
"Verbosity = 5" in /etc/idmapd.conf) I recorded logs while I was
(re)mounting the nfs share and navigating through folders. The static
idmap mapping seems to work:

 59: nfsidmap[9406]: key: 0x1802b55 type: uid value:
git/test00.client.$domain@$REALM timeout 600
 60: nfsidmap[9406]: nfs4_name_to_uid: calling static->name_to_uid
 61: nfsidmap[9406]: static_getpwnam: name
'git/test00.client.$domain@$REALM' mapped to 'git'
 62: nfsidmap[9406]: nfs4_name_to_uid: static->name_to_uid returned 0
 63: nfsidmap[9406]: nfs4_name_to_uid: final return value is 0

.. but as stated above I am still denied access. The logs are attached,
because otherwise Thunderbird rips the formatting apart. I hope that
this clarifies my situation a little and allows for some reproduceability.

Where could I further increase verbosity to see what principal/username
is actually transmitted to the NFS server when I am denied access? Is
there a NFS4 specific mailinglist where I could ask?


Greetings from Germany,
Anton

On 17/07/17 15:02, Anton Semjonov via FreeIPA-users wrote:
> Hello everyone!
> 
> I am trying to setup an NFS export with sec=krb5p on one machine and
> make it accessible to a system user ('git' in this case) on another. I'd
> like to put GitLab backups on my ZFS array via NFS, to be more specific.
> 
> All machines in my little homelab are based on CentOS 7.3.1611 and I use
> two replicated FreeIPA servers with what I believe to be the latest
> release available on CentOS: 4.4.0. Both the storage server and the
> GitLab server are enrolled hosts in my realm.
> 
> After enrolling both machines with ipa-client-install and installing
> @File\ and\ Storage\ Server on one and @Network\ File\ System\ Client on
> the other, I ran ipa-client-automount on both, as I read somewhere that
> it sets up neccessary configuration files for identity mapping?
> 
> I also found a thread on this mailinglist, about a usecase of Apache
> accessing a /var/www directory via kerberized NFS. I believe my usecase
> is very similar and I feel like I am very close to a solution. But I
> just don't understand where things go wrong:
> 
> In this particular case the 'git' user on both machines has different
> UIDs. It was created during the installation of GitLab on the client but
> the UID was already occupied by 'softhsm private keys owner' on the
> server. Thus I created a system user manually, which has a different UID
> though. For the sake of troubleshooting I also tried all the following
> steps with the Apache user, which has UID 48 on both machines - the
> result was the same.
> 
> As this is not an actual user in my realm, I first created a service
> principal of the form git/$HOSTNAME@REALM (or apache/... for that
> matter). I then used ipa-getkeytab to create a keytab in
> /var/lib/gssproxy/clients/$UID.keytab for gssproxy to find. That worked
> nicely as in: The user automagically got access to the mounted NFS share
> while a krb5cc_$UID was created in the directory mentioned above. After
> switching users with su, I can navigate through the mount - as long as
> all the folders have 755 permissions. A folder with 700 permissions and
> owner 'git' is correctly displayed as being owned by 'git' on the client
> - yet I cannot access it! When I create a file or folder in a folder
> with public permissions (777), the owner of the newly created file is
> 'nfsnobody'.
> 
> I also tried setting up a static mapping in /etc/idmapd.conf on both the
> server and client: mapping the service principal to user 'git'. The
> effect was the client displaying the folder being owned by 'nobody' -
> whoops.
> 
> Doing all the above steps with an actual user in the realm works fine.
> Either with the automagic method through gssproxy or by getting a ticket
> with kinit first: I can access a folder with 700 permissions and files
> are created with the correct owner, etc.
> 
> 
> Is there any critical step that I missed? I feel like I am very close ..
> I'd be thankful for any hints.
> 
> Cheers,
> Anton
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
> 
# remount nfs share
  1: rpc.gssd[552]: handle_gssd_upcall: 'mech=krb5 uid=0 service=* enctypes=18,17,16,23,3,1,2 ' (nfs/clnt2)
  2: rpc.gssd[552]: krb5_use_machine_creds: uid 0 tgtname (null)
  3: rpc.gssd[552]: Full hostname for 'zfs0.storage.$domain' is 'zfs0.storage.$domain'
  4: rpc.gssd[552]: Full hostname for 'test00.client.$domain' is 'test00.client.$domain'
  5: rpc.gssd[552]: No key table entry found for test00$@$REALM while getting keytab entry for 'test00$@$REALM'
  6: rpc.gssd[552]: No key table entry found for TEST00$@$REALM while getting keytab entry for 'TEST00$@$REALM'
  7: rpc.gssd[552]: No key table entry found for root/test00.client.$domain@$REALM while getting keytab entry for 'root/test00.client.$domain@$REALM'
  8: rpc.gssd[552]: No key table entry found for nfs/test00.client.$domain@$REALM while getting keytab entry for 'nfs/test00.client.$domain@$REALM'
  9: rpc.gssd[552]: Success getting keytab entry for 'host/test00.client.$domain@$REALM'
 10: rpc.gssd[552]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500578570
 11: rpc.gssd[552]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500578570
 12: gssproxy[534]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 13: gssproxy[531]: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "git", euid: 0, socket: (null)
 14: gssproxy[531]: gssproxy[534]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 15: rpc.gssd[552]: creating tcp client for server zfs0.storage.$domain
 16: rpc.gssd[552]: creating context with server nfs@zfs0.storage.$domain
 17: rpc.gssd[552]: doing downcall: lifetime_rec=86193 acceptor=nfs@zfs0.storage.$domain
 18: rpc.gssd[552]: handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 ' (nfs/clnt2)
 19: rpc.gssd[552]: krb5_use_machine_creds: uid 0 tgtname (null)
 20: rpc.gssd[552]: Full hostname for 'zfs0.storage.$domain' is 'zfs0.storage.$domain'
 21: rpc.gssd[552]: Full hostname for 'test00.client.$domain' is 'test00.client.$domain'
 22: gssproxy[531]: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "git", euid: 0, socket: (null)
 23: gssproxy[531]: gssproxy[534]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 24: rpc.gssd[552]: No key table entry found for test00$@$REALM while getting keytab entry for 'test00$@$REALM'
 25: rpc.gssd[552]: No key table entry found for TEST00$@$REALM while getting keytab entry for 'TEST00$@$REALM'
 26: rpc.gssd[552]: No key table entry found for root/test00.client.$domain@$REALM while getting keytab entry for 'root/test00.client.$domain@$REALM'
 27: rpc.gssd[552]: No key table entry found for nfs/test00.client.$domain@$REALM while getting keytab entry for 'nfs/test00.client.$domain@$REALM'
 28: rpc.gssd[552]: Success getting keytab entry for 'host/test00.client.$domain@$REALM'
 29: rpc.gssd[552]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500578570
 30: rpc.gssd[552]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500578570
 31: gssproxy[534]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 32: rpc.gssd[552]: creating tcp client for server zfs0.storage.$domain
 33: rpc.gssd[552]: creating context with server nfs@zfs0.storage.$domain
 34: rpc.gssd[552]: doing downcall: lifetime_rec=86193 acceptor=nfs@zfs0.storage.$domain
# switch user to git
 35: su[9371]: (to git) root on pts/2
 36: su[9371]: pam_unix(su:session): session opened for user git by (uid=0)
 37: rpc.gssd[552]: handle_gssd_upcall: 'mech=krb5 uid=995 enctypes=18,17,16,23,3,1,2 ' (nfs/clnt2)
 38: rpc.gssd[552]: krb5_not_machine_creds: uid 995 tgtname (null)
 39: gssproxy[531]: Client connected (fd = 12) (pid = 552) (uid = 995) (gid = 992) (context = system_u:system_r:gssd_t:s0)
 40: gssproxy[531]: gp_rpc_execute: executing 6 (GSSX_ACQUIRE_CRED) for service "git", euid: 995, socket: (null)
 41: rpc.gssd[552]: creating tcp client for server zfs0.storage.$domain
 42: rpc.gssd[552]: creating context with server nfs@zfs0.storage.$domain
 43: gssproxy[531]: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "git", euid: 995, socket: (null)
 44: gssproxy[531]: gp_rpc_execute: executing 8 (GSSX_INIT_SEC_CONTEXT) for service "git", euid: 995, socket: (null)
 45: rpc.gssd[552]: doing downcall: lifetime_rec=77748 acceptor=nfs@zfs0.storage.$domain
# navigating and listing directories
 46: nfsidmap[9396]: key: 0x3460b64 type: uid value: admin@$domain timeout 600
 47: nfsidmap[9396]: nfs4_name_to_uid: calling static->name_to_uid
 48: nfsidmap[9396]: nfs4_name_to_uid: static->name_to_uid returned -2
 49: nfsidmap[9396]: nfs4_name_to_uid: calling nsswitch->name_to_uid
 50: nfsidmap[9396]: nss_getpwnam: name 'admin@$domain' domain '$domain': resulting localname 'admin'
 51: nfsidmap[9396]: nfs4_name_to_uid: nsswitch->name_to_uid returned 0
 52: nfsidmap[9396]: nfs4_name_to_uid: final return value is 0
 53: nfsidmap[9398]: key: 0x366bf458 type: gid value: kerberos@$domain timeout 600
 54: nfsidmap[9398]: nfs4_name_to_gid: calling static->name_to_gid
 55: nfsidmap[9398]: nfs4_name_to_gid: static->name_to_gid returned -2
 56: nfsidmap[9398]: nfs4_name_to_gid: calling nsswitch->name_to_gid
 57: nfsidmap[9398]: nfs4_name_to_gid: nsswitch->name_to_gid returned 0
 58: nfsidmap[9398]: nfs4_name_to_gid: final return value is 0
 59: nfsidmap[9406]: key: 0x1802b55 type: uid value: git/test00.client.$domain@$REALM timeout 600
 60: nfsidmap[9406]: nfs4_name_to_uid: calling static->name_to_uid
 61: nfsidmap[9406]: static_getpwnam: name 'git/test00.client.$domain@$REALM' mapped to 'git'
 62: nfsidmap[9406]: nfs4_name_to_uid: static->name_to_uid returned 0
 63: nfsidmap[9406]: nfs4_name_to_uid: final return value is 0
# the 'permission denied' upon cd'ing does not produce any log
 64: su[9371]: pam_unix(su:session): session closed for user git
  1: rpc.gssd[564]: handle_gssd_upcall: 'mech=krb5 uid=0 target=host@test00.client.$domain service=nfs enctypes=18,17,16,23,3,1,2 ' (nfsd4_cb/clnt1)
  2: rpc.gssd[564]: krb5_use_machine_creds: uid 0 tgtname host@test00.client.$domain
  3: rpc.gssd[564]: Full hostname for 'test00.client.$domain' is 'test00.client.$domain'
  4: rpc.gssd[564]: Full hostname for 'zfs0.storage.$domain' is 'zfs0.storage.$domain'
  5: rpc.gssd[564]: Success getting keytab entry for 'nfs/zfs0.storage.$domain@$REALM'
  6: rpc.gssd[564]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500577876
  7: rpc.gssd[564]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500577876
  8: gssproxy[535]: gssproxy[545]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
  9: gssproxy[545]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 10: rpc.gssd[564]: creating tcp client for server test00.client.$domain
 11: rpc.gssd[564]: WARNING: can't create tcp rpc_clnt to server test00.client.$domain for user with uid 0: RPC: Remote system error - No route to host
 12: rpc.gssd[564]: WARNING: Failed to create machine krb5context with cred cache FILE:/tmp/krb5ccmachine_$REALM for server test00.client.$domain
 13: rpc.gssd[564]: WARNING: Machine cache prematurelyexpired or corrupted trying torecreate cache for server test00.client.$domain
 14: rpc.gssd[564]: Full hostname for 'test00.client.$domain' is 'test00.client.$domain'
 15: rpc.gssd[564]: Full hostname for 'zfs0.storage.$domain' is 'zfs0.storage.$domain'
 16: rpc.gssd[564]: Success getting keytab entry for 'nfs/zfs0.storage.$domain@$REALM'
 17: gssproxy[535]: gssproxy[545]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 18: rpc.gssd[564]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500577876
 19: rpc.gssd[564]: INFO: Credentials in CC 'FILE:/tmp/krb5ccmachine_$REALM' are good until 1500577876
 20: gssproxy[545]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
 21: rpc.gssd[564]: creating tcp client for server test00.client.$domain
 22: rpc.gssd[564]: WARNING: can't create tcp rpc_clnt to server test00.client.$domain for user with uid 0: RPC: Remote system error - No route to host
 23: rpc.gssd[564]: WARNING: Failed to create machine krb5context with cred cache FILE:/tmp/krb5ccmachine_$REALM for server test00.client.$domain
 24: rpc.gssd[564]: WARNING: Failed to create machinekrb5 context with any credentialscache for server test00.client.$domain
 25: rpc.gssd[564]: doing error downcall
 26: rpc.idmapd[525]: nfsdcb: authbuf=gss/krb5p authtype=user
 27: rpc.idmapd[525]: nfs4_uid_to_name: calling static->uid_to_name
 28: rpc.idmapd[525]: nfs4_uid_to_name: static->uid_to_name returned -2
 29: rpc.idmapd[525]: nfs4_uid_to_name: calling nsswitch->uid_to_name
 30: rpc.idmapd[525]: nfs4_uid_to_name: nsswitch->uid_to_name returned 0
 31: rpc.idmapd[525]: nfs4_uid_to_name: final return value is 0
 32: rpc.idmapd[525]: Server : (user) id "1863400000" -> name "admin@$domain"
 33: rpc.idmapd[525]: nfsdcb: authbuf=gss/krb5p authtype=group
 34: rpc.idmapd[525]: nfs4_gid_to_name: calling static->gid_to_name
 35: rpc.idmapd[525]: nfs4_gid_to_name: static->gid_to_name returned -2
 36: rpc.idmapd[525]: nfs4_gid_to_name: calling nsswitch->gid_to_name
 37: rpc.idmapd[525]: nfs4_gid_to_name: nsswitch->gid_to_name returned 0
 38: rpc.idmapd[525]: nfs4_gid_to_name: final return value is 0
 39: rpc.idmapd[525]: Server : (group) id "1863400009" -> name "kerberos@$domain"
 40: rpc.idmapd[525]: nfsdcb: authbuf=gss/krb5p authtype=user
 41: rpc.idmapd[525]: nfs4_uid_to_name: calling static->uid_to_name
 42: rpc.idmapd[525]: nfs4_uid_to_name: static->uid_to_name returned 0
 43: rpc.idmapd[525]: nfs4_uid_to_name: final return value is 0
 44: rpc.idmapd[525]: Server : (user) id "995" -> name "git/test00.client.$domain@$REALM"
 45: 
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org

Reply via email to