Hi, Did the test 2.12.4 with the same result. Also, I narrowed it down to SSK only. It also happens without nodemaps being activated.
@Sebastian: I wonder if you did test this with SSK? I was very focused on nodemaps being the cause to start with. Cheers, Hans Henrik On 29.02.2020 23.44, Hans Henrik Happe wrote: > Hi, > > Sorry for the delay. I had to spend some time nursing the glusterfs that > this lustre fs will replace :-) > > Anyway, I've created a procedure to reproduce the issue. It's attached > together with the testing program. > > Basically, its a simple single mgs,mdt,oss setup, with a nodemap, that > maps a client to a fileset. This works fine. However, when turning on > SSK for cli2mdt the issue appears. > > This was for 2.12.3, I will move on to 2.12.4 just to check. > > Cheers, > Hans Henrik > > On 06.02.2020 23.08, Hans Henrik Happe wrote: >> Hi Sebastien, >> >> Thanks for looking into this. >> >> You are right that nodemap deactivation didn't affect the outcome. I >> must have made a mistake and cannot reproduce. >> >> The uid/gid are on the mds. I can do a sudo to the user and run the test >> program successfully. >> >> I forgot to mention that I use SSK in ski mode. >> >> I think I will start from scratch and see if I can reproduce and find >> out at what point it stops working. >> >> Cheers, >> Hans Henrik >> >> On 06.02.2020 18.19, Sebastien Buisson wrote: >>> Hi, >>> >>> I am not able to reproduce your issue. I compiled your C program, in all >>> cases I am not getting Permission Denied. >>> >>> You say that it works when you deactivate the nodemap. But given that you >>> have a fileset on your nodemap entry « sif », when you deactivate it you >>> might end up doing IOs in a different directory. So you might compare >>> different things. >>> Also, does the uid/gid 20501 exist on server side? >>> >>> Cheers, >>> Sebastien. >>> >>>> Le 6 févr. 2020 à 14:29, Hans Henrik Happe <[email protected]> a écrit : >>>> >>>> Hi, >>>> >>>> Thanks for a very quick reply :-) Here are the map: >>>> >>>> # lctl get_param nodemap.sif.* >>>> nodemap.sif.admin_nodemap=1 >>>> nodemap.sif.audit_mode=1 >>>> nodemap.sif.deny_unknown=0 >>>> nodemap.sif.exports= >>>> [ >>>> { nid: 172.25.10.51@tcp, uuid: 56bb9b04-9bb5-d7b5-3f50-d62804690db1 }, >>>> ] >>>> nodemap.sif.fileset=/sif >>>> nodemap.sif.id=2 >>>> nodemap.sif.idmap= >>>> [ >>>> { idtype: uid, client_id: 501, fs_id: 20501 }, >>>> { idtype: gid, client_id: 501, fs_id: 20501 } >>>> ] >>>> nodemap.sif.map_mode=both >>>> nodemap.sif.ranges= >>>> [ >>>> { id: 11, start_nid: 172.25.1.28@tcp, end_nid: 172.25.1.28@tcp }, >>>> { id: 10, start_nid: 172.25.1.27@tcp, end_nid: 172.25.1.27@tcp }, >>>> { id: 9, start_nid: 172.25.10.51@tcp, end_nid: 172.25.10.51@tcp } >>>> ] >>>> nodemap.sif.sepol= >>>> >>>> nodemap.sif.squash_gid=20000 >>>> nodemap.sif.squash_uid=20000 >>>> nodemap.sif.trusted_nodemap=0 >>>> >>>> Cheers, >>>> Hans Henrik >>>> >>>> On 06.02.2020 14.17, Sebastien Buisson wrote: >>>>> Hi, >>>>> >>>>> It might be due to a property on the nodemap you defined. >>>>> Could you please dump your nodemap definition? >>>>> >>>>> Thanks, >>>>> Sebastien. >>>>> >>>>> >>>>>> Le 6 févr. 2020 à 14:14, Hans Henrik Happe <[email protected]> >>>>>> a écrit : >>>>>> >>>>>> Hi, >>>>>> >>>>>> Has anyone had success with gocryptfs 1.7.x on top of a Lustre nodemap? >>>>>> >>>>>> I've tested with Lustre 2.12.3. >>>>>> >>>>>> I found that gocryptfs 1.6 worked. However, with 1.7.x I got a lot of >>>>>> "Permission denied". I tried all permutations of trusted and admin on >>>>>> the nodemap. >>>>>> >>>>>> By stracing a bit, I've created a small peace of code provoking the >>>>>> issue: >>>>>> >>>>>> --- >>>>>> >>>>>> #include <unistd.h> >>>>>> #include <sys/types.h> >>>>>> #include <fcntl.h> >>>>>> #include <stdio.h> >>>>>> >>>>>> int main() { >>>>>> int r; >>>>>> >>>>>> setregid(-1, 501); >>>>>> setreuid(-1, 501); >>>>>> >>>>>> r = open("foo", O_CREAT, S_IRWXU); >>>>>> if (r < 0) { >>>>>> perror("open"); >>>>>> } >>>>>> return 0; >>>>>> } >>>>>> >>>>>> --- >>>>>> >>>>>> >>>>>> >>>>>> When run as root in a directory owned by uid=501 and gid=501 in a >>>>>> nodemap based Lustre fs it returns: >>>>>> >>>>>> open: Permission denied >>>>>> >>>>>> Works when I deactivate nodemap (lctl nodemap_activate 0) or just use a >>>>>> plain local fs. >>>>>> >>>>>> I don't think this is intended behavior for nodemaps, but I might be >>>>>> wrong. >>>>>> >>>>>> Cheers, >>>>>> Hans Henrik >>>>>> _______________________________________________ >>>>>> lustre-discuss mailing list >>>>>> >>>>>> [email protected] >>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >>>> _______________________________________________ >>>> lustre-discuss mailing list >>>> [email protected] >>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org >> _______________________________________________ >> lustre-discuss mailing list >> [email protected] >> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > > _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
