Yes sir - we are indeed able to create files with that group. By chance, are you using /etc/nsswitch.conf? Do your entries for passwd, shadow, and group look something like this? passwd: files ldap shadow: files ldap group: files ldap (We did try to reverse that order to "ldap files". The same warnings get thrown).
On Wed, May 30, 2018 at 11:11 AM, Shawn Weeks <[email protected]> wrote: > Logged in as the user NiFi is running as on the same host are you able to > create files with that group? We use PutFile and none of our groups are > local to the host. > > > > Thanks > > Shawn > > > > *From:* James McMahon <[email protected]> > *Sent:* Wednesday, May 30, 2018 8:21 AM > *To:* [email protected] > *Subject:* Re: User, Group in LDAP appear to be unknown to PutFile > > > > I did indeed configure PutFile as follows: > > Permissions ..... 775 > > Owner ..... nifi > > Group ..... ext_dev > > When nifi is in local /etc/passwd and ext_dev is in local /etc/group, the > PutFile succeeds. > > When neither exists in the local files, I get the Warning in both case and > the file is output with nifi:nifi (the default, as you mentioned Pierre). > > When one is put back in the local file and the other left in the ldap, the > one in the local succeeds and the one only in the ldap still throws the > Warning and defaults on the output file. > > We have also tried to reorder in our nsswitch.conf. For instance, changing > > group: files ldap > > to > > group: ldap files > > This did not change the results. > > > > From the NiFi processor, user and group resolution do not appear to be > looking beyond a check in the local files. > > I did double check that user nifi exists only in either the local file or > the ldap. Likewise for group ext_dev. > > > > > > On Wed, May 30, 2018 at 8:52 AM, Pierre Villard < > [email protected]> wrote: > > By default, PutFile will set the ownership of the file to the user running > the NiFi instance (nifi if NiFi is running as nifi user). Then, if you > configured a different ownership in the processor configuration it'll try > to set the ownership using the username you configured in the processor. > What did you set in the processor parameters for owner/group and permission? > > > > 2018-05-30 14:39 GMT+02:00 James McMahon <[email protected]>: > > Yes sir, sure does. In this instance my user nifi does indeed resolve at > the OS level - I think that gives us some confidence it does resolve. The > lookupPrincipalByName(owner) within the PutFile is where I believe the > failure is rooted, but I do not understand how that function executes its > lookup. I'm trying to dig deeper into that now. > > > > On Wed, May 30, 2018 at 8:30 AM, Pierre Villard < > [email protected]> wrote: > > I think we're saying the same :) Let me rephrase it differently: to set > the owner of a file, the user needs to be resolved at OS level. If the user > does not exist (from the OS point of view), NiFi won't be able to set the > owner (even though the username is in the LDAP configured for NiFi > authentication). Does it make more sense? > > > > $ touch test.file > $ id toto > > id: toto: no such user > > $ chown toto test.file > > chown: toto: illegal user name > > > > Pierre > > > > > > > > > > 2018-05-30 14:13 GMT+02:00 James McMahon <[email protected]>: > > I don't understand this: > "Until you *can't* resolve the user with OS commands, I don't think NiFi > will be able to set the expected owner on the file" > > Did you intend to say can there - don't we want to be able to resolve the > user at the OS as an initial validation that we can get to the ldap and as > prerequisite of Nifi being able to set the owner? And in this case I can > indeed resolve at the OS level. That seems like a good thing to me. > > > > On Wed, May 30, 2018 at 7:53 AM, Pierre Villard < > [email protected]> wrote: > > It depends how your OS is configured, you could leverage tools like SSSD > to resolve users against your LDAP but that's something to be configured at > OS level. > > Until you can't resolve the user with OS commands, I don't think NiFi will > be able to set the expected owner on the file. > > > > 2018-05-30 11:54 GMT+02:00 James McMahon <[email protected]>: > > Hello Pierre, and thank you. The user in this case - nifi - is not in the > local /etc/passwd and is in the ldap. I presume this will force the id > <username> to resolve using the ldap, if it does resolve? At the OS the id > command returns the uid, the gid, and the groups to which user nifi has > membership within the ldap. > > > > On Wed, May 30, 2018 at 4:37 AM, Pierre Villard < > [email protected]> wrote: > > Hi Jim, > > > > LDAP for authentication and authorizations in NiFi has nothing to do with > the processors. > > How processors are running/working is completely independent to the > authN/authZ model you configure for NiFi. > > > > Regarding your error, I'd say that you get this error because user/group > you're setting in the processor configuration cannot be resolved at OS > level (even though they exist in the LDAP, but again, that's totally > unrelated). Something you can quickly check: can you resolve the > username/group on the host where you're using PutFile processor? What do > you get if you execute the following command: id <username>? > > > > Pierre > > > > 2018-05-30 1:14 GMT+02:00 Joe Witt <[email protected]>: > > jim > > > > please only post to one list. > > > users is good for this. > > > > thanks > > joe > > > > On Tue, May 29, 2018, 3:54 PM James McMahon <[email protected]> wrote: > > Good evening. I have recently migrated my nifi service host server from > local resolution of users and groups to use an LDAP server. I configured > login-identity-providers.xml and nifi.security.user.login.identity.provider. > I verified my configuration is known to NiFi by first restarting my nifi > service and then attempting a login to the URL by a user without a cert, > forcing it to resolve using LDAP. This appeared to work. > > I then attempted to set my file owner and file group in a PutFile to a > user and a group that are each in the LDAP. The PutFile throws a Warning > for both owner and group: java.nio.file.attribute. > UserPrincipalNotFoundException. The file is still output by the > processor. It appears to default the user and owner to nifi. > > A cursory review of the PutFile source shows that PutFile employs > getUserPrincipalLookupService() when it seemingly tries to validate the > user and group. > > How can I get this to resolve through the LDAP for the PutFile? > > Thanks for any insights. -Jim > > > > > > > > > > > > > > > > >
