Re: [OpenAFS] Question for admins regarding pts membership output
On 7/15/2022 6:18 PM, Richard Brittain (richard.britt...@dartmouth.edu)
wrote:
On 2022-07-15, 09:04, "Jeffrey E Altman" wrote:
On 7/13/2022 6:07 PM, Richard Brittain (richard.britt...@dartmouth.edu)
wrote:
> I hope that doesn't lead people to expect 'pts membership
system:authuser' to show all users.
>
> Richard
I'm curious. Why would it be wrong for users to expect 'pts membership
system:authuser' and 'pts membership system:anyuser' to list their
membership assuming the caller had the necessary access rights?
Only that the output of system:authuser would be confusingly long, and what
would system:anyuser generate anyway ?. We also have scripts for 'show me
everyone who has access to this entity', which gets complicated with nested
groups, and I couldn't figure out what to display for 'everyone'. It would be
valid to ignore named users in the ACL and just say 'everyone' in that case.
What to display for "everyone" is easy, its "system:anyuser".
The output of system:authuser in OpenAFS would be close to the output of
pts listentries -user | grep -v '@' | grep -v 'anonymous' | gawk
'{print $1}'
In other words, the list of all user entries that are not foreign and
are not "anonymous". it would also exclude any IP address entries.
The output of system:anyuser would be
pts listentries -user | gawk '{print $1}'
again with the exception of all IP address entries. The difference is
that system:anyuser output includes "anonymous" and the foreign entities.
In an AuriStorFS world the system:authuser and system:anyuser lists
would also exclude "machine" and "network" entities.
Enumerating the membership of system:anyuser and system:authuser would
by default be restricted to "-showmembers self" which means that only
members of the system:administrators group would be able to enumerate
the membership.
A cell that wished to offer broader access might set "-showmembers
members" on system:authuser but that would be the same as "-showmembers
anyone" for "system:anyuser". I think the default is appropriate for
all cells.
Tangentially related, we use a wrapper to list AFS groups, which looks up a few
bits of useful information about each member besides their AFS username. This
is very user-friendly, but means lots of LDAP lookups and would take forever on
the full output of system:authuser.
Makes sense. That would take a while for a cell with several hundred
thousand users.
I can imagine a plugin for both the protection service and the pts
client that would allow the protection service to query LDAP or some
other service and return an opaque blob to the pts client to be unpacked
and displayed by the pts plugin.
Jeffrey Altman
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [OpenAFS] Question for admins regarding pts membership output
Only that the output of system:authuser would be confusingly long, and what would system:anyuser generate anyway ?. We also have scripts for 'show me everyone who has access to this entity', which gets complicated with nested groups, and I couldn't figure out what to display for 'everyone'. It would be valid to ignore named users in the ACL and just say 'everyone' in that case. Tangentially related, we use a wrapper to list AFS groups, which looks up a few bits of useful information about each member besides their AFS username. This is very user-friendly, but means lots of LDAP lookups and would take forever on the full output of system:authuser. I didn't know about pts delete automatically removing from groups - that does remove my only real use case for relying on the output of pts membership in decommissioning. Richard On 2022-07-15, 09:04, "Jeffrey E Altman" wrote: On 7/13/2022 6:07 PM, Richard Brittain (richard.britt...@dartmouth.edu) wrote: > I hope that doesn't lead people to expect 'pts membership system:authuser' to show all users. > > Richard I'm curious. Why would it be wrong for users to expect 'pts membership system:authuser' and 'pts membership system:anyuser' to list their membership assuming the caller had the necessary access rights? My primary objection to the existing behavior is that these groups are special and end users / administrators must understand that they are special. If an authorized user can obtain the membership list from 'pts membership system:authuser@foreign' why shouldn't the same be true for 'system:authuser'? If the concern is the cost of generating the result set, its no more expensive then executing 'pts listentries'. In a private response to my original message someone wrote that their cell uses the output of 'pts membership' to generate the list of entities that have access to a file object given the assigned ACL. This is a perfectly reasonable action to expect to work. However, the generated list will be incomplete when 'pts membership system:anyuser' and 'pts membership system:authuser' succeed while at the same time generate empty output. Jeffrey Altman
Re: [OpenAFS] Question for admins regarding pts membership output
On 7/13/2022 6:07 PM, Richard Brittain (richard.britt...@dartmouth.edu) wrote: I hope that doesn't lead people to expect 'pts membership system:authuser' to show all users. Richard I'm curious. Why would it be wrong for users to expect 'pts membership system:authuser' and 'pts membership system:anyuser' to list their membership assuming the caller had the necessary access rights? My primary objection to the existing behavior is that these groups are special and end users / administrators must understand that they are special. If an authorized user can obtain the membership list from 'pts membership system:authuser@foreign' why shouldn't the same be true for 'system:authuser'? If the concern is the cost of generating the result set, its no more expensive then executing 'pts listentries'. In a private response to my original message someone wrote that their cell uses the output of 'pts membership' to generate the list of entities that have access to a file object given the assigned ACL. This is a perfectly reasonable action to expect to work. However, the generated list will be incomplete when 'pts membership system:anyuser' and 'pts membership system:authuser' succeed while at the same time generate empty output. Jeffrey Altman smime.p7s Description: S/MIME Cryptographic Signature
Re: [OpenAFS] Question for admins regarding pts membership output
On 7/14/22 5:49 AM, Dirk Heinrichs wrote: Ed Rude: I think I prefer the new behavior you are suggesting as the default. I'd prefer to have the current behavior as default, as to not break current scripts. Admins can then decide to enhance their scripts as needed instead of being forced to change them because they got broken. On the other hand, I'd prefer a diminishing number of broken scripts vs. a future of less than ideal defaults, especially if some warning is issued ahead of the change. Backwards compatibility has it's place: in the past, mostly. -- +---+ / Todd Lewis, Middleware Services,uto...@email.unc.edu / / "We is confronted with insurmountable opportunities." / / - Walt Kelly, "Pogo" / +---+
Re: [OpenAFS] Question for admins regarding pts membership output
Ed Rude: > I think I prefer the new behavior you are suggesting as the default. I'd prefer to have the current behavior as default, as to not break current scripts. Admins can then decide to enhance their scripts as needed instead of being forced to change them because they got broken. Bye... Dirk -- Dirk Heinrichs Matrix-Adresse: @heini:chat.altum.de GPG Public Key: 80F1540E03A3968F3D79C382853C32C427B48049 Privacy Handbuch: https://www.privacy-handbuch.de OpenPGP_signature Description: OpenPGP digital signature
Re: [OpenAFS] Question for admins regarding pts membership output
As of the writing of this reply there have been several other replies to my original e-mail from Ed Rude, Richard Brittain, and Gary Buhrmaster. As there is some overlap in the responses I will reply once to Dave Botsch's but I intend to touch on the feedback from all of the above. On 7/13/2022 10:07 AM, Dave Botsch (bot...@cnf.cornell.edu) wrote: I suspect our user deprovisioning scripts would break by trying to explicitly remove users from those groups. Though would be easy enough to fix. And I'm in favor of having this extra output. Several replies mentioned that the user deprovisioning scripts use the output of "pts membership "to drive something like: for each in "pts membership " pts removeuser where an error from "pts removeuser" causes the deprovisioning operation to fail. I am curious, is the removal of the user from all groups performed as a step to be followed by pts delete or is the entity left intact to preserve the name-to-id mapping? If the "pts removeuser" calls are followed by "pts delete", then the "pts removeuser" calls are unnecessary overhead since "pts delete " will automatically remove the user from all groups as part of the deletion process. The removal will take a single RPC and reduce the number of UBIK write transactions necessary to perform the operation. On the other hand, if the intent is to preserve the name-to-id mapping, in OpenAFS the will still be a member of "system:anyuser" and "system:authuser". AuriStorFS supports the ability to disable a pts user entity so that the name-to-id mapping can be preserved while at the same time altering the current protection set for the user such that it is equivalent to the "anonymous" user. However, OpenAFS does not have this functionality. In OpenAFS, an attempt to "pts remove user system:anyuser" or "pts remove user system:authuser" will fail with a PRNOENT error "User or group doesn't exist" indicating that the user is not a member of the group. The error is ambiguous because there is no context to inform the caller whether the problem is (1) the "group" from which the "user" is to be removed from does not exist; or (2) the user is not a member of the group which is the intended outcome. I will argue that in this case, the protection service should return success if the attempt to remove a user from a group fails because the user is not a member. Such a change would also remove the ambiguity surrounding the PRNOENT error ensuring that it means that the group does not exist. If such a change was applied to PR_RemoveFromGroup() then "pts removeuser system:anyuser" and "pts removeuser system:authuser" would succeed instead of fail. An unnecessary RPC would be issued but there would be no script failure. Two questions/thoughts would be: 1) If this is a "backwards-incompatible" change (is it?) should it be reserved for the next major version upgrade (2.0) ? That is a question for the OpenAFS release team. I am considering this change for the next AuriStorFS v2022.xx release. 2) Use of a flag to pts membership to include (or not include) explicit and implicit membership, as I might very well want to filter the output... the question then becomes which way should be the "default"? There is no well-defined meaning for implicit vs explicit group membership at the protocol layer. The Protection Service return from the PR_ListElements RPC is an array of integers (pts ids). The concept of implicit vs explicit group membership management is a server side implementation detail. I suspect there are few on this mailing list that remember that when the concept of foreign users was added to the Protection Service there was a compile time option to determine if the membership of "system:authuser@foreign" groups were explicit or implicit. The implicit membership option was removed from the code base many years ago. However, in many regards the implicit membership model makes a lot more sense than the explicit model. The explicit model can result in a foreign user unintentionally missing from system:authuser@foreign. This failure is very difficult to debug. Another reason for converting "system:authuser@foreign" to an implicit membership group is to ensure that the management of local and foreign users is consistent, and to ensure that the generated Current Protection Set for a local and foreign user is consistent. Gary Buhrmaster requested the ability to label group memberships as being explicit or implicit. In my opinion, exposing the explicit vs implicit implementation detail to the pts client would be an abstraction layer violation. For the same reason I would be reluctant to add a "pts membership -explicit" switch. Switches such as -no-system-anyuser and -no-system-authuser could be added and when set the "pts" command could filter out the existence of groups -101 and -102. Although I find such options ugly compared to
Re: [OpenAFS] Question for admins regarding pts membership output
On Wed, Jul 13, 2022 at 1:49 PM Jeffrey E Altman wrote: > The question for cell admins is whether anyone is aware of any internal > scripts which process the output of "pts membership" which will break as > a result of the inclusion of the implicit groups "system:anyuser" and > "system:authuser" in output. > > Your assistance is appreciated. I am no longer a cell admin, but I am sure such scripts (which process the output) exist, and will need modification. I am, however, in favor of expanding the output (although an " (implicit) " append might be useful to help humans interpret the result, and for scripts to be able to parse such). However, while out of scope, I would (long term) prefer the output of commands to be able to generate a machine parseable output (json?) so that parsing output can be more robust(*)(**). Gary (*) I presume like many others I have more than once written a script which parsee the output of a command and experienced breakage when upstream changed the format of the output. (**) And while I do not know what the json would include, an "implicit" boolean flag would seem to be desirable for each array element of the result. ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Question for admins regarding pts membership output
Ditto - our deprovisioning scripts use pts membership output, and I expect this is common. Filtering out system:anyuser etc. would be easy, but a flag to omit those and revert to 'old behaviour' would be even better. I do like the improved transparency of listing them though. I hope that doesn't lead people to expect 'pts membership system:authuser' to show all users. Richard On 7/13/22, 10:08 AM, "openafs-info-ad...@openafs.org on behalf of Dave Botsch" wrote: I suspect our user deprovisioning scripts would break by trying to explicitly remove users from those groups. Though would be easy enough to fix. And I'm in favor of having this extra output. Two questions/thoughts would be: 1) If this is a "backwards-incompatible" change (is it?) should it be reserved for the next major version upgrade (2.0) ? 2) Use of a flag to pts membership to include (or not include) explicit and implicit membership, as I might very well want to filter the output... the question then becomes which way should be the "default"? thanks. On Wed, Jul 13, 2022 at 09:49:29AM -0400, Jeffrey E Altman wrote: > The Protection Service groups fall into two categories. Those with > explicit membership lists and those with implicit membership lists. For > example, the "system:anyuser" and "system:authuser" groups are implicit > whereas "system:administrators", "system:ptsviewers", and > "system:authuser@foreign-realm" groups are explicit. > > The output of "pts membership" only includes memberships in explicit > membership groups. This has a negative impact inexperienced end users that > might be unaware that they are members of the "system:anyuser" and > "system:authuser" groups. This behavior also leads to an inconsistency > between the behavior for foreign and local users because foreign users are > not members of "system:authuser" and are members of > "system:authuser@foreign" which is included in the membership list because > that group has an explicit membership list. > > The AuriStorFS Protection service also makes a distinction between "user" > and "machine" or "network" entities where "machine" and "network" entities > are not members of the "system:authuser" or "system:authuser@foreign" > groups. This distinction is not apparent from the output of "pts > membership" because of the exclusion of implicit groups. > > AuriStor is considering a change to "pts membership" output to include > implicit memberships in the output of "pts membership". With this change the > output of these commands > > $ pts membership anonymous > Groups anonymous (id: 32766) is a member of: > > $ pts membership testuser > Groups anonymous (id: 112) is a member of: > > $ pts membership testuser@foreign > Groups anonymous (id: 43282) is a member of: > system:authuser@foreign > > becomes > > $ pts membership anonymous > Groups anonymous (id: 32766) is a member of: > system:anyuser > > $ pts membership testuser > Groups anonymous (id: 112) is a member of: > system:anyuser > system:authuser > > $ pts membership testuser@foreign > Groups anonymous (id: 43282) is a member of: > system:authuser@foreign > system:anyuser > > The question for cell admins is whether anyone is aware of any internal > scripts which process the output of "pts membership" which will break as a > result of the inclusion of the implicit groups "system:anyuser" and > "system:authuser" in output. > > Your assistance is appreciated. > > Jeffrey Altman > AuriStor, Inc. > -- David William Botsch Programmer/Analyst @CornellCNF bot...@cnf.cornell.edu ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Question for admins regarding pts membership output
I second the inclusion of an explicit way of requesting one behavior or the other. As long as I have a way to explicitly specify both behaviors working around the change in anything that wraps the pts command should be simple enough. I think I prefer the new behavior you are suggesting as the default. Thank you, Ed On Wed, Jul 13, 2022 at 10:08 Dave Botsch wrote: > I suspect our user deprovisioning scripts would break by trying to > explicitly remove users from those groups. Though would be easy enough > to fix. And I'm in favor of having this extra output. > > Two questions/thoughts would be: > > 1) If this is a "backwards-incompatible" change (is it?) should it be > reserved for the next major version upgrade (2.0) ? > > 2) Use of a flag to pts membership to include (or not include) explicit > and implicit membership, as I might very well want to filter the > output... the question then becomes which way should be the "default"? > > thanks. > > On Wed, Jul 13, 2022 at 09:49:29AM -0400, Jeffrey E Altman wrote: > > The Protection Service groups fall into two categories. Those with > > explicit membership lists and those with implicit membership lists. For > > example, the "system:anyuser" and "system:authuser" groups are implicit > > whereas "system:administrators", "system:ptsviewers", and > > "system:authuser@foreign-realm" groups are explicit. > > > > The output of "pts membership" only includes memberships in explicit > > membership groups. This has a negative impact inexperienced end users > that > > might be unaware that they are members of the "system:anyuser" and > > "system:authuser" groups. This behavior also leads to an inconsistency > > between the behavior for foreign and local users because foreign users > are > > not members of "system:authuser" and are members of > > "system:authuser@foreign" which is included in the membership list > because > > that group has an explicit membership list. > > > > The AuriStorFS Protection service also makes a distinction between > "user" > > and "machine" or "network" entities where "machine" and "network" > entities > > are not members of the "system:authuser" or "system:authuser@foreign" > > groups. This distinction is not apparent from the output of "pts > > membership" because of the exclusion of implicit groups. > > > > AuriStor is considering a change to "pts membership" output to include > > implicit memberships in the output of "pts membership". With this change > the > > output of these commands > > > > $ pts membership anonymous > > Groups anonymous (id: 32766) is a member of: > > > > $ pts membership testuser > > Groups anonymous (id: 112) is a member of: > > > > $ pts membership testuser@foreign > > Groups anonymous (id: 43282) is a member of: > > system:authuser@foreign > > > > becomes > > > > $ pts membership anonymous > > Groups anonymous (id: 32766) is a member of: > > system:anyuser > > > > $ pts membership testuser > > Groups anonymous (id: 112) is a member of: > > system:anyuser > > system:authuser > > > > $ pts membership testuser@foreign > > Groups anonymous (id: 43282) is a member of: > > system:authuser@foreign > > system:anyuser > > > > The question for cell admins is whether anyone is aware of any internal > > scripts which process the output of "pts membership" which will break as > a > > result of the inclusion of the implicit groups "system:anyuser" and > > "system:authuser" in output. > > > > Your assistance is appreciated. > > > > Jeffrey Altman > > AuriStor, Inc. > > > > > > -- > > David William Botsch > Programmer/Analyst > @CornellCNF > bot...@cnf.cornell.edu > > ___ > OpenAFS-info mailing list > OpenAFS-info@openafs.org > https://lists.openafs.org/mailman/listinfo/openafs-info > -- Edward A. Rude Systems Administrator - Unix Systems Division of Information Technology
Re: [OpenAFS] Question for admins regarding pts membership output
I suspect our user deprovisioning scripts would break by trying to explicitly remove users from those groups. Though would be easy enough to fix. And I'm in favor of having this extra output. Two questions/thoughts would be: 1) If this is a "backwards-incompatible" change (is it?) should it be reserved for the next major version upgrade (2.0) ? 2) Use of a flag to pts membership to include (or not include) explicit and implicit membership, as I might very well want to filter the output... the question then becomes which way should be the "default"? thanks. On Wed, Jul 13, 2022 at 09:49:29AM -0400, Jeffrey E Altman wrote: > The Protection Service groups fall into two categories. Those with > explicit membership lists and those with implicit membership lists. For > example, the "system:anyuser" and "system:authuser" groups are implicit > whereas "system:administrators", "system:ptsviewers", and > "system:authuser@foreign-realm" groups are explicit. > > The output of "pts membership" only includes memberships in explicit > membership groups. This has a negative impact inexperienced end users that > might be unaware that they are members of the "system:anyuser" and > "system:authuser" groups. This behavior also leads to an inconsistency > between the behavior for foreign and local users because foreign users are > not members of "system:authuser" and are members of > "system:authuser@foreign" which is included in the membership list because > that group has an explicit membership list. > > The AuriStorFS Protection service also makes a distinction between "user" > and "machine" or "network" entities where "machine" and "network" entities > are not members of the "system:authuser" or "system:authuser@foreign" > groups. This distinction is not apparent from the output of "pts > membership" because of the exclusion of implicit groups. > > AuriStor is considering a change to "pts membership" output to include > implicit memberships in the output of "pts membership". With this change the > output of these commands > > $ pts membership anonymous > Groups anonymous (id: 32766) is a member of: > > $ pts membership testuser > Groups anonymous (id: 112) is a member of: > > $ pts membership testuser@foreign > Groups anonymous (id: 43282) is a member of: > system:authuser@foreign > > becomes > > $ pts membership anonymous > Groups anonymous (id: 32766) is a member of: > system:anyuser > > $ pts membership testuser > Groups anonymous (id: 112) is a member of: > system:anyuser > system:authuser > > $ pts membership testuser@foreign > Groups anonymous (id: 43282) is a member of: > system:authuser@foreign > system:anyuser > > The question for cell admins is whether anyone is aware of any internal > scripts which process the output of "pts membership" which will break as a > result of the inclusion of the implicit groups "system:anyuser" and > "system:authuser" in output. > > Your assistance is appreciated. > > Jeffrey Altman > AuriStor, Inc. > -- David William Botsch Programmer/Analyst @CornellCNF bot...@cnf.cornell.edu ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
