Thanks David- this is committed to CVS trunk now.
-Phil
David Metheny wrote:
Round 2...
It turns out that getting the groups a UID is part of is more costly,
especially on systems like LDAP. The LDAP system will have to scan all the
groups to determine that info, where as the call to get the members of a
group don't require the scan. Using NSCD, or something similar, will negate
the cost, but not all environments are configured or have that option.
Attached is a patch that increases the buffer size for the group permission
checking to around 1MB. That seems excessive, but 1024 bytes failed at
around 200 members of a group, and since the system can't determine the
correct size via sysconf, I just picked something that would take a long
time to break.
-----Original Message-----
From: Phil Carns [mailto:[email protected]] On Behalf Of Phil Carns
Sent: Tuesday, March 24, 2009 9:40 AM
To: [email protected]
Cc: [email protected]
Subject: Re: [Pvfs2-developers] patch: PVFS2 group permission checking for
groups with large number of members
Thanks David! This is a big improvement over the way the code worked
before.
The patch is in cvs trunk now with some minor formatting updates.
-Phil
David Metheny wrote:
The attached patch corrects a bug where group permission checking was
failing for a group with 221 members. The patch changes the logic for
group permission checking up a bit. Rather than get the user list from
the group, this patch gets the groups that the user is a member of. I'm
taking the assumption that groups probably have a large number of
members, but users usually have less number of groups. This also seemed
to help out a bit on the required memory allocations.
------------------------------------------------------------------------
*From:* [email protected]
[mailto:[email protected]] *On Behalf Of
*David Metheny
*Sent:* Thursday, March 12, 2009 11:32 AM
*To:* [email protected]
*Subject:* [Pvfs2-developers] PVFS2 group permission checking failing
forgroups with large number of members
I've run into an issue where permissions are failing during group
permission checking (PINT_check_group).
The example case is where userA is a member of groupA. groupA has 221
member users.
drwxrwx--- 1 userB groupA 4096 Mar 11 15:35 subdir
The PINT_check_mode uses the sysconf call to determine the size of the
buffer to use for lookups on groups and users. In this case, the
users/groups are stored in LDAP. The call here is
check_group_gr_buffer_size = sysconf(_SC_GETGR_R_SIZE_MAX). As a
default, if we can't make the sysconf call, it is being set to 1024
bytes. For RHEL4 systems in our environments, the sysconf is also
returning 1024.
In the above situation, 1024 isn't a large enough buffer to allocate all
of the member usernames associated with the group, and the getgrgid_r
function call returns an error.
As easy enough change would be to increase the buffer size used. Another
idea would be to change the call up a bit to get the groups that the
user is part of, and compare those groups against the group for the
file. Are there any thoughts on which approach to implement? I'm not
sure which would be faster in general, but in the environments I'm
seeing, users are in less groups, than groups have members. If we change
the logic on group checking, I'm assuming we would also want to
implement a larger buffer size for both the group and user lookups.
------------------------------------------------------------------------
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
