[
https://issues.apache.org/jira/browse/SENTRY-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293340#comment-16293340
]
Arjun Mishra edited comment on SENTRY-1944 at 12/15/17 10:16 PM:
-----------------------------------------------------------------
Patch not submitted because the getGroupByRoles() method is not replaced.
Instead there is a getGroupByRolesOptimized() method. Once reviewers approve I
will replace the old method.
Below is the test case that was done. And improvements is significant 98%
improvement
{nofromat}
Variable values:
delta_old = 29056111554 ms
delta_old_scaled = 72640278 ms/roles
delta_new = 314906857 ms
delta_new_scaled = 787267 ms/roles
diff = 28741204697 ms
change = 98.9162112885795%
{nofromat}
{noformat}
@Test
public void testGetGroupsByRolesPerformance() throws Exception {
String grantor = "grantor";
int nRoles = 200;
int nGroups = 100;
Set<String> groups = Sets.newHashSet();
Set<String> roles = Sets.newHashSet();
for(int i = 0; i < nGroups; i++) {
String groupName1 = "g1" + i;
groups.add(groupName1);
}
for(int i = 0; i < nRoles; i++) {
String roleName = "r" + i;
roles.add(roleName);
sentryStore.createRole(SEARCH, roleName, grantor);
sentryStore.alterRoleAddGroups(SEARCH, roleName, groups, grantor);
}
//No optimization
long before = System.nanoTime();
Set<String>groupResult1 = sentryStore.getGroupsByRoles(SEARCH, roles);
assertEquals(groups, groupResult1);
long after = System.nanoTime();
long delta_old = after - before;
long delta_old_scaled = delta_old / (2 * nRoles );
//With optimization
before = System.nanoTime();
Set<String>groupResult2 = sentryStore.getGroupsByRolesOptimized(SEARCH,
roles);
assertEquals(groups, groupResult2);
after = System.nanoTime();
long delta_new = after - before;
long delta_new_scaled = delta_new / (2 * nRoles );
long diff = delta_old - delta_new;
double change = (double)diff * 100 / delta_old;
LOGGER.info("Total time for getGroupNameByRoles is {} / {} nanoseconds",
delta_old_scaled, delta_new_scaled);
LOGGER.info("groupNameByRole change is {}%", change);
assertTrue("Is method optimized?", change > 0);
}
{noformat}
was (Author: arjunmishra13):
Patch not submitted because the getGroupByRoles() method is not replaced.
Instead there is a getGroupByRolesOptimized() method. Once reviewers approve I
will replace the old method.
Below is the test case that was done. And improvements is significant 98%
improvement
{nofromat}
Variable values:
delta_old = 29056111554 ms
delta_old_scaled = 72640278 ms/roles
delta_new = 314906857 ms
delta_new_scaled = 787267 ms/roles
diff = 28741204697 ms
change = 98.9162112885795%
{nofromat}
{noformat}
@Test
public void testGetGroupsByRolesPerformance() throws Exception {
String grantor = "grantor";
int nRoles = 200;
int nGroups = 100;
Set<String> groups = Sets.newHashSet();
Set<String> roles = Sets.newHashSet();
for(int i = 0; i < nGroups; i++) {
String groupName1 = "g1" + i;
groups.add(groupName1);
}
for(int i = 0; i < nRoles; i++) {
String roleName = "r" + i;
roles.add(roleName);
sentryStore.createRole(SEARCH, roleName, grantor);
sentryStore.alterRoleAddGroups(SEARCH, roleName, groups, grantor);
}
//No optimization
long before = System.nanoTime();
Set<String>groupResult1 = sentryStore.getGroupsByRoles(SEARCH, roles);
assertEquals(groups, groupResult1);
long after = System.nanoTime();
long delta_old = after - before;
long delta_old_scaled = delta_old / (2 * nRoles );
//With optimization
before = System.nanoTime();
Set<String>groupResult2 = sentryStore.getGroupsByRolesOptimized(SEARCH,
roles);
assertEquals(groups, groupResult2);
after = System.nanoTime();
long delta_new = after - before;
long delta_new_scaled = delta_new / (2 * nRoles );
long diff = delta_old - delta_new;
double change = (double)diff * 100 / delta_old;
LOGGER.info("Total time for getGroupNameByRoles is {} / {} nanoseconds",
delta_old_scaled, delta_new_scaled);
LOGGER.info("groupNameByRole change is {}%", change);
assertTrue("Is method optimized?", change > 0);
}
{noformat}
> Optimize DelegateSentryStore.getGroupsByRoles()
> -----------------------------------------------
>
> Key: SENTRY-1944
> URL: https://issues.apache.org/jira/browse/SENTRY-1944
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
> Affects Versions: 2.0.0
> Reporter: Alexander Kolbasov
> Assignee: Arjun Mishra
> Attachments: SENTRY-1944.01.patch
>
>
> When Solr is using Sentry server for authorization, it issues a lot of calls
> to {{getGroupsByRoles()}} function in {{DelegateSentryStore}}.
> This function isn't very efficient - it walks over each role in the set,
> obtains role by name, get groups for each role, and collects all group names
> into a set.
> It may be possible to optimize it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
