[
https://issues.apache.org/jira/browse/SENTRY-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1937:
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
Here is the test that I used to measure perf difference - it was placed as part
of TestSentryStore
{code}
public void testListRolePerformance() throws Exception {
String grantor = "g1";
int nroles = 200;
int shared_roles = 100;
int ngroups = 100;
Set<String> groups1 = new HashSet<>(ngroups);
Set<TSentryGroup> tGroups1 = new HashSet<>(ngroups);
Set<String> groups2 = new HashSet<>(ngroups);
Set<TSentryGroup> tGroups2 = new HashSet<>(ngroups);
Set<String>allGroups = new HashSet<>();
Set<TSentryGroup>alltGroups = new HashSet<>();
for (int i = 0; i < ngroups; i++) {
String groupName1 = "g1" + i;
String groupName2 = "g2" + i;
groups1.add(groupName1);
TSentryGroup tg1 = new TSentryGroup(groupName1);
TSentryGroup tg2 = new TSentryGroup(groupName2);
tGroups1.add(tg1);
groups2.add(groupName2);
tGroups2.add(tg2);
alltGroups.add(tg1);
alltGroups.add(tg2);
allGroups.add(groupName1);
allGroups.add(groupName2);
}
for (int i = 0; i < nroles; i++) {
String roleNameg1 = "g1r" + i;
String roleNameg2 = "g2r" + i;
sentryStore.createSentryRole(roleNameg1);
sentryStore.createSentryRole(roleNameg2);
sentryStore.alterSentryRoleAddGroups(grantor, roleNameg1, tGroups1);
sentryStore.alterSentryRoleAddGroups(grantor, roleNameg2, tGroups2);
}
for (int i = 0; i < shared_roles; i++) {
String roleName = "grs" + i;
sentryStore.createSentryRole(roleName);
sentryStore.alterSentryRoleAddGroups(grantor, roleName, alltGroups);
}
long before = System.nanoTime();
Set<TSentryRole> resultRoles1 =
sentryStore.getTSentryRolesByGroupName(allGroups, false);
long after = System.nanoTime();
long delta_old = after - before;
long delta_old_scaled = delta_old / (2 * nroles + shared_roles);
before = System.nanoTime();
Set<TSentryRole> resultRoles2 =
sentryStore.getTSentryRolesByGroupNameOptimized(allGroups, false);
//Set<TSentryRole> resultRoles =
sentryStore.getTSentryRolesByGroupNameOptimized(Sets.newHashSet(group1, group2),
// false);
after = System.nanoTime();
long delta_new = after - before;
long delta_new_scaled = delta_new / (2 * nroles + shared_roles);
long diff = delta_old - delta_new;
double change = (double)diff * 100 / delta_old;
LOGGER.info("Total time for getTSentryRolesByGroupName is {} / {}
nanoseconds", delta_old_scaled, delta_new_scaled);
LOGGER.info("getTSentryRolesByGroupName change is {}%", change);
before = System.nanoTime();
Set<TSentryRole> resultRoles3 =
sentryStore.getTSentryRolesByGroupNameOptimized3(allGroups, false);
after = System.nanoTime();
delta_new = after - before;
delta_new_scaled = delta_new / (2 * nroles + shared_roles);
diff = delta_old - delta_new;
change = (double)diff * 100 / delta_old;
LOGGER.info("Total time for getTSentryRolesByGroupName is {} / {}
nanoseconds for set", delta_old_scaled, delta_new_scaled);
LOGGER.info("getTSentryRolesByGroupName change is {}% for set", change);
assertEquals(2 * nroles + shared_roles, resultRoles1.size());
assertEquals(resultRoles1, resultRoles2);
assertEquals(resultRoles1, resultRoles3);
}
{code}
It is important to disable DEBUG logging for the test.
> Optimize performance for listing sentry roles by group name
> -----------------------------------------------------------
>
> Key: SENTRY-1937
> URL: https://issues.apache.org/jira/browse/SENTRY-1937
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
> Affects Versions: 2.0.0
> Reporter: Alexander Kolbasov
> Assignee: Alexander Kolbasov
> Fix For: 2.0.0
>
> Attachments: SENTRY-1937.01.patch
>
>
> It seems that the most common request is listing sentry roles by group name,
> so this is a very hot path. It is important to optimize its performance,
> especially in the presence of many roles.
> [~arjunmishra13] FYI.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
