[
https://issues.apache.org/jira/browse/SENTRY-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612363#comment-16612363
]
Na Li commented on SENTRY-2185:
-------------------------------
[~spena] Kalyan did some inve3stigation in SENTRY-2249.
The reason I think saving full path image takes a long time is because both
AUTHZ_PATHS_MAPPING and AUTHZ_PATH use identity value generation strategy,
therefore dataNucleus does not allow inserting entries in batches. What could
improve the performance is to not use identity value generation strategy in at
least AUTHZ_PATH. That will allow batch insert into DB. The improvement could
be significant if a table has many paths.
> Performance Issue: Saving MAuthzPathsMapping should be done in batch for path
> full snapshot
> -------------------------------------------------------------------------------------------
>
> Key: SENTRY-2185
> URL: https://issues.apache.org/jira/browse/SENTRY-2185
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Affects Versions: 2.1.0
> Reporter: Na Li
> Assignee: Na Li
> Priority: Major
>
> From the code below, for each MAuthzPathsMapping, there is a query to save
> each MAuthzPathsMapping instance. Need to save the changes in batch.
> {code:java}
> public void persistFullPathsImage(final Map<String, Collection<String>>
> authzPaths,
> final long notificationID) throws Exception {
> tm.executeTransactionWithRetry(
> pm -> {
> pm.setDetachAllOnCommit(false); // No need to detach objects
> deleteNotificationsSince(pm, notificationID + 1);
> // persist the notidicationID
> pm.makePersistent(new MSentryHmsNotification(notificationID));
> // persist the full snapshot
> long snapshotID = getCurrentAuthzPathsSnapshotID(pm);
> long nextSnapshotID = snapshotID + 1;
> pm.makePersistent(new MAuthzPathsSnapshotId(nextSnapshotID));
> LOGGER.info("Attempting to commit new HMS snapshot with ID =
> {}", nextSnapshotID);
> for (Map.Entry<String, Collection<String>> authzPath :
> authzPaths.entrySet()) {
> pm.makePersistent(new MAuthzPathsMapping(nextSnapshotID,
> authzPath.getKey(), authzPath.getValue()));
> }
> return null;
> });
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
