[
https://issues.apache.org/jira/browse/SENTRY-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li updated SENTRY-1824:
--------------------------
Description:
The suggestion came from [~lina.li] during the review for SENTRY-1821.
We have several places where we rely on transactions to support
read/modify/write semantics for incrementing IDs. This works but using DB
support is rather expensive and we can user in-core serializations to help with
this at least within a single node and rely on DB for multi-node
synchronization.
1) The DeltaTransactionBlock gets latest changeID and then increases it for
this update.
Since changeID is the primary key of persisted update, when two or more
transactions are
updating changeID at the same time, only one transaction succeeds, and the
other transactions
will fail, and go through retry. It is quite expensive, and cause performance
issue.
When more clients are making updates, transaction may fail to commit after
retries.
2)Permission updates are the main issue, getting worse with more clients.
Path update is from one thread in HMSFollower, independent on the number of
clients, so it is not a big issue.
3) Adding synchronization to serilize the updates can minimize the chance a
transaction fails
4) Therefore, synchronizing all permission updates. In this way, when more
than one threads are updating the
permission, only one thread will update changeID. When it finishes, the next
thread on the same Sentry server
will make update. As a result, no transaction will fail if there is only one
Sentry server. When there are more
Sentry servers, the collision will happen among the Sentry servers, not
depending on how many clients are
updating Sentry, so it is more scalable.
was:
The suggestion came from [~lina.li] during the review for SENTRY-1821.
We have several places where we rely on transactions to support
read/modify/write semantics for incrementing IDs. This works but using DB
support is rather expensive and we can user in-core serializations to help with
this at least within a single node and rely on DB for multi-node
synchronization.
> SentryStore may serialize transactions that rely on unique key
> --------------------------------------------------------------
>
> Key: SENTRY-1824
> URL: https://issues.apache.org/jira/browse/SENTRY-1824
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
> Affects Versions: sentry-ha-redesign
> Reporter: Alexander Kolbasov
> Assignee: Na Li
> Labels: sentry-ha
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1824.001-sentry-ha-redesign.patch,
> SENTRY-1824.002-sentry-ha-redesign.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> The suggestion came from [~lina.li] during the review for SENTRY-1821.
> We have several places where we rely on transactions to support
> read/modify/write semantics for incrementing IDs. This works but using DB
> support is rather expensive and we can user in-core serializations to help
> with this at least within a single node and rely on DB for multi-node
> synchronization.
> 1) The DeltaTransactionBlock gets latest changeID and then increases it for
> this update.
> Since changeID is the primary key of persisted update, when two or more
> transactions are
> updating changeID at the same time, only one transaction succeeds, and the
> other transactions
> will fail, and go through retry. It is quite expensive, and cause
> performance issue.
> When more clients are making updates, transaction may fail to commit after
> retries.
> 2)Permission updates are the main issue, getting worse with more clients.
> Path update is from one thread in HMSFollower, independent on the number of
> clients, so it is not a big issue.
> 3) Adding synchronization to serilize the updates can minimize the chance a
> transaction fails
> 4) Therefore, synchronizing all permission updates. In this way, when more
> than one threads are updating the
> permission, only one thread will update changeID. When it finishes, the next
> thread on the same Sentry server
> will make update. As a result, no transaction will fail if there is only one
> Sentry server. When there are more
> Sentry servers, the collision will happen among the Sentry servers, not
> depending on how many clients are
> updating Sentry, so it is more scalable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
