[
https://issues.apache.org/jira/browse/SENTRY-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1824:
---------------------------------------
Description:
The suggestion came from [~lina.li] during the review for SENTRY-1821.
We have several places where we rely on transactions to support
read/modify/write semantics for incrementing IDs. This works but using DB
support is rather expensive and we can user in-core serializations to help with
this at least within a single node and rely on DB for multi-node
synchronization.
This isn't much of a problem for path updates since they are driven by
HMSFollower which usually runs on a single leader node, but permission updates
originate from clients directly and may be highly concurrent.
We are internally serializing all permissions update anyway, so doing partial
serialization on every node helps.
The patch serializes all calls that affect permission updates.
was:
The suggestion came from [~lina.li] during the review for SENTRY-1821.
We have several places where we rely on transactions to support
read/modify/write semantics for incrementing IDs. This works but using DB
support is rather expensive and we can user in-core serializations to help with
this at least within a single node and rely on DB for multi-node
synchronization.
1) The DeltaTransactionBlock gets latest changeID and then increases it for
this update.
Since changeID is the primary key of persisted update, when two or more
transactions are
updating changeID at the same time, only one transaction succeeds, and the
other transactions
will fail, and go through retry. It is quite expensive, and cause performance
issue.
When more clients are making updates, transaction may fail to commit after
retries.
2)Permission updates are the main issue, getting worse with more clients.
Path update is from one thread in HMSFollower, independent on the number of
clients, so it is not a big issue.
3) Adding synchronization to serilize the updates can minimize the chance a
transaction fails
4) Therefore, synchronizing all permission updates. In this way, when more
than one threads are updating the
permission, only one thread will update changeID. When it finishes, the next
thread on the same Sentry server
will make update. As a result, no transaction will fail if there is only one
Sentry server. When there are more
Sentry servers, the collision will happen among the Sentry servers, not
depending on how many clients are
updating Sentry, so it is more scalable.
> SentryStore may serialize transactions that rely on unique key
> --------------------------------------------------------------
>
> Key: SENTRY-1824
> URL: https://issues.apache.org/jira/browse/SENTRY-1824
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
> Affects Versions: sentry-ha-redesign
> Reporter: Alexander Kolbasov
> Assignee: Na Li
> Labels: sentry-ha
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1824.001-sentry-ha-redesign.patch,
> SENTRY-1824.002-sentry-ha-redesign.patch,
> SENTRY-1824.003-sentry-ha-redesign.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> The suggestion came from [~lina.li] during the review for SENTRY-1821.
> We have several places where we rely on transactions to support
> read/modify/write semantics for incrementing IDs. This works but using DB
> support is rather expensive and we can user in-core serializations to help
> with this at least within a single node and rely on DB for multi-node
> synchronization.
> This isn't much of a problem for path updates since they are driven by
> HMSFollower which usually runs on a single leader node, but permission
> updates originate from clients directly and may be highly concurrent.
> We are internally serializing all permissions update anyway, so doing partial
> serialization on every node helps.
> The patch serializes all calls that affect permission updates.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
