[
https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024268#comment-16024268
]
Alexander Kolbasov commented on SENTRY-1784:
--------------------------------------------
Here is the original code:
{code}
List<K> getAllUpdatesFrom(long seqNum) throws Exception {
// No newer updates available than the requested one.
long curSeqNum = deltaRetriever.getLatestDeltaID();
if (seqNum > curSeqNum) {
return Collections.emptyList();
}
// Checks if there is newer deltas exists in the persistent storage.
// If there is, returns a list of delta updates.
if ((seqNum != SentryStore.INIT_CHANGE_ID) &&
deltaRetriever.isDeltaAvailable(seqNum)) {
Collection<K> deltas = deltaRetriever.retrieveDelta(seqNum);
if (!deltas.isEmpty()) {
return new LinkedList<>(deltas);
}
}
// Otherwise, a complete snapshot will be returned.
List<K> retVal = new LinkedList<>();
retVal.add(imageRetriever.retrieveFullImage());
return retVal;
}
{code}
Let's see what happens when seqNum is zero.
{code}
long curSeqNum = deltaRetriever.getLatestDeltaID();
if (seqNum > curSeqNum) {
return Collections.emptyList();
}
{code}
We do not have any deltas yet, so curSeqNum should be zero. Condition seqNum >
curSeqNum is false, so we skip this.
{code}
if ((seqNum != SentryStore.INIT_CHANGE_ID) &&
deltaRetriever.isDeltaAvailable(seqNum)) {
...
{code}
In this case seqNum (0) is equal to INIT_CHANGE_ID(0) so this condition is
skipped as well
{code}
// Otherwise, a complete snapshot will be returned.
List<K> retVal = new LinkedList<>();
retVal.add(imageRetriever.retrieveFullImage());
return retVal;
{code}
So we do get the full image here. What am I missing?
> DBUpdateForwarder returns empty update list to HDFS instead of full update
> --------------------------------------------------------------------------
>
> Key: SENTRY-1784
> URL: https://issues.apache.org/jira/browse/SENTRY-1784
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
> Affects Versions: sentry-ha-redesign
> Reporter: Na Li
> Assignee: Na Li
> Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1784.001-sentry-ha-redesign.patch
>
>
> When HDFS first asks Sentry for update, the request changeID is 1. If there
> is no entries in permission change table (current changeID is 0), empty
> update list is returned to HDFS because the current code behavior is that no
> update is returned if request changeID is larger than the current chaneID.
> Once this problem is fixed (returning full update when request changeID = 1,
> and current changeID = 0), another issue of the current code is that after a
> full update is sent to client, and there is no entries in permission change
> table (current changeID is 0), the next request changeID is still 1,
> so full update is returned for the second time. Full update will be sent for
> each request until permission change table has at least one entry.
> The desired behavior is for HDFS to get full update when it sends the first
> request to Sentry. Once it gets full update, the following requests should
> get delta updates unless some updates are purged before they are sent to HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
