[
https://issues.apache.org/jira/browse/SENTRY-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037796#comment-16037796
]
Na Li commented on SENTRY-1792:
-------------------------------
The possible cause is when multiple threads are updating perm change table
(multiple requests for changes are issued through hive, and go to sentry
service, and they can be in its own thread), thread_a get changeID_a from DB,
processing entry, and then save it as change entry_A in DB. thread_b get
changeID_b from DB, processing entry, and then save it as change entry_B in DB.
If thread_A gets changeID_b before thread_B gets changeID_b, but finishes after
thread_B, then we can see the hole at short time interval. eventually, we don't
see the hole. Vamsee and I checked the DB changeID, they are continuous even
after several days run.
fix is to extract continuous list from non-continuous list starting with
requested ID.
Examples of the change list. the temporary hole is always at the end of the list
2017-06-05 14:02:42,489 ERROR
org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm
delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and
expected size = 7, from changeID: 1078683. The table may get corrupted. perm
changeID list: 1078683,1078684,1078685,1078686,1078689
2017-06-05 14:03:01,669 ERROR
org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm
delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and
expected size = 6, from changeID: 1078958. The table may get corrupted. perm
changeID list: 1078958,1078959,1078960,1078961,1078963
2017-06-05 14:03:14,645 ERROR
org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm
delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 8 and
expected size = 9, from changeID: 1079144. The table may get corrupted. perm
changeID list: 1079144,1079145,1079146,1079147,1079148,1079149,1079150,1079152
2017-06-05 14:03:26,735 ERROR
org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm
delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 7 and
expected size = 8, from changeID: 1079324. The table may get corrupted. perm
changeID list: 1079324,1079325,1079326,1079327,1079328,1079329,1079331
> Too many full snapshot for Permission change request
> ----------------------------------------------------
>
> Key: SENTRY-1792
> URL: https://issues.apache.org/jira/browse/SENTRY-1792
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Reporter: Na Li
>
> Sentry server returns full permission snapshot almost every 10 seconds. The
> desired behavior is to reduce the chance of getting full snapshot
> significantly.
> From debug message, we can see the returned change list is empty.
> That causes Sentry to send back full snapshot instead of delta changes. It
> has performance penalty because it takes much longer to get a full snapshot
> than getting the delta changes.
> 2017-06-05 14:00:21,030 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1076942, curSeqNum:1076948 ; type:
> PermImageRetriever
> 2017-06-05 14:00:31,905 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1076949, curSeqNum:1077064 ; type:
> PermImageRetriever
> 2017-06-05 14:01:00,041 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1077354, curSeqNum:1077361 ; type:
> PermImageRetriever
> 2017-06-05 14:01:25,735 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1077630, curSeqNum:1077636 ; type:
> PermImageRetriever
> 2017-06-05 14:01:42,197 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1077820, curSeqNum:1077826 ; type:
> PermImageRetriever
> 2017-06-05 14:01:53,491 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1077827, curSeqNum:1077962 ; type:
> PermImageRetriever
> 2017-06-05 14:02:09,472 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1078147, curSeqNum:1078155 ; type:
> PermImageRetriever
> 2017-06-05 14:02:21,527 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1078300, curSeqNum:1078305 ; type:
> PermImageRetriever
> 2017-06-05 14:02:35,761 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1078483, curSeqNum:1078489 ; type:
> PermImageRetriever
> 2017-06-05 14:02:50,079 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1078683, curSeqNum:1078686 ; type:
> PermImageRetriever
> 2017-06-05 14:03:09,889 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1078958, curSeqNum:1078961 ; type:
> PermImageRetriever
> 2017-06-05 14:03:21,641 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1079144, curSeqNum:1079150 ; type:
> PermImageRetriever
> 2017-06-05 14:03:32,600 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1079324, curSeqNum:1079331 ; type:
> PermImageRetriever
> 2017-06-05 14:03:46,223 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1079537, curSeqNum:1079546 ; type:
> PermImageRetriever
> 2017-06-05 14:03:59,870 INFO org.apache.sentry.hdfs.DBUpdateForwarder: ####
> GetAllUpdatesFrom return full image with isDeltaAvailable:true,
> isDeltaEmpty:true, reqSeqNum:1079762, curSeqNum:1079769 ; type:
> PermImageRetriever
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
