hgudladona commented on issue #13356: URL: https://github.com/apache/hudi/issues/13356#issuecomment-2963168969
Here is what we have found, turns out the reason for all the refreshes is the mismatch of the timeline hash. There is no scenario where the timeline has will match but the local timeline is ahead of the instant reported by client [here](https://github.com/apache/hudi/blob/master/hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/RequestHandler.java#L651-L658). AFAICT, what this means is the code below is dead. You should consider renaming the function - as even if the timeline server view is ahead, the hash mismatch will cause it to be treated as behind. ``` // As a safety check, even if hash is same, ensure instant is present return !localTimeline.containsOrBeforeTimelineStarts(lastKnownInstantFromClient); ``` Realistically, at least in our case, without fixing the logic in `isLocalViewBehind` regardless of the locking mechanism we will end up doing many refreshes in async mode. To fix this behavior, We may need to introduce a tracker to skip subsequent refreshes upon a first refresh for the same timeline hash from the client Please advise how to proceed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
