zhaoyudi-creator commented on issue #19360: URL: https://github.com/apache/hudi/issues/19360#issuecomment-5089337566
> there might be some space for optimization for the fs view loading on server, we did try to omit some actions like clean for the timeline comparision before sync and cause data loss: [#6179](https://github.com/apache/hudi/pull/6179), might need to think through the strategy to refresh the fs view. Wanted to confirm the intended semantics of RequestHandler#isLocalViewBehind: is a sync() / reloadActiveTimeline() only needed when the client is ahead of the server (the client knows a completed instant the server hasn't loaded yet)? I agree with the rationale behind #6179 introducing the timeline hash — under async table services, completion order may differ from instant time, so judging staleness from the latest commit alone is unreliable. But the hash comparison is directionless: a mismatch can also mean the server is already ahead of the client (it merely has newer clean / compaction instants). In that case sync() produces an empty diff (read 0 instants) — a no-op reload; and with incr.timeline.sync.enable defaulting to false, it becomes a full clear() + rebuild of the whole view. Proposed fix (keeping the hash, not falling back to the latest commit): on a hash mismatch, truncate the server timeline to the client's lastInstant and compare hashes again — if (!HoodieTimeline.INVALID_INSTANT_TS.equals(lastKnownInstantFromClient) && localTimeline.findInstantsBeforeOrEquals(lastKnownInstantFromClient) .getTimelineHash().equals(timelineHashFromClient)) { return false; // server is ahead, not behind } return true; Would like to confirm: is this direction sound? Is there any case where the hashes mismatch, the server is ahead, yet a reload is still required? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
