goutamadwant opened a new pull request, #19946:
URL: https://github.com/apache/hudi/pull/19946

   ### Describe the issue this Pull Request addresses
   
   Closes #19360.
   
   A remote filesystem view retains its client timeline between requests. When 
the server already contains that timeline plus newer instants, unequal full 
hashes repeatedly trigger a sync that reloads the same timeline and discards 
cached partition views.
   
   ### Summary and Changelog
   
   Implement the exact-prefix approach proposed and discussed in the issue. On 
a full-hash mismatch, require a valid client boundary present on the server and 
compare the server timeline through that boundary with the client hash. Skip 
refresh only when the hashes match.
   
   Keep the existing filtering and hashing rules, full-hash equality check, 
synchronization, and conservative refresh for divergent or archived prefixes. 
The same decision serves the initial refresh and post-request consistency 
checks.
   
   Add HTTP regressions for exact extensions, boundary validation, archival, 
out-of-order completion, action/state differences, filtering, and changes 
during request handling. Add real file-backed view tests for repeated remote 
file-slice requests and refreshing before returning a newer base file.
   
   ### Impact
   
   Avoid unnecessary timeline reloads and partition-view rebuilds when the 
server is a proven extension of the client timeline. No request protocol, 
public API, configuration, or storage-format changes.
   
   In the local file-backed regression, 32 remote file-slice requests across 
eight warmed partitions went from 32 syncs, 32 timeline reloads, and 64 
directory listings to zero of each, with the same expected file selections. 
This is operation-count evidence, not an end-to-end workload speedup 
measurement.
   
   ### Risk Level
   
   High: filesystem-view refresh decisions affect file selection. The 
optimization retains the complete prefix hash, including requested time, 
action, and state, rather than using the latest timestamp alone. Divergence and 
out-of-order completion controls verify that necessary refreshes remain in 
place.
   
   Focused validation passed all 14 tests in `TestTimelineViewRefresh` and 
`TestRequestHandler`.
   
   Broader validation passed all 96 timeline-service tests, including 54 remote 
filesystem-view tests, plus 51 shared filesystem-view tests. Checkstyle and RAT 
passed. These local tests do not replace a production HDFS/Spark/Flink 
row-level workload replay.
   
   ### Documentation Update
   
   None.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to