danny0405 commented on PR #20651:
URL: https://github.com/apache/hudi/pull/20651#issuecomment-6845276625
Reviewing head `300756f645b6963a63ef55ef4c728cecc841eda0`, I see a
correctness concern with positive N and a separate configuration propagation
gap.
**1. A matching suffix can suppress a necessary refresh and accept stale
file listings.**
The shortened hash is used for both the refresh decision and final
validation, so the relaxation applies even when no archival occurred. For
example, with N = 1:
```text
Server cached: [commit001, clean003]
Client fresh: [commit001, commit002, clean003]
^ completed late
```
Both sides hash only `clean003`, and the client's last instant is also
present on the server. `isLocalViewBehind()` therefore returns false: the
server skips sync and passes final validation. If its partition view was warmed
before `commit002` completed, it can return the base file from `001` even
though the client expects the updated file from `002`.
The existing
`TestTimelineViewRefresh.testRealViewRefreshesOutOfOrderCompletion` already
exercises essentially this real-file scenario with full hashing. Please extend
it to send N = 1 and verify that the returned base file is still `002`.
Increasing N moves this blind spot further back; it does not eliminate it.
The same issue applies to pending-compaction transitions outside the window:
a server can continue treating a completed compaction as pending, which affects
base-file and file-slice selection. In the opposite direction, an older client
can accept a server that has observed an unseen older replace commit. The new
`testInstantsBeforeHashedInstantsAreNotCompared` explicitly permits that
mismatch, but its mocked empty file listing does not establish that actual file
selection remains correct.
These are code-traced findings; I have not run a modified real-file test or
reproduced end-to-end data corruption. The default N = 0 preserves existing
behavior, but opt-in status and N <= the retention count do not establish
correctness when enabled. I would want real-file late-completion, compaction,
and restore coverage before treating this as a low-risk refresh optimization.
**2. The new config is not copied into the embedded server's remote-view
config.**
[`EmbeddedTimelineService#getRemoteFileSystemViewConfig()`](https://github.com/apache/hudi/blob/300756f645b6963a63ef55ef4c728cecc841eda0/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineService.java#L210-L225)
builds a fresh `FileSystemViewStorageConfig` and copies selected client
properties, but does not copy `getRemoteTimelineHashNumInstants()`.
Consequently, remote views constructed from this returned config still receive
the default 0, even when the writer configured a positive value. This is
particularly relevant because #20650 reports the embedded timeline server path.
Please propagate the setting through that builder and add a test through the
embedded-service configuration path. The new test that directly constructs
`RemoteHoodieTableFileSystemView` with a configured builder bypasses this gap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]