tkhurana opened a new pull request, #2590: URL: https://github.com/apache/phoenix/pull/2590
The forwarder copied a fallback-cluster log file straight onto its final <ts>_<origin>.plog name on the peer via FileUtil.copy (overwrite=false). Because FileUtil.copy publishes the file at that replay-eligible name before close() completes, the standby replay consumer could pick up the half-written file, force-recover the HDFS lease, and cause the forwarder's close() to fail with LeaseExpiredException. That threw out of processFile before the STORE_AND_FORWARD -> SYNC_AND_FORWARD throughput check, delaying the mode transition. Fix: copy into a per-shard .staging subdirectory (<shard>/.staging/<ts>_<origin>.plog, overwrite=true to reclaim orphans) and atomically rename it up to the final .plog in the shard directory once fully written. The staged file keeps its real .plog name but is invisible to replay because every listing gates on FileStatus.isFile(), which skips the subdirectory and anything inside it. Replay only ever sees the atomically published file. Delivery remains at-least-once (unchanged); replay idempotency makes a re-publish safe. Tests: forwarder staging/rename/retry/orphan cases in ReplicationLogDiscoveryForwarderTest; .staging-subdirectory exclusion across the listing entry points in ReplicationLogTrackerTest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
