ColinLeeo opened a new issue, #1054:
URL: https://github.com/apache/tsfile/issues/1054

   ## Motivation
   
   Users run TsFile on NAS appliances and on-premises storage exposed through 
NFS mounts. Nothing in the repository documents or tests NFS behavior, and a 
few implementation details interact with NFS semantics in ways that have never 
been verified. We should either confirm that reading works with a documented 
configuration, or fix what does not.
   
   ## Current implementation (verified)
   
   - **Java read**: `TsFileSequenceReader` obtains its input through 
`FSFactoryProducer.getFileInputFactory()`; `LocalTsFileInput` wraps 
`FileChannel.open(path, READ)`.
   - **C++ read**: `TsFileReader::open(path)` creates a 
`LocalRandomAccessReadFile`. The default read backend is 
`FileReadBackend::PREAD`; `MMAP` is opt-in through 
`common::set_file_read_backend`, the C API `tsfile_set_file_read_backend`, and 
`set_file_read_backend` in the Python binding. With `AUTO`, the file is mmapped 
when possible and falls back to `pread`.
   - `LocalRandomAccessReadFile::generation()` fingerprints a file by 
`(st_size, st_mtime_ns)` and rejects the descriptor if `st_dev`/`st_ino` 
changed. The dataset index (`MappedDatasetIndex`) is `mmap`ed.
   - No advisory or OS file locking is used on either path (no `flock`, `fcntl` 
locks, or `FileLock`).
   - The C++ writer and `RestorableTsFileIOWriter` truncate and rewrite the 
metadata tail when reopening a file for append or recovery.
   
   ## Risks that need verification (hypotheses, not confirmed defects)
   
   1. **mmap backend over NFS**: page-cache coherence when another client 
appends or truncates, and `SIGBUS` if the file shrinks while mapped.
   2. **`generation()` under NFS attribute caching** (`actimeo`): a stale 
`mtime`/size could keep a cached dataset index valid after a file was 
rewritten. Verify both file-replacement and in-place-append scenarios.
   3. **Restorable writer recovery**: truncate + rewrite over NFS, and whether 
the reopened file becomes visible consistently to readers on other clients.
   4. **Java `FileChannel` on NFS**, including interrupt handling 
(`LocalTsFileInput` catches `ClosedByInterruptException` and turns it into 
`-1`).
   5. **Symlinks / hard links** and `st_ino` stability across NFS clients.
   6. **Read performance**: random `pread` over NFS vs mmap; which backend 
should be recommended.
   
   ## Proposed work
   
   - Add a verification harness (script or gated integration test) that runs a 
read matrix against a directory supplied by an environment variable such as 
`TSFILE_NFS_TEST_DIR`, covering:
     - read-only mount, default/PREAD backend and MMAP backend;
     - file written on another host, then read locally;
     - file appended on another host, then read locally;
     - file replaced while a reader is open;
     - symlinked and hard-linked paths.
   - Run the same matrix for Java and C++ (and Python if inexpensive).
   - Fix defects found, or document the limitation and the supported 
configuration (for example: force `PREAD` on NFS, avoid long `actimeo` for 
files that are rewritten).
   - Document the support statement in the read/write documentation and the 
per-language guidance.
   
   ## Acceptance criteria
   
   - A documented statement for each language: which read backends are 
supported over NFS, recommended settings, and known limitations.
   - An automated or manually invocable test covering the scenarios above, 
runnable by pointing at an NFS mount.
   - Any incompatibility found is either fixed or documented with a clear 
workaround.
   - No regression to local-file read behavior or performance.
   
   ## Related
   
   - #903 — mmap as an optional local read backend
   - #930 — remote random-access reading for TsFile
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to