ColinLeeo opened a new issue, #1054:
URL: https://github.com/apache/tsfile/issues/1054
## Motivation
Users run TsFile on NAS appliances and on-premises storage exposed through
NFS mounts. Nothing in the repository documents or tests NFS behavior, and a
few implementation details interact with NFS semantics in ways that have never
been verified. We should either confirm that reading works with a documented
configuration, or fix what does not.
## Current implementation (verified)
- **Java read**: `TsFileSequenceReader` obtains its input through
`FSFactoryProducer.getFileInputFactory()`; `LocalTsFileInput` wraps
`FileChannel.open(path, READ)`.
- **C++ read**: `TsFileReader::open(path)` creates a
`LocalRandomAccessReadFile`. The default read backend is
`FileReadBackend::PREAD`; `MMAP` is opt-in through
`common::set_file_read_backend`, the C API `tsfile_set_file_read_backend`, and
`set_file_read_backend` in the Python binding. With `AUTO`, the file is mmapped
when possible and falls back to `pread`.
- `LocalRandomAccessReadFile::generation()` fingerprints a file by
`(st_size, st_mtime_ns)` and rejects the descriptor if `st_dev`/`st_ino`
changed. The dataset index (`MappedDatasetIndex`) is `mmap`ed.
- No advisory or OS file locking is used on either path (no `flock`, `fcntl`
locks, or `FileLock`).
- The C++ writer and `RestorableTsFileIOWriter` truncate and rewrite the
metadata tail when reopening a file for append or recovery.
## Risks that need verification (hypotheses, not confirmed defects)
1. **mmap backend over NFS**: page-cache coherence when another client
appends or truncates, and `SIGBUS` if the file shrinks while mapped.
2. **`generation()` under NFS attribute caching** (`actimeo`): a stale
`mtime`/size could keep a cached dataset index valid after a file was
rewritten. Verify both file-replacement and in-place-append scenarios.
3. **Restorable writer recovery**: truncate + rewrite over NFS, and whether
the reopened file becomes visible consistently to readers on other clients.
4. **Java `FileChannel` on NFS**, including interrupt handling
(`LocalTsFileInput` catches `ClosedByInterruptException` and turns it into
`-1`).
5. **Symlinks / hard links** and `st_ino` stability across NFS clients.
6. **Read performance**: random `pread` over NFS vs mmap; which backend
should be recommended.
## Proposed work
- Add a verification harness (script or gated integration test) that runs a
read matrix against a directory supplied by an environment variable such as
`TSFILE_NFS_TEST_DIR`, covering:
- read-only mount, default/PREAD backend and MMAP backend;
- file written on another host, then read locally;
- file appended on another host, then read locally;
- file replaced while a reader is open;
- symlinked and hard-linked paths.
- Run the same matrix for Java and C++ (and Python if inexpensive).
- Fix defects found, or document the limitation and the supported
configuration (for example: force `PREAD` on NFS, avoid long `actimeo` for
files that are rewritten).
- Document the support statement in the read/write documentation and the
per-language guidance.
## Acceptance criteria
- A documented statement for each language: which read backends are
supported over NFS, recommended settings, and known limitations.
- An automated or manually invocable test covering the scenarios above,
runnable by pointing at an NFS mount.
- Any incompatibility found is either fixed or documented with a clear
workaround.
- No regression to local-file read behavior or performance.
## Related
- #903 — mmap as an optional local read backend
- #930 — remote random-access reading for TsFile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]