luoluoyuyu commented on PR #19258:
URL: https://github.com/apache/iotdb/pull/19258#issuecomment-6849366037
Self-review after another pass over the final patch. The design holds up,
and I believe the ownership scoping, the `Set`-based multi-directory cleanup
and the unclosed-TsFile guard are the real correctness fixes here. Two things
should change before merge, plus some follow-ups I will take in this PR.
### P1 – the fallback warning uses `%s` with an SLF4J logger
`PipeTsFileResourceManager.createHardLinkOrCopy()` logs
`LOG_FAILED_TO_CREATE_THE_HARDLINK_...` through `LOGGER.warn(msg, source,
target, linkFailure)`, but the message contains `%s`. SLF4J does not substitute
`%s`, so both paths are dropped and the log prints the raw template. Every
other `LOG_*` constant in `DataNodePipeMessages` uses `{}` (`{}` and `%s` both
normalize to `ARG` in the constant name, so only the hash suffix changes).
```java
// en
"Failed to create the hardlink of the file {} to {}, falling back to copying
the file";
// zh
"为文件 {} 创建到 {} 的硬链接失败,改为拷贝该文件";
```
The constant has to be renamed to the new hash suffix (`..._89FED96B`), and
the arguments passed as `(sourceFile, hardlinkOrCopiedFile)` so that the
sentence reads in the right direction.
### P1 – the copy fallback is not atomic
`FileUtils.deleteFileIfExist(target)` followed by
`FileUtils.copyFile(source, target)` writes the final path directly. A crash or
a full disk in the middle of the copy leaves a truncated file with a perfectly
valid pipe file name; the next pin only heals it after `createHardLink` fails
with `FileAlreadyExists` and `haveSameMD5()` reads both files in full. Copy to
`<target>.unfinished` first, rename atomically (`ATOMIC_MOVE`, falling back to
`REPLACE_EXISTING`), and delete the temporary file when the copy fails.
### P2 – resolved pipe paths should be canonical
The old helper built the directory from `getCanonicalPath()`, so the
returned path was always canonical. The new mapping is canonical only in the
data-directory-prefix branch; `getExistingPipeBaseDir()` and the anchor
fallback build from `file.getAbsoluteFile()`. With a symlinked data directory
(`/data -> /mnt/disk1`, `/tmp -> /private/tmp`) the same-scope check compares a
canonical path with a non-canonical one, and
`hardlinkOrCopiedFileToPipeTsFileResourceMap` (keyed by `file.getPath()`) can
hold two entries for one physical file, where the first entry to reach zero
deletes the file that the other one still references. Canonicalizing once in
`getPipeTsFileDir()` restores the old invariant.
### P2 – copy fallback cost and observability
When `<dataDir>/sequence` is a separate mount point, every TsFile is now
copied in full on the assigner/extractor path while the segment lock is held
(2x disk and doubled write I/O). Copying is still better than pinning forever,
but it has to be visible to operators: at least the (fixed) warning, ideally a
counter next to the other pipe resource metrics. A follow-up could resolve the
pipe directory onto the file's own mount point (`FileStore`) so that the
hardlink always succeeds and no copy is needed at all.
### P3 – the startup cleaner matches a little too broadly
`collectInterruptedStalePipeDirs()` now matches any directory whose name
*contains* `.startup-cleaning-`, while the old code required `<pipe hardlink
base dir>.startup-cleaning-`. Matching by prefix
(`getPipeHardlinkBaseDirName()` at the data-directory level and
`getPipeHardlinkTsFileDirName()` under the pipe directory) keeps the cleanup
scoped to directories that this cleaner created.
### P3 – test coverage
- `testUnclosedTsFileIsNotCopiedWhenHardlinkFails()` hardcodes
`<ROOT>/pipe/tsfile`. With `dn_data_dirs=target/data` and `ROOT` outside it,
every manager test only exercises the anchor fallback, so the main production
branch (data-directory prefix hit) has no manager-level coverage. Deriving the
path from `getHardlinkOrCopiedFileInPipeDir()` removes that layout coupling.
- A `@TestOnly` `setInstanceForTest(Supplier<String[]>)` / `reset()` on
`PipeTsFileDirPathResolver` would let tests point the resolver at real data
directories and cover (a) a pin of a TsFile under a configured data directory
and (b) `cleanPipeTsFileDir()` for a pipe spanning two data directories. The
`Set` cleanup is one of the core fixes and is currently only covered at the
mapping level.
- A "fails once, succeeds on retry" case (block the pipe directory, the pin
fails and leaves nothing behind, unblock, the pin succeeds with the complete
content) would guard the truncated-copy behaviour.
### Description and checklist
- Per-TsFile resolution itself is not a behaviour change: the old
`getPipeTsFileDirPath()` already resolved per file and produced the same
`<dataDir>/pipe/tsfile[/<pipeName>]` layout with the same flattened file names.
The actual fixes are the ownership scoping, the `Set`-based multi-directory
cleanup, self-link avoidance, the cross-device/unclosed fallback, following the
hot reload of `dn_data_dirs`, and no NPE for files outside the data
directories. I will rewrite the description (including the two-disk example
table, whose behaviour is actually unchanged) around those points.
- `tested in a multi-disk cluster` is still unchecked. I will do a manual
run before merge: `dn_data_dirs=/data1,/data2`, write and flush TsFiles on both
disks, `drop pipe`, restart, and verify that no `pipe/tsfile` files are left
behind on either disk.
I will push the P1 fixes and the canonicalization as a follow-up commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]