mrproliu opened a new pull request, #1146:
URL: https://github.com/apache/skywalking-banyandb/pull/1146
## Why
On cold/idle data nodes, `take snapshot` (backup) and FODC data inspection
(`CollectDataInfo` / `inspectAll`) intermittently failed with a nil-index
panic
and bluge `series index open failed (exclusive lock)` errors.
**Root cause:** both paths forced *every* segment open to read it.
Re-opening an
**idle-closed** cold segment re-acquired the bluge writer's exclusive
directory
lock (`bluge.pid`) and raced the idle-reclaimer, producing the lock churn
and the
nil-index panic. Backup and inspection are read-only and should never
reopen a
quiescent segment.
## What
Backup and inspection no longer reopen idle-closed segments.
**Backup (`TakeFileSnapshot`)** — per-segment dispatch under the segment
lock:
- **Open** segment → snapshot via its live series index + shard tables,
pinned by a reference.
- **Closed** segment → recursive **hard-link** of its quiescent on-disk
directory, excluding transient/non-current artifacts (`bluge.pid` lock,
`failed-parts/`, `external-segment-temp/`, `*.tmp`) so
the result matches what the open path produces.
**Inspection (`CollectDataInfo`)** — reads closed segments straight from
disk:
- Shard part stats via the latest `.snp` manifest → part `metadata.json`
(`CollectClosedShardInfo`), so orphan parts are excluded just like the
in-memory snapshot.
- Series-index doc count via a **read-only** bluge `OpenReader`
(`ReadOnlyDocCount`) that never takes the exclusive lock.
- All disk reads are best-effort: a segment concurrently removed by
retention reports zero instead of panicking.
**Supporting changes:**
- Merged the two segment selectors into a single
`SelectSegments(timeRange, reopenClosed bool)` (consistent with the existing
`segments(ctx, reopenClosed)`): `true` = query (reopen + refresh idle
timer), `false` = read-only stats (pin only if already open, no reopen, no
refresh).
- Extracted `storage.ReadSnapshotPartNames` as the single reader for the
`.snp` part manifest, reused by inspection and by each data type's
`readSnapshot`.
- `fs.CreateHardLink` now prunes a filtered directory's whole subtree via
`filepath.SkipDir`.
## Behavior notes
- For a **closed** segment, inspection reports an empty
`InvertedIndexInfo` (stream) / `SidxInfo` (trace), because reading them would
reopen a bluge index — the exact churn this change avoids. These
are populated only for open segments; documented at the call sites.
- The closed-segment backup hard-links all on-disk part directories
(including not-yet-GC'd orphans); they are harmless and cleaned up by the
recovery path on next open.
## Testing
- Unit + integration tests driving the real controller flow (`OpenTSDB` →
`CreateSegmentIfNotExist` → `IndexDB().Insert` → `closeIdleSegments`):
open/closed/mixed/deleted/empty-segment snapshots,
restorability via read-only doc count, hard-link inode sharing,
transient-artifact exclusion (incl. `bluge.pid`), and concurrent
snapshot-vs-reclaimer (`-race`).
- Full `SelectSegments` matrix: `reopenClosed` × {closed, open, in-use,
non-overlapping} — asserting refCount transitions, reopen vs. no-reopen, and
idle-timer refresh semantics.
- Real on-disk part stats for measure/stream/trace, verifying the storage
layer reads the shared `metadata.json` totals and excludes orphan parts.
- `make lint` clean; full storage suite passes under `-race`.
- [ ] If this pull request closes/resolves/fixes an existing issue, replace
the issue number. Fixes apache/skywalking#<issue number>.
- [x] Update the [`CHANGES`
log](https://github.com/apache/skywalking-banyandb/blob/main/CHANGES.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
