Greetings,
I’d like to introduce ImportWAL, a command-line utility for recovering data
from WAL files into an existing IoTDB instance.
When a DataNode fails or its WAL directory must be moved aside—for example,
because of WAL corruption or insufficient disk space—the WAL may still contain
unflushed writes, deletes, and MemTable snapshots. ImportWAL makes it possible
to extract and replay this recoverable data without restoring the entire
original DataNode.
The tool is designed for operational recovery scenarios and supports both IoTDB
data models. It can process an individual WAL file or a WAL directory, replay
data into a prepared target schema, and provide progress and recovery
statistics. Operators can also review potentially destructive or unsupported
entries during replay and keep the original WAL files available for
verification and archival.
A typical recovery workflow is:
1. Stop the affected DataNode and preserve the original WAL files.
2. Prepare the target IoTDB schema or table structure.
3. Replay the WAL data into an isolated or restored target instance.
4. Verify recovered devices, tables, time ranges, and business-level
statistics.
5. Archive the source WAL files and replay logs after validation.
Example:
tools/import-wal.sh \
-f /path/to/wal \
-h 127.0.0.1 \
-p 6667 \
-u root
The implementation is available in PR #18516
(https://github.com/apache/iotdb/pull/18516). Feedback on the recovery workflow
and operational usability would be very welcome.
Best regards,
Tian Jiang
