Sunymfc opened a new issue, #18709:
URL: https://github.com/apache/iotdb/issues/18709

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/iotdb/issues) and 
found nothing similar.
   
   
   ### Version
   
   On IoTDB 2.0.8 (Windows, all-bin)
   
   ### Describe the bug and provide the minimal reproduce step
   
   
   On IoTDB 2.0.8 (Windows, all-bin), three Pipe processor threads concurrently 
wait tens of seconds for memory to parse sequence TsFiles. At the same time, 
compaction is running on the same databases. Cross-space compaction then fails 
while reading an older TsFile: the native `FileChannel.read` returns Windows 
error 1450, `ERROR_NO_SYSTEM_RESOURCES` (`系统资源不足,无法完成请求的服务。`).
   This is not a Java heap OutOfMemoryError. The read fails inside 
`sun.nio.ch.FileDispatcherImpl.read0`.
   2.0.10 PR #17542 ("Do not stop pipe for Waited for memory to parse TsFile") 
only keeps the Pipe subtask from stopping on that timeout. It does not address 
the native read failure under the same concurrent Pipe + compaction load.
   
   
   ### What did you expect to see?
   
   Pipe TsFile parsing and compaction should back off when memory or OS read 
resources are short, instead of three Pipe workers and several compaction tasks 
reading TsFiles at once until Windows returns `ERROR_NO_SYSTEM_RESOURCES`.
   
   
   ### What did you see instead?
   
   Pipe workers block 20–39 seconds waiting for parse memory and then report 
temporary OOM. Concurrently, compaction cannot read a TsFile because the OS 
reports insufficient system resources. The file involved in the failed read 
(`...\11\2952\...`) is not one of the files the Pipe workers were parsing 
(`...\11\2959\...` and `...\12\2959\...`).
   
   ## Error
   ```text
   2026-09-24 03:51:20,892 [pool-15-IoTDB-Compaction-Schedule-3] ERROR 
o.a.t.r.r.LocalTsFileInput:88 - 从当前位置读取 
D:\apache-iotdb-2.0.8-all-bin\apache-iotdb-2.0.8-all-bin\data\datanode\data\sequence\root.ShipData\11\2952\1785369603984-1-5-2.tsfile
 时出错
   2026-09-24 03:51:21,229 [pool-15-IoTDB-Compaction-Schedule-3] ERROR 
o.a.i.d.s.d.c.s.i.RewriteCrossSpaceCompactionSelector:429 - root.ShipData 
cannot select file for cross space compaction
   org.apache.iotdb.db.exception.MergeException: java.io.IOException: 
系统资源不足,无法完成请求的服务。
        at 
org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl.RewriteCrossSpaceCompactionSelector.selectOneTaskResources(RewriteCrossSpaceCompactionSelector.java:153)
        at 
org.apache.iotdb.db.storageengine.dataregion.compaction.selector.impl.RewriteCrossSpaceCompactionSelector.selectCrossSpaceTask(RewriteCrossSpaceCompactionSelector.java:375)
        ...
   Caused by: org.apache.tsfile.exception.NotCompatibleTsFileException: 
java.io.IOException: 系统资源不足,无法完成请求的服务。
        at 
org.apache.tsfile.read.TsFileSequenceReader.loadFileVersion(TsFileSequenceReader.java:439)
        at 
org.apache.tsfile.read.TsFileSequenceReader.<init>(TsFileSequenceReader.java:316)
        ...
   Caused by: java.io.IOException: 系统资源不足,无法完成请求的服务。
        at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at 
java.base/sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:60)
        at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:341)
        at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:307)
        at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:284)
        at 
java.base/sun.nio.ch.FileChannelImpl.implRead(FileChannelImpl.java:251)
        at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:231)
        at 
org.apache.tsfile.read.reader.LocalTsFileInput.read(LocalTsFileInput.java:83)
        at 
org.apache.tsfile.read.TsFileSequenceReader.loadFileVersion(TsFileSequenceReader.java:429)
   ```
   Pipe side, same window:
   ```text
   2026-09-24 03:51:07,196 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-3] INFO  
o.a.i.d.p.e.c.t.PipeTsFileInsertionEvent:803 - Wait for memory enough for 
parsing ...\sequence\root.ShipData\11\2959\1789622109110-279-1-0.tsfile for 
20.007 seconds.
   2026-09-24 03:51:15,768 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-2] INFO  
o.a.i.d.p.e.c.t.PipeTsFileInsertionEvent:803 - Wait for memory enough for 
parsing ...\sequence\root.ShipData\12\2959\1789624287446-87-0-0.tsfile for 
20.02 seconds.
   2026-09-24 03:51:19,501 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-2] WARN  
o.a.i.c.p.r.l.PipePeriodicalLogReducer:51 - Parse TsFile 
...\1789624287446-87-0-0.tsfile error. Because: TimeoutException: Waited 23.753 
seconds for memory to parse TsFile
   2026-09-24 03:51:25,629 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-1] WARN  
... TimeoutException: Waited 25.243 seconds for memory to parse TsFile 
(...\1789623847592-339-0-0.tsfile)
   2026-09-24 03:51:26,256 [pool-48-IoTDB-Pipe-Processor-Executor-Pool-3] WARN  
... TimeoutException: Waited 39.067 seconds for memory to parse TsFile 
(...\1789622109110-279-1-0.tsfile)
   ```
   
   ### Anything else?
   
   ## Timeline (2026-09-24 03:51)
   1. Pipe Pool-1, Pool-2, and Pool-3 each log `Wait for memory enough for 
parsing` every 10 seconds, on three different sequence TsFiles (`root.ShipData` 
partitions 11 and 12). Waits reach 20–30 seconds.
   2. Cross-space compaction of `root.ShipData-12` starts (1 seq file + 1 unseq 
file, about 14.7 MB) and finishes.
   3. Pipe Pool-2 gives up after 23.753 seconds: `TimeoutException: Waited 
23.753 seconds for memory to parse TsFile`, then `Temporarily out of memory in 
pipe event processing`.
   4. Unsequence and sequence inner compaction of `root.ShipData-11` start 
together (5 files / 3 MB and 5 files / 9 MB).
   5. Cross-space compaction selection for `root.ShipData` fails while reading 
`sequence\root.ShipData\11\2952\1785369603984-1-5-2.tsfile`. Cause: 
`java.io.IOException: 系统资源不足,无法完成请求的服务。`
   6. The two remaining Pipe workers then time out as well (25.243 s and 39.067 
s).
   7. Inner compaction of `root.ShipData-11` still completes. A JVM compiler 
replay file `replay_pid15624.log` was written in this window; 
`hs_err_pid15624.log` was not captured.
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to