[
https://issues.apache.org/jira/browse/IOTDB-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuan Tian reassigned IOTDB-6279:
--------------------------------
Assignee: liaolanyu
> Query module exceptionally shows out of memory
> ----------------------------------------------
>
> Key: IOTDB-6279
> URL: https://issues.apache.org/jira/browse/IOTDB-6279
> Project: Apache IoTDB
> Issue Type: Bug
> Reporter: Sicheng Yu
> Assignee: liaolanyu
> Priority: Major
> Attachments: car.zip
>
>
> 问题：
> data 文件夹大小一共 55 MB 左右，启动 datanode 的 jvm 有 2G，执行查询 select count( * ) from
> root.** group by level=1,2 一直显示内存不足，iotdb-common.properties 中配置了
> datanode_memory_proportion=3:10:1:1:1:1，enable_mem_control=false
> 复现步骤：
> 1. 执行语句 load 'path/to/dataset' onSuccess=none
> 2. 执行查询 select count( * ) from root.** group by level=1,2
> 3. 显示：Msg: org.apache.iotdb.jdbc.IoTDBSQLException: 301: Query is aborted
> since it requests more memory than can be allocated.
> Problem:
> The size of the data folder is about 55 MB, and the jvm that starts datanode
> is 2G, but the query select count( * ) from root.** group by level=1,2 keeps
> showing out of memory.
> Steps to reproduce:
> 1. execute the statement load 'path/to/dataset' onSuccess=none
> 2. Execute the query select count( * ) from root.** group by level=1,2
> 3. show: Msg: org.apache.iotdb.jdbc.IoTDBSQLException: 301: Query is aborted
> since it requests more memory than can be allocated.
> ERROR Logs：
> 136028 [Query-Worker-Thread-11$20231226_040029_00044_1.1.0.3] INFO
> o.a.i.d.s.b.TimeSeriesMetadataCache - TimeSeriesMetadataCache size =
> 454669257
> 136031 [Query-Worker-Thread-1$20231226_040029_00044_1.1.0.7] INFO
> o.a.i.d.s.buffer.BloomFilterCache - BloomFilterCache size = 2273346
> 136113 [Query-Worker-Thread-0$20231226_040029_00044_1.1.0.6] WARN
> o.a.i.d.q.e.memory.MemoryPool - Cannot reserve 334626(Max: 123016) bytes
> memory from MemoryPool for planNodeId1321-5
> 136113 [Query-Worker-Thread-0$20231226_040029_00044_1.1.0.6] WARN
> o.a.i.d.q.e.s.AbstractDriverThread - [ExecuteFailed]
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Query is
> aborted since it requests more memory than can be allocated.
> at
> org.apache.iotdb.db.queryengine.execution.driver.Driver.processInternal(Driver.java:249)
> at
> org.apache.iotdb.db.queryengine.execution.driver.Driver.lambda$processFor$1(Driver.java:150)
> at
> org.apache.iotdb.db.queryengine.execution.driver.Driver.tryWithLock(Driver.java:326)
> at
> org.apache.iotdb.db.queryengine.execution.driver.Driver.processFor(Driver.java:131)
> at
> org.apache.iotdb.db.queryengine.execution.schedule.DriverTaskThread.execute(DriverTaskThread.java:83)
> at
> org.apache.iotdb.db.queryengine.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:73)
> Caused by: java.lang.IllegalArgumentException: Query is aborted since it
> requests more memory than can be allocated.
> at
> org.apache.iotdb.db.queryengine.execution.memory.MemoryPool.reserve(MemoryPool.java:239)
> at
> org.apache.iotdb.db.queryengine.execution.exchange.SharedTsBlockQueue.add(SharedTsBlockQueue.java:225)
> at
> org.apache.iotdb.db.queryengine.execution.exchange.sink.LocalSinkChannel.send(LocalSinkChannel.java:148)
> at
> org.apache.iotdb.db.queryengine.execution.driver.Driver.processInternal(Driver.java:241)
> ... 5 common frames omitted
> 136123 [Query-Worker-Thread-0$20231226_040029_00044_1.1.0.6] WARN
> o.a.i.d.q.e.s.DriverScheduler - The task 20231226_040029_00044_1.1.0.6 is
> aborted. All other tasks in the same query will be cancelled
> 136444 [pool-27-IoTDB-ClientRPC-Processor-1$20231226_040029_00044_1] WARN
> o.a.i.d.q.p.e.QueryExecution - [ResultHandleAborted]
> 136444 [pool-27-IoTDB-ClientRPC-Processor-1] WARN
> o.a.i.db.utils.ErrorHandlingUtils - Status code: 301, Query Statement:
> "select count(*) from root.** group by level=1,2". executeStatement failed
> org.apache.iotdb.commons.exception.IoTDBException:
> org.apache.iotdb.commons.exception.IoTDBException: Query is aborted since it
> requests more memory than can be allocated.
> at
> org.apache.iotdb.db.queryengine.plan.execution.QueryExecution.dealWithException(QueryExecution.java:547)
> at
> org.apache.iotdb.db.queryengine.plan.execution.QueryExecution.getResult(QueryExecution.java:535)
> at
> org.apache.iotdb.db.queryengine.plan.execution.QueryExecution.getByteBufferBatchResult(QueryExecution.java:565)
> at
> org.apache.iotdb.db.utils.QueryDataSetUtils.convertQueryResultByFetchSize(QueryDataSetUtils.java:604)
> at
> org.apache.iotdb.db.protocol.thrift.impl.ClientRPCServiceImpl.lambda$static$0(ClientRPCServiceImpl.java:245)
> at
> org.apache.iotdb.db.protocol.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:323)
> at
> org.apache.iotdb.db.protocol.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:744)
> at
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3869)
> at
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3849)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at
> org.apache.iotdb.db.protocol.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: org.apache.iotdb.commons.exception.IoTDBException: Query is
> aborted since it requests more memory than can be allocated.
> at
> org.apache.iotdb.db.queryengine.plan.execution.QueryExecution.getResult(QueryExecution.java:502)
> ... 13 common frames omitted
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
