xiongcp opened a new issue, #41223: URL: https://github.com/apache/doris/issues/41223
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version doris 2.1.6 ### What's Wrong? doris版本：2.1.6 FE和BE混合部署 大约十多张表，数据导入后大小总计为500M，单张表数据最大为百万级别 机器信息   ``` 测试代码 并发执行SQL，SQL基本都是秒内返回 func main() { // 设置日志文件 logFile, err := os.OpenFile("run.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0666) if err != nil { fmt.Println("Error opening log file:", err) return } defer logFile.Close() log.SetOutput(logFile) args := os.Args[1:] if len(args) < 1 { fmt.Println("Usage: main run_count") } count, err := strconv.Atoi(args[0]) if err != nil { fmt.Println("Usage: main run_count need int val") } for i := 0; i < (count); i++ { fmt.Printf("run time :%d \n", i) run() } } func run() { // 数据库连接信息 dsn := "root:test@tcp(xxxxxxxx:9030)/test" // 连接到 MySQL 数据库 db, err := sql.Open("mysql", dsn) if err != nil { log.Println("Error connecting to database:", err) fmt.Println("Error occurred while connecting to the database. Check error.log for details.") return } defer db.Close() // 设置最大空闲连接数和最大打开连接数 db.SetMaxOpenConns(10) db.SetMaxIdleConns(5) // 检查数据库连接 if err := db.Ping(); err != nil { log.Println("Error pinging database:", err) fmt.Println("Error occurred while pinging the database. Check error.log for details.") return } // 打开 SQL 文件 SQL文件中有3万条业务SQL file, err := os.Open("test.sql") if err != nil { log.Println("Error opening SQL file:", err) fmt.Println("Error occurred while opening the SQL file. Check error.log for details.") return } defer file.Close() // 使用缓冲读取 SQL 文件 reader := bufio.NewReader(file) // 控制并发执行的 goroutine 数量，设定最大并发数 maxConcurrency := 5 sem := make(chan struct{}, maxConcurrency) var wg sync.WaitGroup var total, successCount, failureCount int startTime := time.Now() // 按行读取并执行 SQL 语句 for { line, err := reader.ReadString('\n') if err != nil { break } sqlStatement := strings.TrimSpace(line) if sqlStatement == "" { continue } total++ wg.Add(1) sem <- struct{}{} // 占用一个 goroutine go func(query string) { defer wg.Done() defer func() { <-sem }() // 释放 goroutine //start := time.Now() _, err := db.Exec(query) if err != nil { log.Printf("Error executing SQL query: %s, Error: %v", query, err) failureCount++ } else { successCount++ } // 记录每条 SQL 执行的耗时 //log.Printf("Executed SQL query: %s, Duration: %s", query, time.Since(start)) }(sqlStatement) } // 等待所有的 goroutine 完成 wg.Wait() // 打印并记录执行结果 log.Printf("Execution completed. Total: %d %d, Failure: %d, Total Duration: %s", total, successCount, failureCount, time.Since(startTime)) fmt.Printf("Execution completed. Total: %d, Success: %d, Failure: %d\n", total, successCount, failureCount) } ``` 在持续进行查询时，BE内存不断上涨，导致使用超限内存的SQL无法再执行。但是在停止SQL执行后。保持在26.5G   停止查询SQL后的BE的info日志持续打印  be.warning日志  be mem_tracker   mem_tracker?type=global    ### What You Expected? 持续的查询不会导致内存持续的上升，停止查询后应该释放部分内存。 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
