WinkerDu opened a new pull request, #28589: URL: https://github.com/apache/doris/pull/28589
## Proposed changes Issue Number: close #xxx My organization try to accelerate HMS Lake query based on #24830, which introduces BE using broker to scan for HMS catalog. In real scenarios, it appears that there is a memory leak of broker inputStreams. The current timeout detection is targeted at clients and is not applicable in broker scan scenarios (as there will always be requests from clients). As long as the inputStream is not properly closed, it will remain in memory <img width="1622" alt="image" src="https://github.com/apache/doris/assets/11454734/1b77bd49-724e-436b-b7af-b5ca5603be9d";> <!--Describe your changes.--> This pr introduces 2 broker conf: 1. `enable_input_stream_expire_check`: which indicates whether enable inputStream expire check. 2. `input_stream_expire_seconds`: which indicates the timeout seconds for inputStream since last update. ## Further comments If this is a relatively large or complex change, kick off the discussion at [[email protected]](mailto:[email protected]) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
