wizhuo opened a new issue, #34011: URL: https://github.com/apache/doris/issues/34011
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 2.1.1 ### What's Wrong? We have many log tables, and logs are imported via Logstash HTTP. The BE node experiences OOM every two days. The memory accumulates slowly, suspecting a memory leak. Our machine has 16 cores and 32 GB of RAM. the be log: W20240421 07:46:11.495478 2456926 fragment_mgr.cpp:988] Query 7b3a7c2384934ea6-b94285f703f9018c does not exists, failed to cancel it W20240421 07:46:11.495497 2456926 fragment_mgr.cpp:988] Query e3d70ef9825a4c86-88b3701f4bdb743a does not exists, failed to cancel it W20240421 07:46:11.495535 2456926 fragment_mgr.cpp:988] Query 5935a8d098b14d66-b3d61fa0d0fa456f does not exists, failed to cancel it W20240421 07:46:11.495554 2456926 fragment_mgr.cpp:988] Query bddae462a0b0415f-be10ffa686ce1221 does not exists, failed to cancel it W20240421 07:46:11.495573 2456926 fragment_mgr.cpp:988] Query eb2372a014fb413a-98840965271fbc30 does not exists, failed to cancel it W20240421 07:46:11.495591 2456926 fragment_mgr.cpp:988] Query 6cf55872374e4988-9a635fc8e6fae361 does not exists, failed to cancel it W20240421 07:46:11.495611 2456926 fragment_mgr.cpp:988] Query e4431e3b8a1e4312-bb2b8871252150d0 does not exists, failed to cancel it W20240421 07:46:11.495630 2456926 fragment_mgr.cpp:988] Query 270dc844510c4afe-9a073904d483f98a does not exists, failed to cancel it W20240421 07:46:11.495662 2456926 fragment_mgr.cpp:988] Query 53765cfd53a54992-88ba91f00ff14bb1 does not exists, failed to cancel it W20240421 07:46:11.495682 2456926 fragment_mgr.cpp:988] Query 6da040a53e2443ba-b2d8a0105ac45625 does not exists, failed to cancel it W20240421 07:46:11.495699 2456926 fragment_mgr.cpp:988] Query 70848f92102d4119-a64e09371b27c192 does not exists, failed to cancel it W20240421 07:46:11.495718 2456926 fragment_mgr.cpp:988] Query a73d50375fe04f47-b30345d1a846e72c does not exists, failed to cancel it W20240421 07:46:11.495736 2456926 fragment_mgr.cpp:988] Query b26b4bc8c73a4392-88b462c33ecc2c3f does not exists, failed to cancel it W20240421 07:46:11.495766 2456926 fragment_mgr.cpp:988] Query df587b3c71d84232-a39a901dc07533bb does not exists, failed to cancel it W20240421 07:46:11.495787 2456926 fragment_mgr.cpp:988] Query 7ad462da539d4c9a-938b4c0c74708c77 does not exists, failed to cancel it W20240421 07:46:11.495805 2456926 fragment_mgr.cpp:988] Query ad2bceda0356498e-8782e44b4b213f97 does not exists, failed to cancel it W20240421 07:46:11.495824 2456926 fragment_mgr.cpp:988] Query c84929a9fc194557-b61a5f488217d728 does not exists, failed to cancel it W20240421 07:46:11.495847 2456926 fragment_mgr.cpp:988] Query a6abbc51c4b14418-8cde3de08640f6e3 does not exists, failed to cancel it W20240421 07:46:11.495877 2456926 fragment_mgr.cpp:988] Query 394dd06ec501435c-b154fb24806068a9 does not exists, failed to cancel it W20240421 07:46:11.495904 2456926 fragment_mgr.cpp:988] Query 6809cd016de24c98-b4668381cd1132f2 does not exists, failed to cancel it W20240421 07:46:11.495923 2456926 fragment_mgr.cpp:988] Query ecabfff9a6b04d92-a573cd79b1f45655 does not exists, failed to cancel it W20240421 07:46:11.495941 2456926 fragment_mgr.cpp:988] Query f433831d57e7460f-bec300b393f046f3 does not exists, failed to cancel it W20240421 07:46:11.495967 2456926 fragment_mgr.cpp:988] Query 894c355af2e64009-af922214b3e62419 does not exists, failed to cancel it W20240421 07:46:11.495985 2456926 fragment_mgr.cpp:988] Query 9ffdb9b96bac45fa-927f9031aa0682b4 does not exists, failed to cancel it W20240421 07:46:11.496006 2456926 fragment_mgr.cpp:988] Query 83aa834e12f34f7f-8189681f96cdf6f3 does not exists, failed to cancel it W20240421 07:46:11.496023 2456926 fragment_mgr.cpp:988] Query f9481e118a1d4d59-86717ba432a8bd52 does not exists, failed to cancel it W20240421 07:46:11.496042 2456926 fragment_mgr.cpp:988] Query 81b85c63cc1f4224-88f7e82a42e1c495 does not exists, failed to cancel it I20240421 07:46:11.496073 2456926 mem_tracker_limiter.cpp:504] [MemoryGC] GC free process top memory used query, cancel finished, 106 tasks canceled, memory size being freed: 957439375, consist of: Query#Id=8726d97aa9744993-b53135887e186228 memory used 2576384 Bytes,Query#Id=923036e880774daf-87eb369cb9a5c836 memory used 4395008 Bytes,Query#Id=ca9e91ae8894648-aa7f5e26df43a020 memory used 4395008 Bytes,Query#Id=d5087f110b6e40cb-869b39c08a8f11f3 memory used 4395008 Bytes,Query#Id=8b1780180fb545fe-841bebb925b02342 memory used 4397056 Bytes,Query#Id=1eda79619e11477f-84adefbbbc13b2af memory used 4431872 Bytes,Query#Id=5b1ca20ff059452e-9e1d928960cd3731 memory used 4677632 Bytes,Query#Id=dead87f0c6864083-976ff86d6164d267 memory used 4714496 Bytes,Query#Id=35ab6357a0964f41-866e8409642d8a51 memory used 5132288 Bytes,Query#Id=90bc36c6c5cf4f72-a77a2f0bf668a526 memory used 5134336 Bytes,Query#Id=1011ba364c614769-aa2c2c71608e71b1 memory used 5146624 Bytes,Query#Id=c8d6801edde64419-82f2a46ffce46 f4c memory used 5242880 Bytes,Query#Id=88050dfe82874408-bfa5f6c6e83bf1e8 memory used 5746688 Bytes,Query#Id=2c4e20c178ed409a-b6ce81c56843b5d9 memory used 5758976 Bytes,Query#Id=64511a867fa146e3-bee14429bb49ba13 memory used 5795840 Bytes,Query#Id=cef1d704c42d4856-b88c4f724f51d7f8 memory used 5820416 Bytes,Query#Id=e88a16b6578b4f01-85180200a56aecab memory used 5885952 Bytes,Query#Id=664f22b0f07c4b8a-9a93052d3cd62bbf memory used 5906432 Bytes,Query#Id=9f0fdf6bfd704cb0-a39c5f8dc7e93bbd memory used 6041600 Bytes,Query#Id=e14143d10ac54e08-bce7c0882032a800 memory used 6062080 Bytes,Query#Id=ec371fe31b5a418b-89c8e6e414f6aebc memory used 6062080 Bytes,Query#Id=2aaa3684d14dd4-b4792e617efaa300 memory used 6090752 Bytes,Query#Id=de253d614b524cda-b90a2d26456f1344 memory used 6123520 Bytes,Query#Id=8ac0758882ad4325-a821ff31268547a6 memory used 6152192 Bytes,Query#Id=47f00d0368884b4e-b0a3b66a018fa4b2 memory used 6164480 Bytes,Query#Id=f5f88a9caf024749-84a8e0aa5f00497e memory used 6246400 Bytes,Que ry#Id=d11591e0143e4dcc-849e72e86be780ff memory used 6266880 Bytes,Query#Id=a3224c8e59094d07-96c221b030fb912c memory used 6987776 Bytes,Query#Id=83a804ea5b2b4b2e-a20c67ebda300cd0 memory used 7041024 Bytes,Query#Id=ae9f5ba0ef37478f-9b36e1ac7ec41459 memory used 7041024 Bytes,Query#Id=b3ff7e80f03445b0-914c23f14a6198d3 memory used 7041024 Bytes,Query#Id=3e5f13873f864075-a6c426c2c5ca0b7f memory used 7051264 Bytes,Query#Id=e67943ae78d74a64-a538547393815e16 memory used 7094272 Bytes,Query#Id=a1b3a9f4746b4242-b6415b5119055839 memory used 7127040 Bytes,Query#Id=bb487883d42240b1-80393cb76ff1cc8f memory used 7127040 Bytes,Query#Id=8fe14d86073248c8-8a5005ed4b79f730 memory used 7467008 Bytes,Query#Id=32b40784ff574679-90a2b1e8049aed1a memory used 8353792 Bytes,Query#Id=4e68e702080f4184-b1245b50927f72e0 memory used 8437760 Bytes,Query#Id=926d8267756c4f60-a35e0e71ded7d9ce memory used 8437760 Bytes,Query#Id=d534bf63678441a5-909f9fa65df6a4a4 memory used 8437760 Bytes,Query#Id=241c3230a26e4cb7-a51ac812 683a5a3c memory used 8499200 Bytes,Query#Id=cc3b152cbd1c4a47-a03ab254a4e10454 memory used 8499200 Bytes,Query#Id=c059afad98c4ae0-95181fe3c94eb08c memory used 10174464 Bytes,Query#Id=32b2306cffa04a6a-af5cc725f8401b18 memory used 10338304 Bytes,Query#Id=d978775ad0eb4b10-9129673c4e6639d7 memory used 10338304 Bytes,Query#Id=61969e889f2c4171-a3e27cc970821d65 memory used 10391552 Bytes,Query#Id=9e0479d61c44a6e-91060b7dfccc213c memory used 10391552 Bytes,Query#Id=3448e2a731e74500-be98349db6bf85d5 memory used 10481664 Bytes,Query#Id=3efedd02129e411b-a074a1760e0d7f57 memory used 10481664 Bytes,Query#Id=e79bb7f1b75d48a0-8028079a74d119d5 memory used 10481664 Bytes,Query#Id=1c11574eb5e844c2-ba946bf270ddafc9 memory used 10575872 Bytes,Query#Id=2c7bfdb9216641fe-ae80b8d1b6ba2009 memory used 10575872 Bytes,Query#Id=565ed93e4dbc4bd3-a2344983073dacdd memory used 10575872 Bytes,Query#Id=656a77e67fc34d4a-8e81281c8801b44a memory used 10575872 Bytes,Query#Id=7d3c69d46dd442ee-b50f29b6d3e8ec76 memory used 10575872 Bytes,Query#Id=c203c51f5341400f-9d3ded6063fee084 memory used 10575872 Bytes,Query#Id=db8f68ae587d4c5e-aa3fc64c9e5e72a5 memory used 10575872 Bytes,Query#Id=ea105c5ab7854b06-917f4d92ddf31f82 memory used 10575872 Bytes,Query#Id=53ba502ce9944797-913f0fc4bd2b72f0 memory used 10596352 Bytes,Query#Id=c6b7b1a173204f54-92a8f32cb747a0af memory used 10596352 Bytes,Query#Id=d65fc2764d9c47ef-9687ca40c49e9c24 memory used 10602244 Bytes,Query#Id=e46b94d89aa843c0-881f95305253c92b memory used 10602244 Bytes,Query#Id=920384419d8b4da7-9e5f4e0a5e83b9c5 memory used 10612736 Bytes,Query#Id=9fff7054f27542c8-bde6f0f87dd39775 memory used 10633216 Bytes,Query#Id=2a4cae5d515e44af-84268fbe62581d48 memory used 10637312 Bytes,Query#Id=181e8e9bed3044ad-9a0cc9fc35353c88 memory used 10655492 Bytes,Query#Id=42eaa7f4f9224767-a1159bdb91b24b2c memory used 10655492 Bytes,Query#Id=59e19ef196e7463e-9003b2943bcf56fa memory used 10655492 Bytes,Query#Id=6042561ca2d447b2-a958db44ba295b85 memory used 10655492 Bytes,Qu ery#Id=a8c4ab1f51fc490d-b9849f8045cdcf16 memory used 10655492 Bytes,Query#Id=a8f3f1dc7af44bc8-b4054356565bd3bd memory used 10655492 Bytes,Query#Id=752c08c182e24b4e-b9de45c0deef7123 memory used 10674176 Bytes,Query#Id=dc21a008e8e44ccb-b9fe2456ad71ffb4 memory used 10682368 Bytes,Query#Id=b77c976141b64866-b760ef6745aa1fcb memory used 10694656 Bytes,Query#Id=77d85d78a70b46ff-aa552915e922adb2 memory used 10735652 Bytes,Query#Id=bea6a2d97c4049e0-aa32d60cf2666cd2 memory used 10735652 Bytes,Query#Id=97e139e7e45b47a5-950da3ea96e73e34 memory used 10756096 Bytes,Query#Id=cb5709ed1947471c-b7826cda6d450239 memory used 10878976 Bytes,Query#Id=9b99939667fc49ac-877f5e3541b31731 memory used 10969166 Bytes,Query#Id=7b3a7c2384934ea6-b94285f703f9018c memory used 11022433 Bytes,Query#Id=e3d70ef9825a4c86-88b3701f4bdb743a memory used 11022433 Bytes,Query#Id=5935a8d098b14d66-b3d61fa0d0fa456f memory used 11043361 Bytes,Query#Id=bddae462a0b0415f-be10ffa686ce1221 memory used 11053489 Bytes,Query#Id=eb2372a014 fb413a-98840965271fbc30 memory used 11053489 Bytes,Query#Id=6cf55872374e4988-9a635fc8e6fae361 memory used 11096450 Bytes,Query#Id=e4431e3b8a1e4312-bb2b8871252150d0 memory used 11096450 Bytes,Query#Id=270dc844510c4afe-9a073904d483f98a memory used 11132111 Bytes,Query#Id=53765cfd53a54992-88ba91f00ff14bb1 memory used 11132111 Bytes,Query#Id=6da040a53e2443ba-b2d8a0105ac45625 memory used 11156049 Bytes,Query#Id=70848f92102d4119-a64e09371b27c192 memory used 11156049 Bytes,Query#Id=a73d50375fe04f47-b30345d1a846e72c memory used 11156049 Bytes,Query#Id=b26b4bc8c73a4392-88b462c33ecc2c3f memory used 11156049 Bytes,Query#Id=df587b3c71d84232-a39a901dc07533bb memory used 11161600 Bytes,Query#Id=7ad462da539d4c9a-938b4c0c74708c77 memory used 11161744 Bytes,Query#Id=ad2bceda0356498e-8782e44b4b213f97 memory used 11161744 Bytes,Query#Id=c84929a9fc194557-b61a5f488217d728 memory used 11194463 Bytes,Query#Id=a6abbc51c4b14418-8cde3de08640f6e3 memory used 11206800 Bytes,Query#Id=394dd06ec501435c-b154fb2480 6068a9 memory used 11218225 Bytes,Query#Id=6809cd016de24c98-b4668381cd1132f2 memory used 11227280 Bytes,Query#Id=ecabfff9a6b04d92-a573cd79b1f45655 memory used 11242065 Bytes,Query#Id=f433831d57e7460f-bec300b393f046f3 memory used 11242065 Bytes,Query#Id=894c355af2e64009-af922214b3e62419 memory used 11317392 Bytes,Query#Id=9ffdb9b96bac45fa-927f9031aa0682b4 memory used 11907143 Bytes,Query#Id=83aa834e12f34f7f-8189681f96cdf6f3 memory used 11907175 Bytes,Query#Id=f9481e118a1d4d59-86717ba432a8bd52 memory used 11907266 Bytes,Query#Id=81b85c63cc1f4224-88f7e82a42e1c495 memory used 13875772 Bytes I20240421 07:46:11.496119 2456257 tablet_manager.cpp:1346] begin to get tablet. tablet_id=140416 I20240421 07:46:11.496150 2456926 mem_info.cpp:217] [MemoryGC] before free top memory overcommit load in full GC, Type:load, Memory Tracker Summary MemTrackerLimiter Label=Load#Id=7840ff88a1291760-862f6d23690dc28d, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=7f4905b2ecd2528c-e099a97aa481a4b6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=1b4fc97e83fc3cc6-a13a7de819fa819d, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=e47619c7a918151-10b936be332051b3, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=244612867dfc05ab-e5d4369c72792db6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=694d524be39de91c-f8974e48f480e90, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=61478d510ef97bb5-7db9a40ba8d74180, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c04c421013141449-e06dbc7f462dc1a0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=4d4dc4af37270cfc-b14a3e5bda6d1dbb, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=3e4f1751aec99ae0-56a94e39028c4fa2, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=d246be9364f0f272-136497ba126673b2, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=a3490dceef4267b4-20bc1c937ce4cea8, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c04036b4d7c12055-70752a265dbf49a8, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=af42e3160448deff-8f68fa20e28b299b, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=2f4cf77d666a556b-95a60db590cb72bd, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=MemTableMemoryLimiter, Type=load, Limit=-1.00 B(-1 B), Used=698.41 KB(715171 B), Peak=256.78 MB(269255225 B) MemTrackerLimiter Label=Load#Id=d242f35b1c1f6526-17dcf5261ce231be, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=b5476f8f4292c96e-19c215fba22728b0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=5743ef992c158eaa-84c7f3c345445d97, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=32495a87c728bee8-1540a02367040d95, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=f941ddc603b6e7cf-67e0608afe8d3ba0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=854c6b6f60a77be1-103fbcc9690c23a6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=304ee74ca129fd37-f32dc5ca4ba62589, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c54c6d0653d1d08b-a9029e3d8d3d18ac, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) I20240421 07:46:11.496408 2456926 mem_tracker_limiter.cpp:559] [MemoryGC] GC free process top memory overcommit load, , start seek all load, running query and load num: 23 I20240421 07:46:11.496490 2456926 mem_tracker_limiter.cpp:597] [MemoryGC] GC free process top memory overcommit load, seek finished, seek 24 tasks. among them, 0 tasks can be canceled; 24 small tasks that were skipped; 0 tasks is being canceled and has not been completed yet; I20240421 07:46:11.496505 2456926 mem_tracker_limiter.cpp:605] [MemoryGC] GC free process top memory overcommit load, finished, no task need be canceled. I20240421 07:46:11.496531 2456926 mem_info.cpp:230] [MemoryGC] before free top memory load in full GC, Type:load, Memory Tracker Summary MemTrackerLimiter Label=Load#Id=7840ff88a1291760-862f6d23690dc28d, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=7f4905b2ecd2528c-e099a97aa481a4b6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=1b4fc97e83fc3cc6-a13a7de819fa819d, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=e47619c7a918151-10b936be332051b3, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=244612867dfc05ab-e5d4369c72792db6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=694d524be39de91c-f8974e48f480e90, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=61478d510ef97bb5-7db9a40ba8d74180, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c04c421013141449-e06dbc7f462dc1a0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=4d4dc4af37270cfc-b14a3e5bda6d1dbb, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=3e4f1751aec99ae0-56a94e39028c4fa2, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=d246be9364f0f272-136497ba126673b2, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=a3490dceef4267b4-20bc1c937ce4cea8, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c04036b4d7c12055-70752a265dbf49a8, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=af42e3160448deff-8f68fa20e28b299b, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=2f4cf77d666a556b-95a60db590cb72bd, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=MemTableMemoryLimiter, Type=load, Limit=-1.00 B(-1 B), Used=698.41 KB(715171 B), Peak=256.78 MB(269255225 B) MemTrackerLimiter Label=Load#Id=d242f35b1c1f6526-17dcf5261ce231be, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=b5476f8f4292c96e-19c215fba22728b0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=5743ef992c158eaa-84c7f3c345445d97, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=32495a87c728bee8-1540a02367040d95, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=f941ddc603b6e7cf-67e0608afe8d3ba0, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=854c6b6f60a77be1-103fbcc9690c23a6, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=304ee74ca129fd37-f32dc5ca4ba62589, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) MemTrackerLimiter Label=Load#Id=c54c6d0653d1d08b-a9029e3d8d3d18ac, Type=load, Limit=2.00 GB(2147483648 B), Used=0(0 B), Peak=0(0 B) I20240421 07:46:11.496711 2456926 mem_tracker_limiter.cpp:427] [MemoryGC] GC free process top memory used load, , start seek all load, running query and load num: 23 I20240421 07:46:11.496786 2456926 mem_tracker_limiter.cpp:474] [MemoryGC] GC free process top memory used load, seek finished, seek 24 tasks. among them, 24 tasks will be canceled, 715171 memory size prepare free; 0 tasks is being canceled and has not been completed yet; I20240421 07:46:11.496811 2456926 pipeline_fragment_context.cpp:164] PipelineFragmentContext::cancel|query_id=1b4fc97e83fc3cc6-a13a7de819fa819d|fragment_id=1|instance_id=1b4fc97e83fc3cc6-a13a7de819fa819e|reason=MEMORY_LIMIT_EXCEED|message=Process has no memory available, cancel top memory used load: load memory tracker <Load#Id=1b4fc97e83fc3cc6-a13a7de819fa819d> consumption 0, backend xxxx process memory used 26.80 GB exceed limit 25.58 GB or sys available memory 1.63 GB less than low water mark 1.60 GB. Execute again after enough memory, details see be.INFO. W20240421 07:46:11.496832 2456926 pipeline_fragment_context.cpp:179] PipelineFragmentContext 1b4fc97e83fc3cc6-a13a7de819fa819e|1b4fc97e83fc3cc6-a13a7de819fa819d is canceled, cancel message: Process has no memory available, cancel top memory used load: load memory tracker <Load#Id=1b4fc97e83fc3cc6-a13a7de819fa819d> consumption 0, backend 10.111.212.248 process memory used 26.80 GB exceed limit 25.58 GB or sys available memory 1.63 GB less than low water mark 1.60 GB. Execute again after enough memory, details see be.INFO. I20240421 07:46:11.496851 2456926 vdata_stream_mgr.cpp:187] cancelling all streams for fragment=1b4fc97e83fc3cc6-a13a7de819fa819e ### What You Expected? not oom ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
