GitHub user Xuanwo added a comment to the discussion: (rust) Huge memory consumption, when recv+streaming 1k chunks to S3
Hi, I think the problem happen at the usage of `Bytes::from`. ```rust let mut chunk = vec![0u8; 128 * 1024]; // ... write 1k into the chunk chunk.truncate(count); let data = Bytes::from(chunk); <--- note: the chuck here still have 128Ki cap. ``` Please try calling `shrink_to_fit` before `Bytes::from` in the cases where we alloc a lot but write slowly. --- I got gpt-5.3-codex to have a simpler repro script: https://play.rust-lang.org/?version=stable&mode=release&edition=2024&gist=8b08363e629816114c245ece1d86bad7 Hoping my answer addressed your problem. GitHub link: https://github.com/apache/opendal/discussions/7200#discussioncomment-15889991 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
