GitHub user michaelrommel closed the discussion with a comment: (rust) Huge memory consumption, when recv+streaming 1k chunks to S3
Not yet @Xuanwo! I need to first rule out that it is due to my code. I have patched both libunftp and unftp-sbe-opendal to add two things I need - once a STOR request comes in, I reach out to a valkey instance and get some metadata. Then I store the file in a different location based on the metadata. In addition I add the metadata to the context and I have a `put_meta()` function parallel to `put()` that stores the file in S3 together with the metadata. Then I have implemented a custom AWS Elastic Container metadata credential fetch mechanism, because the current version of opendal 0.55 cannot work with the newer reqsign crate that would support the ECS metadata... So I always need to make sure, that a minimal example exhibits the same behaviour before I open an issue... GitHub link: https://github.com/apache/opendal/discussions/7200#discussioncomment-15900772 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
