We have identified the leak. https://github.com/apache/beam/issues/28246
has the details and workarounds.
On Mon, Aug 28, 2023 at 9:57 AM Valentyn Tymofieiev
wrote:
> This appears to be a recent issue reported also by others (e.g.
> https://github.com/apache/beam/issues/28142), it's being active
This appears to be a recent issue reported also by others (e.g.
https://github.com/apache/beam/issues/28142), it's being actively
investigated. Therefore, it is unlikely that memory fragmentation is an
issue.
On Tue, Aug 22, 2023 at 5:21 PM Valentyn Tymofieiev
wrote:
> Hi, thanks for reaching ou
Hi, thanks for reaching out.
I'd be curious to see whether the memory consumption patterns you observe
change if you switch the memory allocator library.
For example, you could try to use a custom container, install jemalloc and
enable it. See: https://beam.apache.org/documentation/runtime/enviro
