scwhittle commented on issue #34749: URL: https://github.com/apache/beam/issues/34749#issuecomment-2854192799
The heaps stacks are a snapshot of memory used and where it was allocated, not the rate of allocations. So I don't think it show that the allocations are happening repeatedly for a single thread, just that the memory overhead of allocating per-thread adds up (since there can be 300-500 threads). If the DatumReader and DatumWriter are thread-safe, can we just remove the thread-locals used for them here? https://github.com/apache/beam/blob/master/sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/coders/AvroCoder.java#L380 There still may be some benefit of a static cache somewhere (since there may be multiple AvroCoder instances that are the same) but I think we can see if just removing the thread-local helps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
