damccorm commented on issue #21797: URL: https://github.com/apache/beam/issues/21797#issuecomment-1180484317
I think I found a good way to do this today. If we limit the memory size of the worker with a syscall to set the rlimit, instead of throwing an OOM the process will fail gracefully and return control back to boot.go. As long as we then immediately take a heap dump, we get a good picture of what the heap looks like on failure. This seems like its about as good as we can do with Go, but it should get us mostly on par with the other SDKs. FWIW, its not totally unlike what Java does [here](https://github.com/apache/beam/blob/262f2b7f91ac879cb8921a3e7d59d0315c9df9c4/sdks/java/container/boot.go#L238), though its definitely not exactly the same either -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
