damccorm commented on issue #21797: URL: https://github.com/apache/beam/issues/21797#issuecomment-1179366645
Oh yikes, when there truly is an OOM, the whole stack panics unrecoverably (including boot.go). I guess that makes sense because its subject to the same memory constraints as its subprocess? Regardless, that means that our only real options are: 1) Try to do something predictive of memory issues 2) Do nothing 3) Do something like what I did in #22196 - I don't see a heap dump being super useful in the cases that would be caught by that though, so I'm not inclined to do it. 4) Do something at the runner level. It looks like maybe python does something like that? I'll look into their approach more next week https://github.com/apache/beam/blob/bd0ac384b13c39e577ef211d5cb346ae0021e796/sdks/python/apache_beam/runners/worker/worker_status.py#L87 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
