xinglin commented on PR #6519: URL: https://github.com/apache/hadoop/pull/6519#issuecomment-1923208190
> > The new code would remove that bad connection object and a new good one will be created next time. > Do we log the bad connection which was cleaned up? > > What happens if there is an OOM when creating the new connection? Wondering is we can get into a live-lock. > > Please do excuse the many questions, I'm not familiar with this code so this is to help my understanding. made some code change. prefer to have a single try-catch block to avoid code duplication. Totally valid question and i'd have to chat with Gobblin team to understand more about their use case and confirm whether this PR would indeed mitigate their job hangings. If we remain low on memory, we will keep getting OOM and yes, we will get into live-lock. However, the difference is in the original implementation, if we ever hit a single OOM, we will end up with a bad connection for this JVM permanently and all RPC requests to that destination will be blocked forever. For gobblin, they'd have to manually delete that JVM (and recreate a new one). With the new code, we don't have that problem and can tolerate certain degree of OOMs. And the expectation is we won't be hitting OOM all the time. If we are hitting OOM all the time, we should look for memory leak or bump memory/heap size. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
