qbx2 opened a new pull request, #55: URL: https://github.com/apache/flink-connector-kafka/pull/55
# What is the purpose of the change In the CachingTopicSelector, there is possibility to cause memory leak when the internal logic fails to check cache size by race condition. (https://github.com/apache/flink-connector-kafka/blob/d89a082180232bb79e3c764228c4e7dbb9eb6b8b/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/sink/KafkaRecordSerializationSchemaBuilder.java#L287-L289) This PR fixes the memory leak issue by changing the logic to be tolerable to the failure. # Brief change log Fix memory leak in CachingTopicSelector that can be triggered by race condition. # Verifying this change  - By analyzing the java heap dump, I identified memory leak in CachingTopicSelector. As in the screenshot, cache has 47,769 elements. If the internal logic didn't fail, the number of elements should be less than or equal to CACHE_RESET_SIZE (which is 5). - Because writing unit tests for this kind of bug, I applied a hotfix to our production workload. Before applying the hotfix, the memory leak is observed in the workload in 7 days. After the patch, the issue is not being observed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
