KKcorps opened a new pull request, #15978: URL: https://github.com/apache/pinot/pull/15978
This pull request refactors the handling of the `StreamMetadataProvider` in the `RealtimeSegmentDataManager` class to improve thread safety, reduce redundant object creation, and enhance error handling. The key changes include replacing the direct reference with an `AtomicReference`, introducing a new method to manage the creation and retrieval of the provider, and updating all relevant usages to align with the new approach. ### Thread safety improvements: * Replaced the `_partitionMetadataProvider` field with an `AtomicReference` to ensure thread-safe updates to the `StreamMetadataProvider`. (`[pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.javaL306-R306](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL306-R306)`) ### Refactoring of metadata provider management: * Introduced a new method, `getOrCreatePartitionMetadataProvider`, to centralize the logic for creating or retrieving the `StreamMetadataProvider`. This method ensures that only one instance is created and shared among threads, while handling redundant object closures. (`[pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.javaL1959-R1996](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL1959-R1996)`) ### Updates to existing methods: * Updated methods like `getPartitionToLagState`, `fetchStreamOffset`, and `setPartitionParameters` to use `getOrCreatePartitionMetadataProvider` instead of directly accessing or creating the `StreamMetadataProvider`. This ensures consistent usage of the new thread-safe approach. (`[[1]](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL1060-R1065)`, `[[2]](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL1846-R1855)`, `[[3]](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL1891-R1899)`) * Added null checks and appropriate logging for scenarios where the `StreamMetadataProvider` could not be created. (`[pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.javaR1908-R1915](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaR1908-R1915)`) ### Cleanup enhancements: * Updated the `closePartitionMetadataProvider` method to use `AtomicReference.getAndSet` for safely clearing the provider and ensuring proper cleanup. (`[pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.javaL1321-R1326](diffhunk://#diff-fd2e3e273d7742190b27e7b26472460ff5dc6038f895e8647972b205fa8edccaL1321-R1326)`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
