ChenXi created KAFKA-21490:
------------------------------
Summary: Broker shutdown does not release cached share partitions
and session metrics
Key: KAFKA-21490
URL: https://issues.apache.org/jira/browse/KAFKA-21490
Project: Kafka
Issue Type: Bug
Components: core, unit tests
Reporter: ChenXi
*Problem*
When a broker shuts down, {{SharePartitionManager.close()}} does not fully
release the resources it owns.
Two related cleanup gaps exist:
# Cached {{SharePartition}} instances are not closed.
* {{SharePartitionManager.close()}} does not drain the partition cache.
* {{SharePartition.close()}} is responsible for deregistering per-partition
metrics.
* Cached partitions and their state therefore remain referenced after manager
shutdown.
# {{ShareSessionCache}} has no close lifecycle.
* The cache registers {{{}ShareSessionsCount{}}},
{{{}SharePartitionsCount{}}}, and {{ShareSessionEvictionsPerSec}} in its
constructor.
* The gauge suppliers capture the cache instance.
* {{SharePartitionManager.close()}} does not clear the session cache or
deregister its metrics.
* Session, group member, partition count, and connection-mapping state remain
reachable through the old cache.
As a result, broker or manager shutdown can leave metrics registered and
cache-owned state unreachable by the application but still referenced by the
global Yammer registry.
*Expected Behavior*
When {{SharePartitionManager}} is closed:
* every cached {{SharePartition}} is removed from the partition cache and
closed;
* per-partition metrics are deregistered;
* partition listeners are removed and delayed share-fetch waiters are notified;
* the owned {{ShareSessionCache}} is closed;
* its three metrics are deregistered;
* session, group member, partition count, and connection mappings are cleared;
* manager close is idempotent;
* a failure while cleaning one partition or the session cache does not prevent
the timer from being closed;
* the timer is closed last.
*Steps to Reproduce*
# Create a {{SharePartitionManager}} with:
* one or more cached {{SharePartition}} instances;
* a {{ShareSessionCache}} containing at least one session.
# Call {{{}sharePartitionManager.close(){}}}.
# Inspect the partition cache and the Yammer metrics registry.
Current behavior:
* cached partitions remain in the partition cache;
* per-partition metrics remain registered;
* session-cache metrics remain registered;
* session-cache state and connection mappings are not cleared.
Note:In a normal production broker shutdown, the JVM exits immediately
afterward, so these objects are reclaimed by process termination rather than by
the share component.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)