[
https://issues.apache.org/jira/browse/FLINK-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-10761.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.7.2
1.6.4
master: aa92438c4aa6218c5d7e67083152733271697089
1.7: 45d53d27e4dec934057404674e38c6eab2e66e62
1.6: cb85e62b5b122b7cbb9eeb72a116dd133467a920
> MetricGroup#getAllVariables can deadlock
> ----------------------------------------
>
> Key: FLINK-10761
> URL: https://issues.apache.org/jira/browse/FLINK-10761
> Project: Flink
> Issue Type: Bug
> Components: Metrics
> Affects Versions: 1.5.5, 1.6.2, 1.7.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.6.4, 1.7.2, 1.8.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{AbstractMetricGroup#getAllVariables}} acquires the locks of both the
> current and all parent groups when assembling the variables map. This can
> lead to a deadlock if metrics are registered concurrently on a child and
> parent if the child registration is applied first and the reporter uses said
> method (which many do).
> Assume we have a MetricGroup Mc(hild) and Mp(arent).
> 2 separate threads Tc and Tp each register a metric on their respective
> group, acquiring the lock.
> Let's assume that Tc has a slight headstart.
> Tc will now call {{MetricRegistry#register}} first, acquiring the MR lock.
> Tp will block on this lock.
> Tc now iterates over all reporters calling
> {{MetricReporter#notifyOfAddedMetric}}. Assume that in this method
> {{MetricGroup#getAllVariables}} is called on Mc by Tc.
> Tc still holds the lock to Mc, and attempts to acquire the lock to Mp.
> The lock to Mp is still held by Tp however, which waits for the MR lock to be
> released by Tc.
> Thus a deadlock is created. This may deadlock anything, be it minor threads,
> tasks, or entire components.
> This has not surfaced so far since usually metrics are no longer added to a
> group once children have been created (since the component initialization at
> that point is complete).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
