[
https://issues.apache.org/jira/browse/HADOOP-10468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061022#comment-14061022
]
Haohui Mai commented on HADOOP-10468:
-------------------------------------
Looking at the code for a while to refresh my memory. Here is a quick recap:
# Metric2 implements hierarchical configuration with {{SubsetConfiguration}}.
The configuration key has the format {{foo.bar.spam}}, where each parts of the
configuration is a hierarchy. For example, {{spam}} is a child of {{bar}}, and
{{bar}} is a children of {{foo}. Metric2 turns some parts of the hierarchy keys
in lowercase: (MetricsConfig:87)
{code}
MetricsConfig(Configuration c, String prefix) {
super(c, prefix.toLowerCase(Locale.US), ".");
}
{code}
All keys of immediate children will be stored in lowercase strings
(MetricConfig:151)
{code}
Map<String, MetricsConfig> getInstanceConfigs(String type) {
Map<String, MetricsConfig> map = Maps.newHashMap();
MetricsConfig sub = subset(type);
for (String key : sub.keys()) {
Matcher matcher = INSTANCE_REGEX.matcher(key);
if (matcher.matches()) {
String instance = matcher.group(1);
if (!map.containsKey(instance)) {
map.put(instance, sub.subset(instance));
}
}
}
return map;
}
{code}
# To look up the value in the hierarchical configuration, {{MetricConfig}}
first finds the key in itself and then reconstructs the full key for its parent
by concatenating the prefix, the delimiter and the key itself
(SubsetConfiguration:88):
{code}
protected String getParentKey(String key)
{
if ("".equals(key) || key == null)
{
return prefix;
}
else
{
return delimiter == null ? prefix + key : prefix + delimiter + key;
}
}
{code}
In this test case, the reconstructed key is
{{test.sink.collector.queue.capacity}} instead of
{{test.sink.Collector.queue.capacity}}, which leads to the failure.
> TestMetricsSystemImpl.testMultiThreadedPublish fails intermediately
> -------------------------------------------------------------------
>
> Key: HADOOP-10468
> URL: https://issues.apache.org/jira/browse/HADOOP-10468
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.5.0
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Priority: Blocker
> Attachments: HADOOP-10468.000.patch, HADOOP-10468.001.patch,
> HADOOP-10468.2.patch
>
>
> {{TestMetricsSystemImpl.testMultiThreadedPublish}} can fail intermediately
> due to the insufficient size of the sink queue:
> {code}
> 2014-04-06 21:34:55,269 WARN impl.MetricsSinkAdapter
> (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
> queue and can't consume the given metrics.
> 2014-04-06 21:34:55,270 WARN impl.MetricsSinkAdapter
> (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
> queue and can't consume the given metrics.
> 2014-04-06 21:34:55,271 WARN impl.MetricsSinkAdapter
> (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full
> queue and can't consume the given metrics.
> {code}
> The unit test should increase the default queue size to avoid intermediate
> failure.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
