[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/2650


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[ex00]

Github user ex00 commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r88007685
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("A", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName));
+   // ignore all next filters for scope -  because 
scopeString cached with only first filter
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, this));
+   assertEquals("A-B-C-D-4", 
group.getMetricIdentifier(metricName, new CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", 
"RR").replace("1", "4");
+   }
+   }));
+   countSuccessChecks++;
+   }
+   }
+   public static class TestReporter2 extends TestReporter1 {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, this));
+   // ignore all next filters -  because scopeString 
cached with only first filter
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName));
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+  

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r87781042
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
--- End diff --

we can store these in a field instead. We can retrieve all reporters from 
the registry and cast them to the required type, and use a new getter to 
retrieve the count.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r87780687
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("A", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName));
+   // ignore all next filters for scope -  because 
scopeString cached with only first filter
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, this));
+   assertEquals("A-B-C-D-4", 
group.getMetricIdentifier(metricName, new CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", 
"RR").replace("1", "4");
+   }
+   }));
+   countSuccessChecks++;
+   }
+   }
+   public static class TestReporter2 extends TestReporter1 {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, this));
+   // ignore all next filters -  because scopeString 
cached with only first filter
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName));
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r86991226
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r87781446
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("A", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName));
+   // ignore all next filters for scope -  because 
scopeString cached with only first filter
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, this));
+   assertEquals("A-B-C-D-4", 
group.getMetricIdentifier(metricName, new CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", 
"RR").replace("1", "4");
+   }
+   }));
+   countSuccessChecks++;
+   }
+   }
+   public static class TestReporter2 extends TestReporter1 {
--- End diff --

This is a weird inheritance chain. It would make more sense to create a 
common ScopeCheckingTestReporter class that encapsulates the counting, with an 
abstract method `checkScopes(Metric metric, String metricName, MetricGroup 
group)`.
The specific checks are then done in subclasses of that class.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r87780439
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("A", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName));
+   // ignore all next filters for scope -  because 
scopeString cached with only first filter
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, this));
+   assertEquals("A-B-C-D-4", 
group.getMetricIdentifier(metricName, new CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", 
"RR").replace("1", "4");
+   }
+   }));
+   countSuccessChecks++;
+   }
+   }
+   public static class TestReporter2 extends TestReporter1 {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, this));
+   // ignore all next filters -  because scopeString 
cached with only first filter
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName));
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r86765901
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
--- End diff --

for consistency purposes this should be replaced with `assertEquals(4, 
TestReporter1.countSuccessChecks)`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r87780651
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroupTest.java
 ---
@@ -44,4 +52,132 @@ protected QueryScopeInfo 
createQueryServiceMetricInfo(CharacterFilter filter) {

registry.shutdown();
}
+
+   // for test case: one filter for different reporters with different of 
scope delimiter
+   protected static CharacterFilter staticCharacterFilter = new 
CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("C", "RR");
+   }
+   };
+
+   @Test
+   public void filteringForMultipleReporters() {
+   TestReporter1.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   config.setString(ConfigConstants.METRICS_REPORTERS_LIST, 
"test1,test2");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter1.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, 
TestReporter2.class.getName());
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test1." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "-");
+   config.setString(ConfigConstants.METRICS_REPORTER_PREFIX + 
"test2." + ConfigConstants.METRICS_REPORTER_SCOPE_DELIMITER, "!");
+
+
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroup = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   tmGroup.counter(1);
+   testRegistry.shutdown();
+   assert TestReporter1.countSuccessChecks == 4;
+   }
+
+   @Test
+   public void filteringForNullReporters() {
+   MetricRegistryTest.countSuccessChecks = 0;
+   Configuration config = new Configuration();
+   config.setString(ConfigConstants.METRICS_SCOPE_NAMING_TM, 
"A.B.C.D");
+   MetricRegistry testRegistry = new 
MetricRegistryTest(MetricRegistryConfiguration.fromConfiguration(config));
+   TaskManagerMetricGroup tmGroupForTestRegistry = new 
TaskManagerMetricGroup(testRegistry, "host", "id");
+   assert testRegistry.getReporters().size() == 0;
+   tmGroupForTestRegistry.counter(1);
+   testRegistry.shutdown();
+   assert MetricRegistryTest.countSuccessChecks == 1;
+   }
+
+   public static class TestReporter1 extends TestReporter {
+   protected static int countSuccessChecks = 0;
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("A", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName));
+   // ignore all next filters for scope -  because 
scopeString cached with only first filter
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));
+   assertEquals("A-B-C-D-1", 
group.getMetricIdentifier(metricName, this));
+   assertEquals("A-B-C-D-4", 
group.getMetricIdentifier(metricName, new CharacterFilter() {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", 
"RR").replace("1", "4");
+   }
+   }));
+   countSuccessChecks++;
+   }
+   }
+   public static class TestReporter2 extends TestReporter1 {
+   @Override
+   public String filterCharacters(String input) {
+   return input.replace("B", "RR");
+   }
+   @Override
+   public void notifyOfAddedMetric(Metric metric, String 
metricName, MetricGroup group) {
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, this));
+   // ignore all next filters -  because scopeString 
cached with only first filter
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName));
+   assertEquals("A!RR!C!D!1", 
group.getMetricIdentifier(metricName, staticCharacterFilter));

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[ex00]

Github user ex00 commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r85543811
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroup.java
 ---
@@ -169,19 +176,7 @@ public String getMetricIdentifier(String metricName) {
 * @return fully qualified metric name
 */
public String getMetricIdentifier(String metricName, CharacterFilter 
filter) {
-   if (scopeString == null) {
-   if (filter != null) {
-   scopeString = ScopeFormat.concat(filter, 
registry.getDelimiter(), scopeComponents);
-   } else {
-   scopeString = 
ScopeFormat.concat(registry.getDelimiter(), scopeComponents);
-   }
-   }
-
-   if (filter != null) {
-   return scopeString + registry.getDelimiter() + 
filter.filterCharacters(metricName);
-   } else {
-   return scopeString + registry.getDelimiter() + 
metricName;
-   }
+   return getMetricIdentifier(metricName, filter, -1);
--- End diff --

if reporter index not [correct for 
array](https://github.com/apache/flink/pull/2650/files#diff-3cbfc4ef461f4bac5ddd197c77fe19b1R191)
 then in this block not use ``reporterIndex`` and call 
`MetricRegistry#getDelimiter()`
I will change to
``` java
if (scopeStrings.length == 0 || (reporterIndex < 0 || reporterIndex >= 
scopeStrings.length)) {
if (filter != null) {
String newScopeString = 
ScopeFormat.concat(filter, registry.getDelimiter(), scopeComponents);
return newScopeString + registry.getDelimiter() 
+ filter.filterCharacters(metricName);
} else {
String newScopeString = 
ScopeFormat.concat(registry.getDelimiter(), scopeComponents);
return newScopeString + registry.getDelimiter() 
+ metricName;
}
}
```
and warning message will not show



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r85533501
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroup.java
 ---
@@ -169,19 +176,7 @@ public String getMetricIdentifier(String metricName) {
 * @return fully qualified metric name
 */
public String getMetricIdentifier(String metricName, CharacterFilter 
filter) {
-   if (scopeString == null) {
-   if (filter != null) {
-   scopeString = ScopeFormat.concat(filter, 
registry.getDelimiter(), scopeComponents);
-   } else {
-   scopeString = 
ScopeFormat.concat(registry.getDelimiter(), scopeComponents);
-   }
-   }
-
-   if (filter != null) {
-   return scopeString + registry.getDelimiter() + 
filter.filterCharacters(metricName);
-   } else {
-   return scopeString + registry.getDelimiter() + 
metricName;
-   }
+   return getMetricIdentifier(metricName, filter, -1);
--- End diff --

If you call `getMetricIdentifier(...)` the reporter index will be passed to 
`MetricRegistry#getDelimiter()`. For a negative index this function logs a 
warning, assuming that some error has occurred that caused the index to be 
negative. Since you pass `-1` as the reporter index you will thus always 
trigger a warning.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r85533847
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroup.java
 ---
@@ -101,6 +105,8 @@ public AbstractMetricGroup(MetricRegistry registry, 
String[] scope, A parent) {
this.registry = checkNotNull(registry);
this.scopeComponents = checkNotNull(scope);
this.parent = parent;
+   scopeStrings = new String[registry.getReporters().size()];
--- End diff --

for consistency it would be neat to prefix these with `this.`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/2650#discussion_r85533745
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/metrics/groups/AbstractMetricGroup.java
 ---
@@ -85,9 +86,12 @@
 *  For example ["host-7", "taskmanager-2", "window_word_count", 
"my-mapper" ]. */
private final String[] scopeComponents;
 
-   /** The metrics scope represented by this group, as a concatenated 
string, lazily computed.
+   /** Array the metrics scope represented by this group, as a 
concatenated string, lazily computed.
--- End diff --

"Array containing the metrics scope represented by this group for each 
reporter, ..."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2650: [FLINK-4563] [metrics] scope caching not adjusted ...

[ex00]

GitHub user ex00 opened a pull request:

https://github.com/apache/flink/pull/2650

[FLINK-4563] [metrics] scope caching not adjusted for multiple reporters

Hello. 
It is implementation FLINK-4563.
In ``AbstractMetricGroup.java`` added characterFilter and 
firstReporterIndex for cached scopeString


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ex00/flink FLINK-4563

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/2650.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2650


commit ad3034b4d9764426dd5f969b20c12aa3145cbf84
Author: Anton Mushin 
Date:   2016-10-17T13:23:01Z

[FLINK-4563] [metrics] scope caching not adjusted for multiple reporters




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---