[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417411#comment-16417411
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5734
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411154#comment-16411154
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user sihuazhou commented on the issue:
https://github.com/apache/flink/pull/5734
Hi @tillrohrmann could you please have a look at this PR?
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407926#comment-16407926
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user sihuazhou commented on the issue:
https://github.com/apache/flink/pull/5734
CC: @zentol I've addressed your comments and Travis given green light,
could you please have a look again?
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407703#comment-16407703
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user sihuazhou commented on a diff in the pull request:
https://github.com/apache/flink/pull/5734#discussion_r176029020
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskExecutor.java
---
@@ -292,6 +292,13 @@ public void start() throws Exception {
throwable = ExceptionUtils.firstOrSuppressed(t,
throwable);
}
+ try {
+ // it will call close() recursively from the parent to
children
+ taskManagerMetricGroup.close();
--- End diff --
I was intended to catch maybe some `RuntimeException`... I will just remove
the `try catch`.
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407695#comment-16407695
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5734#discussion_r176028191
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskExecutor.java
---
@@ -292,6 +292,13 @@ public void start() throws Exception {
throwable = ExceptionUtils.firstOrSuppressed(t,
throwable);
}
+ try {
+ // it will call close() recursively from the parent to
children
+ taskManagerMetricGroup.close();
--- End diff --
this method never throws exceptions
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407696#comment-16407696
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5734#discussion_r176028213
--- Diff:
flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
---
@@ -265,7 +265,11 @@ class TaskManager(
case t: Exception => log.error("FileCache did not shutdown
properly.", t)
}
-taskManagerMetricGroup.close()
+try {
+ taskManagerMetricGroup.close()
--- End diff --
same as above
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407688#comment-16407688
]
ASF GitHub Bot commented on FLINK-9026:
---
Github user sihuazhou commented on the issue:
https://github.com/apache/flink/pull/5734
CC: @tillrohrmann
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407686#comment-16407686
]
ASF GitHub Bot commented on FLINK-9026:
---
GitHub user sihuazhou opened a pull request:
https://github.com/apache/flink/pull/5734
[FLINK-9026][Metrics] Close the TaskManagerMetricGroup when the
TaskExecutor is shut down
## What is the purpose of the change
We should close the `TaskManagerMetricGroup` when the `TaskExecutor` is
shutdown.
## Brief change log
- close the `TaskManagerMetricGroup` when the `TaskExecutor` is shutdown.
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes /
**no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (not applicable / docs /
JavaDocs / **not documented**)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sihuazhou/flink FLINK_9026
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5734.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5734
commit 94ecbc87e53b4fff306a864971f164c765122194
Author: sihuazhou
Date: 2018-03-21T09:46:31Z
close the TaskManagerMetricGroup when the TaskExecutor is shut down
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407601#comment-16407601
]
Till Rohrmann commented on FLINK-9026:
--
I think it is often better to close/unregister resources in the scope where
they have been opened/registered. This makes resource management much easier.
But then we should at least close the {{TaskManagerMetricGroup}} when the
{{TaskExecutor}} is shut down.
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407403#comment-16407403
]
Sihua Zhou commented on FLINK-9026:
---
Hi [~till.rohrmann] I'm a bit confused about "We should unregister Tasks from
the TaskManagerMetricGroup when they have reached a final state", I found the
{{Task}} will close the {{TaskMetricGroup}} they held when they reach a final
state, the {{TaskMetricGroup.close()}} will call
{{parent.removeTaskMetricGroup(executionId);}}, so the {{TaskMetricGroup}} will
be unregistered from the {{TaskManagerMetricGroup}} currently.
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Assignee: Sihua Zhou
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406000#comment-16406000
]
Till Rohrmann commented on FLINK-9026:
--
No, I haven't started yet. Please go ahead and take it.
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9026) Unregister finished tasks from TaskManagerMetricGroup and close it
[
https://issues.apache.org/jira/browse/FLINK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405130#comment-16405130
]
Sihua Zhou commented on FLINK-9026:
---
Hi [~till.rohrmann] have you already work on this? Or I'd like to take this
ticket.
> Unregister finished tasks from TaskManagerMetricGroup and close it
> --
>
> Key: FLINK-9026
> URL: https://issues.apache.org/jira/browse/FLINK-9026
> Project: Flink
> Issue Type: Bug
> Components: Metrics
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Till Rohrmann
>Priority: Major
> Labels: flip-6
> Fix For: 1.5.0
>
>
> We should unregister {{Tasks}} from the {{TaskManagerMetricGroup}} when they
> have reached a final state. Moreover, we should close the
> {{TaskManagerMetricGroup}} either in the {{TaskExecutor#postStop}} method or
> let the caller do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
