[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=305503=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305503 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 03/Sep/19 09:49 Start Date: 03/Sep/19 09:49 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-527388066 Thanks @avijayanhwx for the contribution. I have committed this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305503) Time Spent: 4h 20m (was: 4h 10m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=305499=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305499 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 03/Sep/19 09:48 Start Date: 03/Sep/19 09:48 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-527387521 Thanks @avijayanhwx for working on this. I am +1 on the change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305499) Time Spent: 4h (was: 3h 50m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=305500=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305500 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 03/Sep/19 09:48 Start Date: 03/Sep/19 09:48 Worklog Time Spent: 10m Work Description: bshashikant commented on pull request #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305500) Time Spent: 4h 10m (was: 4h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=305358=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305358 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 03/Sep/19 00:27 Start Date: 03/Sep/19 00:27 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-527262612 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 36 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 31 | Maven dependency ordering for branch | | +1 | mvninstall | 645 | trunk passed | | +1 | compile | 429 | trunk passed | | +1 | checkstyle | 92 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 915 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 181 | trunk passed | | 0 | spotbugs | 436 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 640 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 41 | Maven dependency ordering for patch | | +1 | mvninstall | 553 | the patch passed | | +1 | compile | 386 | the patch passed | | +1 | javac | 386 | the patch passed | | +1 | checkstyle | 87 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 731 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 175 | the patch passed | | +1 | findbugs | 658 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 290 | hadoop-hdds in the patch passed. | | -1 | unit | 1716 | hadoop-ozone in the patch failed. | | +1 | asflicense | 51 | The patch does not generate ASF License warnings. | | | | 7924 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestOzoneManagerHA | | | hadoop.ozone.om.TestSecureOzoneManager | | | hadoop.hdds.scm.pipeline.TestRatisPipelineProvider | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle shellcheck shelldocs | | uname | Linux d862b077c625 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 915cbc9 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/6/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/6/testReport/ | | Max. process+thread count | 5404 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service hadoop-ozone/dist hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/6/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305358) Time Spent: 3h 50m (was: 3h 40m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=304018=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-304018 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 30/Aug/19 03:23 Start Date: 30/Aug/19 03:23 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-526440912 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 75 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 72 | Maven dependency ordering for branch | | +1 | mvninstall | 612 | trunk passed | | +1 | compile | 386 | trunk passed | | +1 | checkstyle | 85 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 866 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 182 | trunk passed | | 0 | spotbugs | 441 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 645 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 42 | Maven dependency ordering for patch | | +1 | mvninstall | 564 | the patch passed | | +1 | compile | 391 | the patch passed | | +1 | javac | 391 | the patch passed | | +1 | checkstyle | 89 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 759 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 178 | the patch passed | | +1 | findbugs | 665 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 335 | hadoop-hdds in the patch passed. | | -1 | unit | 2115 | hadoop-ozone in the patch failed. | | +1 | asflicense | 53 | The patch does not generate ASF License warnings. | | | | 8398 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestSecureOzoneManager | | | hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion | | | hadoop.ozone.TestOzoneConfigurationFields | | | hadoop.ozone.scm.node.TestQueryNode | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle shellcheck shelldocs | | uname | Linux fa4a4cf2f8f9 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 7bebad6 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/5/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/5/testReport/ | | Max. process+thread count | 5409 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service hadoop-ozone/dist hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/5/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 304018) Time Spent: 3h 40m (was: 3.5h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303880=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303880 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 29/Aug/19 20:12 Start Date: 29/Aug/19 20:12 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-526342725 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 146 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 76 | Maven dependency ordering for branch | | +1 | mvninstall | 631 | trunk passed | | +1 | compile | 404 | trunk passed | | +1 | checkstyle | 77 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 927 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | trunk passed | | 0 | spotbugs | 432 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 637 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 37 | Maven dependency ordering for patch | | +1 | mvninstall | 588 | the patch passed | | +1 | compile | 396 | the patch passed | | +1 | javac | 396 | the patch passed | | +1 | checkstyle | 77 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 729 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | the patch passed | | +1 | findbugs | 738 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 388 | hadoop-hdds in the patch passed. | | -1 | unit | 2234 | hadoop-ozone in the patch failed. | | +1 | asflicense | 104 | The patch does not generate ASF License warnings. | | | | 8695 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | | hadoop.ozone.client.rpc.TestOzoneRpcClientForAclAuditLog | | | hadoop.ozone.TestOzoneConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 7f6f6a097d5b 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f600fbb | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/4/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/4/testReport/ | | Max. process+thread count | 4913 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/4/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303880) Time Spent: 3.5h (was: 3h 20m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent:
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303878 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 29/Aug/19 20:10 Start Date: 29/Aug/19 20:10 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-526342286 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 83 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 64 | Maven dependency ordering for branch | | +1 | mvninstall | 591 | trunk passed | | +1 | compile | 393 | trunk passed | | +1 | checkstyle | 83 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 902 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 184 | trunk passed | | 0 | spotbugs | 475 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 706 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 38 | Maven dependency ordering for patch | | +1 | mvninstall | 580 | the patch passed | | +1 | compile | 424 | the patch passed | | +1 | javac | 424 | the patch passed | | +1 | checkstyle | 85 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 699 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 182 | the patch passed | | +1 | findbugs | 669 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 253 | hadoop-hdds in the patch failed. | | -1 | unit | 2441 | hadoop-ozone in the patch failed. | | +1 | asflicense | 52 | The patch does not generate ASF License warnings. | | | | 8620 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.ozoneimpl.TestOzoneContainer | | | hadoop.hdds.scm.pipeline.TestRatisPipelineProvider | | | hadoop.ozone.TestOzoneConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6538f14d34bb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f600fbb | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/3/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/3/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/3/testReport/ | | Max. process+thread count | 5058 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303878) Time Spent: 3h 20m (was: 3h 10m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major >
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303818=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303818 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 29/Aug/19 17:46 Start Date: 29/Aug/19 17:46 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-526290906 > Thanks @avijayanhwx for updating. Can we also add some tests for the added metric in TestCSMMetrics ? > Sorry for not mentioning it in the earlier review Added unit test for the metrics. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303818) Time Spent: 3h 10m (was: 3h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303568 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 29/Aug/19 10:18 Start Date: 29/Aug/19 10:18 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-526122511 Thanks @avijayanhwx for updating. Can we also add some tests for the added metric in TestCSMMetrics ? Sorry for not mentioning it in the earlier review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303568) Time Spent: 3h (was: 2h 50m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303314 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 29/Aug/19 00:46 Start Date: 29/Aug/19 00:46 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525975961 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 90 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 624 | trunk passed | | +1 | compile | 354 | trunk passed | | +1 | checkstyle | 69 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 921 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | trunk passed | | 0 | spotbugs | 418 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 626 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 547 | the patch passed | | +1 | compile | 372 | the patch passed | | +1 | javac | 372 | the patch passed | | +1 | checkstyle | 75 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 727 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 159 | the patch passed | | +1 | findbugs | 658 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 347 | hadoop-hdds in the patch passed. | | -1 | unit | 2977 | hadoop-ozone in the patch failed. | | +1 | asflicense | 57 | The patch does not generate ASF License warnings. | | | | 8944 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.TestOzoneConfigurationFields | | | hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient | | | hadoop.ozone.client.rpc.Test2WayCommitInRatis | | | hadoop.ozone.client.rpc.TestOzoneRpcClient | | | hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | | hadoop.ozone.TestStorageContainerManager | | | hadoop.ozone.TestSecureOzoneCluster | | | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | | hadoop.ozone.client.rpc.TestContainerStateMachineFailures | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c24b9fed27ec 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 872cdf4 | | Default Java | 1.8.0_212 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/2/testReport/ | | Max. process+thread count | 5365 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service U: hadoop-hdds/container-service | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303314) Time Spent: 2h 50m (was: 2h 40m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 >
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303086 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 18:28 Start Date: 28/Aug/19 18:28 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525866953 > Thanks @avijayanhwx . Yes, the per op applyTransaction latency metrics is already there but we don't have writeStateMachineData latency metric which again runs in a different executor. Can we add that as well. This can be done as a separate jira as well. Added rate metric for writeStateMachineData as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303086) Time Spent: 2h 40m (was: 2.5h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[
https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303074=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303074
]
ASF GitHub Bot logged work on HDDS-1783:
Author: ASF GitHub Bot
Created on: 28/Aug/19 18:14
Start Date: 28/Aug/19 18:14
Worklog Time Spent: 10m
Work Description: adoroszlai commented on pull request #1363: HDDS-1783 :
Latency metric for applyTransaction in ContainerStateMach…
URL: https://github.com/apache/hadoop/pull/1363#discussion_r318723464
##
File path:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/CSMMetrics.java
##
@@ -181,6 +183,11 @@ public long getNumBytesCommittedCount() {
return numBytesCommittedCount.value();
}
+ @VisibleForTesting
+ public MutableRate getApplyTransactionLatency() {
Review comment:
Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 303074)
Time Spent: 2.5h (was: 2h 20m)
> Latency metric for applyTransaction in ContainerStateMachine
>
>
> Key: HDDS-1783
> URL: https://issues.apache.org/jira/browse/HDDS-1783
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Datanode
>Reporter: Supratim Deka
>Assignee: Aravindan Vijayan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> applyTransaction is invoked from the Ratis pipeline and the
> ContainerStateMachine
> uses a async executor to complete the task.
>
> We require a latency metric to track the performance of log apply operations
> in the state machine. This will measure the end-to-end latency of apply which
> includes the queueing delay in the executor queues. Combined with the latency
> measurement in HddsDispatcher, this will be an indicator if the executors are
> overloaded.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=303062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303062 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 17:24 Start Date: 28/Aug/19 17:24 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525841965 Thanks @avijayanhwx . Yes, the per op applyTransaction latency metrics is already there but we don't have writeStateMachineData latency metric which again runs in a different executor. Can we add that as well. This can be done as a separate jira as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303062) Time Spent: 2h 20m (was: 2h 10m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302987 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:55 Start Date: 28/Aug/19 15:55 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525808488 > Thanks @avijayanhwx for working on this. The changes look good. In applyTransaction, we execute different cmds which will vary with latency. For example, for a write chunk commit, its a rename operation but for putBlock , it is Rocks DB operation and for close container, its again a disk sync operation of container state. Depending upon the operation the latency will vary as well. Can we have per cmd latency metric? @bshashikant Isn't the per operation latency covered in org.apache.hadoop.ozone.container.common.transport.server.ratis.CSMMetrics#opsLatency? This metric was only intended to cover the whole latency including thread scheduling. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302987) Time Spent: 2h (was: 1h 50m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302985 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:55 Start Date: 28/Aug/19 15:55 Worklog Time Spent: 10m Work Description: avijayanhwx commented on pull request #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302985) Time Spent: 1h 40m (was: 1.5h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302988 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:55 Start Date: 28/Aug/19 15:55 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525807821 @bshashikant Isn't the per operation latency covered in org.apache.hadoop.ozone.container.common.transport.server.ratis.CSMMetrics#opsLatency? This metric was only intended to cover the whole latency including thread scheduling. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302988) Time Spent: 2h 10m (was: 2h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302986 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:55 Start Date: 28/Aug/19 15:55 Worklog Time Spent: 10m Work Description: avijayanhwx commented on pull request #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363 …ine. applyTransaction is invoked from the Ratis pipeline and the ContainerStateMachine uses a async executor to complete the task. We require a latency metric to track the performance of log apply operations in the state machine. This will measure the end-to-end latency of apply which includes the queueing delay in the executor queues. Combined with the latency measurement in HddsDispatcher, this will be an indicator if the executors are overloaded. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302986) Time Spent: 1h 50m (was: 1h 40m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302984=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302984 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:55 Start Date: 28/Aug/19 15:55 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525808488 > Thanks @avijayanhwx for working on this. The changes look good. In applyTransaction, we execute different cmds which will vary with latency. For example, for a write chunk commit, its a rename operation but for putBlock , it is Rocks DB operation and for close container, its again a disk sync operation of container state. Depending upon the operation the latency will vary as well. Can we have per cmd latency metric? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302984) Time Spent: 1.5h (was: 1h 20m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[
https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302983=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302983
]
ASF GitHub Bot logged work on HDDS-1783:
Author: ASF GitHub Bot
Created on: 28/Aug/19 15:54
Start Date: 28/Aug/19 15:54
Worklog Time Spent: 10m
Work Description: avijayanhwx commented on pull request #1363: HDDS-1783
: Latency metric for applyTransaction in ContainerStateMach…
URL: https://github.com/apache/hadoop/pull/1363#discussion_r318661205
##
File path:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/CSMMetrics.java
##
@@ -181,6 +183,11 @@ public long getNumBytesCommittedCount() {
return numBytesCommittedCount.value();
}
+ @VisibleForTesting
+ public MutableRate getApplyTransactionLatency() {
Review comment:
Thanks @adoroszlai . I will remove the annotation, and keep the API since we
may use it in the future.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 302983)
Time Spent: 1h 20m (was: 1h 10m)
> Latency metric for applyTransaction in ContainerStateMachine
>
>
> Key: HDDS-1783
> URL: https://issues.apache.org/jira/browse/HDDS-1783
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Datanode
>Reporter: Supratim Deka
>Assignee: Aravindan Vijayan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> applyTransaction is invoked from the Ratis pipeline and the
> ContainerStateMachine
> uses a async executor to complete the task.
>
> We require a latency metric to track the performance of log apply operations
> in the state machine. This will measure the end-to-end latency of apply which
> includes the queueing delay in the executor queues. Combined with the latency
> measurement in HddsDispatcher, this will be an indicator if the executors are
> overloaded.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302982 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 15:53 Start Date: 28/Aug/19 15:53 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525807821 @bshashikant Isn't the per operation latency covered in org.apache.hadoop.ozone.container.common.transport.server.ratis.CSMMetrics#opsLatency? This metric was only intended to cover the whole latency including thread scheduling. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302982) Time Spent: 1h 10m (was: 1h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302839 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 13:15 Start Date: 28/Aug/19 13:15 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525740015 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 37 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 609 | trunk passed | | +1 | compile | 364 | trunk passed | | +1 | checkstyle | 72 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 934 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 167 | trunk passed | | 0 | spotbugs | 430 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 628 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 546 | the patch passed | | +1 | compile | 369 | the patch passed | | +1 | javac | 369 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 734 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 163 | the patch passed | | +1 | findbugs | 643 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 305 | hadoop-hdds in the patch passed. | | -1 | unit | 1825 | hadoop-ozone in the patch failed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 7671 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1363 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6829995c22d3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 55cc115 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/1/testReport/ | | Max. process+thread count | 4360 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/container-service U: hadoop-hdds/container-service | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1363/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302839) Time Spent: 1h (was: 50m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[
https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302726
]
ASF GitHub Bot logged work on HDDS-1783:
Author: ASF GitHub Bot
Created on: 28/Aug/19 10:04
Start Date: 28/Aug/19 10:04
Worklog Time Spent: 10m
Work Description: adoroszlai commented on pull request #1363: HDDS-1783 :
Latency metric for applyTransaction in ContainerStateMach…
URL: https://github.com/apache/hadoop/pull/1363#discussion_r318496771
##
File path:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/CSMMetrics.java
##
@@ -181,6 +183,11 @@ public long getNumBytesCommittedCount() {
return numBytesCommittedCount.value();
}
+ @VisibleForTesting
+ public MutableRate getApplyTransactionLatency() {
Review comment:
Why is this exposed? I don't see it in any tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 302726)
Time Spent: 50m (was: 40m)
> Latency metric for applyTransaction in ContainerStateMachine
>
>
> Key: HDDS-1783
> URL: https://issues.apache.org/jira/browse/HDDS-1783
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Datanode
>Reporter: Supratim Deka
>Assignee: Aravindan Vijayan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> applyTransaction is invoked from the Ratis pipeline and the
> ContainerStateMachine
> uses a async executor to complete the task.
>
> We require a latency metric to track the performance of log apply operations
> in the state machine. This will measure the end-to-end latency of apply which
> includes the queueing delay in the executor queues. Combined with the latency
> measurement in HddsDispatcher, this will be an indicator if the executors are
> overloaded.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302708 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 09:28 Start Date: 28/Aug/19 09:28 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525662020 Thanks @avijayanhwx for working on this. The changes look good. In applyTransaction, we execute different cmds which will vary with latency. For example, for a write chunk commit, its a rename operation but for putBlock , it is Rocks DB operation and for close container, its again a disk sync operation of container state. Depending upon the operation the latency will vary as well. Can we have per cmd latency metric? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302708) Time Spent: 40m (was: 0.5h) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302707 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 09:27 Start Date: 28/Aug/19 09:27 Worklog Time Spent: 10m Work Description: bshashikant commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525662020 Thanks @avijayanhwx for working on this. The changes look good. Inn applyTransaction, we execute different cmds which will vary with latency. For example, for a write chunk commit, its a rename operation but for putBlock , it is Rocks DB operation and for close container, its again a disk sync operation of container state. Depending upon the operation the latency will vary as well. Can we have per cmd latency metric? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302707) Time Spent: 0.5h (was: 20m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302536=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302536 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 05:40 Start Date: 28/Aug/19 05:40 Worklog Time Spent: 10m Work Description: avijayanhwx commented on pull request #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363 …ine. applyTransaction is invoked from the Ratis pipeline and the ContainerStateMachine uses a async executor to complete the task. We require a latency metric to track the performance of log apply operations in the state machine. This will measure the end-to-end latency of apply which includes the queueing delay in the executor queues. Combined with the latency measurement in HddsDispatcher, this will be an indicator if the executors are overloaded. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302536) Remaining Estimate: 0h Time Spent: 10m > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1783) Latency metric for applyTransaction in ContainerStateMachine
[ https://issues.apache.org/jira/browse/HDDS-1783?focusedWorklogId=302537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302537 ] ASF GitHub Bot logged work on HDDS-1783: Author: ASF GitHub Bot Created on: 28/Aug/19 05:40 Start Date: 28/Aug/19 05:40 Worklog Time Spent: 10m Work Description: avijayanhwx commented on issue #1363: HDDS-1783 : Latency metric for applyTransaction in ContainerStateMach… URL: https://github.com/apache/hadoop/pull/1363#issuecomment-525592439 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302537) Time Spent: 20m (was: 10m) > Latency metric for applyTransaction in ContainerStateMachine > > > Key: HDDS-1783 > URL: https://issues.apache.org/jira/browse/HDDS-1783 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Aravindan Vijayan >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > applyTransaction is invoked from the Ratis pipeline and the > ContainerStateMachine > uses a async executor to complete the task. > > We require a latency metric to track the performance of log apply operations > in the state machine. This will measure the end-to-end latency of apply which > includes the queueing delay in the executor queues. Combined with the latency > measurement in HddsDispatcher, this will be an indicator if the executors are > overloaded. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
