[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319861=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319861
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 22:26
Start Date: 27/Sep/19 22:26
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on pull request #1541: 
HDDS-1146. Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329271150
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
 ##
 @@ -66,16 +67,25 @@ public void onMessage(final 
IncrementalContainerReportFromDatanode report,
 nodeManager.addContainer(dd, id);
 processContainerReplica(dd, replicaProto);
   } catch (ContainerNotFoundException e) {
+success = false;
 LOG.warn("Container {} not found!", replicaProto.getContainerID());
   } catch (NodeNotFoundException ex) {
+success = false;
 LOG.error("Received ICR from unknown datanode {} {}",
 report.getDatanodeDetails(), ex);
   } catch (IOException e) {
+success = false;
 LOG.error("Exception while processing ICR for container {}",
 replicaProto.getContainerID());
   }
 }
 
+if (success) {
+  getContainerManager().notifyContainerReportProcessing(false, true);
+} else {
 
 Review comment:
   As discussed offline, will change this in later jira. For now we shall go 
with the current way.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319861)
Time Spent: 2h 50m  (was: 2h 40m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319859=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319859
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 22:24
Start Date: 27/Sep/19 22:24
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on issue #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536114953
 
 
   Test failures are not related to this patch.
   Thank You @anuengineer and @avijayanhwx for the review.
   I will commit this to the trunk.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319859)
Time Spent: 2.5h  (was: 2h 20m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319860=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319860
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 22:24
Start Date: 27/Sep/19 22:24
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on pull request #1541: 
HDDS-1146. Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319860)
Time Spent: 2h 40m  (was: 2.5h)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319858=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319858
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 22:22
Start Date: 27/Sep/19 22:22
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on pull request #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329270479
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
 ##
 @@ -66,16 +67,25 @@ public void onMessage(final 
IncrementalContainerReportFromDatanode report,
 nodeManager.addContainer(dd, id);
 processContainerReplica(dd, replicaProto);
   } catch (ContainerNotFoundException e) {
+success = false;
 LOG.warn("Container {} not found!", replicaProto.getContainerID());
   } catch (NodeNotFoundException ex) {
+success = false;
 LOG.error("Received ICR from unknown datanode {} {}",
 report.getDatanodeDetails(), ex);
   } catch (IOException e) {
+success = false;
 LOG.error("Exception while processing ICR for container {}",
 replicaProto.getContainerID());
   }
 }
 
+if (success) {
+  getContainerManager().notifyContainerReportProcessing(false, true);
+} else {
 
 Review comment:
   IMO, we should change it to 1 line instead of if-else. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319858)
Time Spent: 2h 20m  (was: 2h 10m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319851=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319851
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:50
Start Date: 27/Sep/19 21:50
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on issue #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536106044
 
 
   > @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
   
   If the user wants to know numContainerReportsProcessed successfully in 
system. And he wants to compare with failure count, then it might be useful.
   
   And also I got to know yesterday it would be good to have metrics in the 
system for some actions happened/performed. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319851)
Time Spent: 2h 10m  (was: 2h)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319847=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319847
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:43
Start Date: 27/Sep/19 21:43
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on issue #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536106044
 
 
   > @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
   
   If the user wants to know numContainerReportsProcessed successfully in 
system. And he wants to compare with failure count, then it might be useful.
   
   And also I got to know yesterday it would be good to have metrics in the 
system for some actions happened. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319847)
Time Spent: 2h  (was: 1h 50m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319846=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319846
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:43
Start Date: 27/Sep/19 21:43
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on issue #1541: HDDS-1146. Adding 
container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536101491
 
 
   @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319846)
Time Spent: 1h 50m  (was: 1h 40m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319844
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:33
Start Date: 27/Sep/19 21:33
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on issue #1541: HDDS-1146. Adding 
container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536101491
 
 
   @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
   
   If the user wants to know numContainerReportsProcessed successfully in 
system. And he wants to compare with failure count, then it might be useful.
   
   And also I got to know yesterday it would be good to have metrics in the 
system for some actions happened. 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319844)
Time Spent: 1h 40m  (was: 1.5h)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319843=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319843
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:32
Start Date: 27/Sep/19 21:32
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on issue #1541: HDDS-1146. Adding 
container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536101491
 
 
   @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
   
   If the user wants to know numContainerReportsProcessed successfully in 
system. And he wants to compare with failure count, then it might be useful.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319843)
Time Spent: 1.5h  (was: 1h 20m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319841=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319841
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:30
Start Date: 27/Sep/19 21:30
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on pull request #1541: 
HDDS-1146. Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329258003
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
 ##
 @@ -66,16 +67,25 @@ public void onMessage(final 
IncrementalContainerReportFromDatanode report,
 nodeManager.addContainer(dd, id);
 processContainerReplica(dd, replicaProto);
   } catch (ContainerNotFoundException e) {
+success = false;
 LOG.warn("Container {} not found!", replicaProto.getContainerID());
   } catch (NodeNotFoundException ex) {
+success = false;
 LOG.error("Received ICR from unknown datanode {} {}",
 report.getDatanodeDetails(), ex);
   } catch (IOException e) {
+success = false;
 LOG.error("Exception while processing ICR for container {}",
 replicaProto.getContainerID());
   }
 }
 
+if (success) {
+  getContainerManager().notifyContainerReportProcessing(false, true);
+} else {
 
 Review comment:
   We can do that, just due to copy/paste from ICR code it is like this. (End 
result is same)
   Let me know if you want to change it?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319841)
Time Spent: 1h 20m  (was: 1h 10m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319840=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319840
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:27
Start Date: 27/Sep/19 21:27
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on pull request #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329256932
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
 ##
 @@ -66,16 +67,25 @@ public void onMessage(final 
IncrementalContainerReportFromDatanode report,
 nodeManager.addContainer(dd, id);
 processContainerReplica(dd, replicaProto);
   } catch (ContainerNotFoundException e) {
+success = false;
 LOG.warn("Container {} not found!", replicaProto.getContainerID());
   } catch (NodeNotFoundException ex) {
+success = false;
 LOG.error("Received ICR from unknown datanode {} {}",
 report.getDatanodeDetails(), ex);
   } catch (IOException e) {
+success = false;
 LOG.error("Exception while processing ICR for container {}",
 replicaProto.getContainerID());
   }
 }
 
+if (success) {
+  getContainerManager().notifyContainerReportProcessing(false, true);
+} else {
 
 Review comment:
   Any reason we are not doing 
_getContainerManager().notifyContainerReportProcessing(false, success)_ ?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319840)
Time Spent: 1h 10m  (was: 1h)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319838=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319838
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:25
Start Date: 27/Sep/19 21:25
Worklog Time Spent: 10m 
  Work Description: avijayanhwx commented on issue #1541: HDDS-1146. Adding 
container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536101491
 
 
   @bharatviswa504 If you don't mind, can you tell me where a metric like 
numContainerReportsSuccess would be useful? Failures I guess may be useful.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319838)
Time Spent: 1h  (was: 50m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Bharat Viswanadham
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319833=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319833
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:06
Start Date: 27/Sep/19 21:06
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on issue #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536096677
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | 0 | reexec | 1737 | Docker mode activated. |
   ||| _ Prechecks _ |
   | +1 | dupname | 0 | No case conflicting files found. |
   | +1 | @author | 0 | The patch does not contain any @author tags. |
   | +1 | test4tests | 0 | The patch appears to include 1 new or modified test 
files. |
   ||| _ trunk Compile Tests _ |
   | 0 | mvndep | 77 | Maven dependency ordering for branch |
   | -1 | mvninstall | 48 | hadoop-hdds in trunk failed. |
   | -1 | mvninstall | 48 | hadoop-ozone in trunk failed. |
   | -1 | compile | 22 | hadoop-hdds in trunk failed. |
   | -1 | compile | 14 | hadoop-ozone in trunk failed. |
   | +1 | checkstyle | 68 | trunk passed |
   | +1 | mvnsite | 0 | trunk passed |
   | +1 | shadedclient | 947 | branch has no errors when building and testing 
our client artifacts. |
   | -1 | javadoc | 23 | hadoop-hdds in trunk failed. |
   | -1 | javadoc | 18 | hadoop-ozone in trunk failed. |
   | 0 | spotbugs | 1050 | Used deprecated FindBugs config; considering 
switching to SpotBugs. |
   | -1 | findbugs | 38 | hadoop-hdds in trunk failed. |
   | -1 | findbugs | 19 | hadoop-ozone in trunk failed. |
   ||| _ Patch Compile Tests _ |
   | 0 | mvndep | 29 | Maven dependency ordering for patch |
   | -1 | mvninstall | 35 | hadoop-hdds in the patch failed. |
   | -1 | mvninstall | 39 | hadoop-ozone in the patch failed. |
   | -1 | compile | 23 | hadoop-hdds in the patch failed. |
   | -1 | compile | 18 | hadoop-ozone in the patch failed. |
   | -1 | javac | 23 | hadoop-hdds in the patch failed. |
   | -1 | javac | 18 | hadoop-ozone in the patch failed. |
   | +1 | checkstyle | 62 | the patch passed |
   | +1 | mvnsite | 0 | the patch passed |
   | +1 | whitespace | 0 | The patch has no whitespace issues. |
   | +1 | shadedclient | 798 | patch has no errors when building and testing 
our client artifacts. |
   | -1 | javadoc | 19 | hadoop-hdds in the patch failed. |
   | -1 | javadoc | 18 | hadoop-ozone in the patch failed. |
   | -1 | findbugs | 31 | hadoop-hdds in the patch failed. |
   | -1 | findbugs | 19 | hadoop-ozone in the patch failed. |
   ||| _ Other Tests _ |
   | -1 | unit | 28 | hadoop-hdds in the patch failed. |
   | -1 | unit | 24 | hadoop-ozone in the patch failed. |
   | +1 | asflicense | 28 | The patch does not generate ASF License warnings. |
   | | | 4336 | |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.1 Server=19.03.1 base: 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/1541 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient findbugs checkstyle |
   | uname | Linux 8278c62831ff 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | personality/hadoop.sh |
   | git revision | trunk / 13b427f |
   | Default Java | 1.8.0_222 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-mvninstall-hadoop-hdds.txt
 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-mvninstall-hadoop-ozone.txt
 |
   | compile | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-compile-hadoop-hdds.txt
 |
   | compile | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-compile-hadoop-ozone.txt
 |
   | javadoc | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-javadoc-hadoop-hdds.txt
 |
   | javadoc | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-javadoc-hadoop-ozone.txt
 |
   | findbugs | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-findbugs-hadoop-hdds.txt
 |
   | findbugs | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/branch-findbugs-hadoop-ozone.txt
 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/1/artifact/out/patch-mvninstall-hadoop-hdds.txt
 |
   | mvninstall | 

[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319832=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319832
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 21:06
Start Date: 27/Sep/19 21:06
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on issue #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#issuecomment-536096562
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | 0 | reexec | 1407 | Docker mode activated. |
   ||| _ Prechecks _ |
   | +1 | dupname | 1 | No case conflicting files found. |
   | +1 | @author | 0 | The patch does not contain any @author tags. |
   | +1 | test4tests | 0 | The patch appears to include 1 new or modified test 
files. |
   ||| _ trunk Compile Tests _ |
   | 0 | mvndep | 40 | Maven dependency ordering for branch |
   | -1 | mvninstall | 48 | hadoop-hdds in trunk failed. |
   | -1 | mvninstall | 47 | hadoop-ozone in trunk failed. |
   | -1 | compile | 22 | hadoop-hdds in trunk failed. |
   | -1 | compile | 14 | hadoop-ozone in trunk failed. |
   | +1 | checkstyle | 68 | trunk passed |
   | +1 | mvnsite | 0 | trunk passed |
   | +1 | shadedclient | 949 | branch has no errors when building and testing 
our client artifacts. |
   | -1 | javadoc | 21 | hadoop-hdds in trunk failed. |
   | -1 | javadoc | 19 | hadoop-ozone in trunk failed. |
   | 0 | spotbugs | 1049 | Used deprecated FindBugs config; considering 
switching to SpotBugs. |
   | -1 | findbugs | 36 | hadoop-hdds in trunk failed. |
   | -1 | findbugs | 19 | hadoop-ozone in trunk failed. |
   ||| _ Patch Compile Tests _ |
   | 0 | mvndep | 28 | Maven dependency ordering for patch |
   | -1 | mvninstall | 36 | hadoop-hdds in the patch failed. |
   | -1 | mvninstall | 38 | hadoop-ozone in the patch failed. |
   | -1 | compile | 24 | hadoop-hdds in the patch failed. |
   | -1 | compile | 16 | hadoop-ozone in the patch failed. |
   | -1 | javac | 24 | hadoop-hdds in the patch failed. |
   | -1 | javac | 16 | hadoop-ozone in the patch failed. |
   | +1 | checkstyle | 61 | the patch passed |
   | +1 | mvnsite | 0 | the patch passed |
   | +1 | whitespace | 0 | The patch has no whitespace issues. |
   | +1 | shadedclient | 773 | patch has no errors when building and testing 
our client artifacts. |
   | -1 | javadoc | 20 | hadoop-hdds in the patch failed. |
   | -1 | javadoc | 17 | hadoop-ozone in the patch failed. |
   | -1 | findbugs | 32 | hadoop-hdds in the patch failed. |
   | -1 | findbugs | 18 | hadoop-ozone in the patch failed. |
   ||| _ Other Tests _ |
   | -1 | unit | 27 | hadoop-hdds in the patch failed. |
   | -1 | unit | 26 | hadoop-ozone in the patch failed. |
   | +1 | asflicense | 32 | The patch does not generate ASF License warnings. |
   | | | 3947 | |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.1 Server=19.03.1 base: 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/1541 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient findbugs checkstyle |
   | uname | Linux 13e1622b27d9 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | personality/hadoop.sh |
   | git revision | trunk / 13b427f |
   | Default Java | 1.8.0_222 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-mvninstall-hadoop-hdds.txt
 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-mvninstall-hadoop-ozone.txt
 |
   | compile | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-compile-hadoop-hdds.txt
 |
   | compile | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-compile-hadoop-ozone.txt
 |
   | javadoc | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-javadoc-hadoop-hdds.txt
 |
   | javadoc | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-javadoc-hadoop-ozone.txt
 |
   | findbugs | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-findbugs-hadoop-hdds.txt
 |
   | findbugs | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/branch-findbugs-hadoop-ozone.txt
 |
   | mvninstall | 
https://builds.apache.org/job/hadoop-multibranch/job/PR-1541/2/artifact/out/patch-mvninstall-hadoop-hdds.txt
 |
   | mvninstall | 

[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319783=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319783
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 20:04
Start Date: 27/Sep/19 20:04
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on pull request #1541: 
HDDS-1146. Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329231264
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
 ##
 @@ -179,4 +179,12 @@ ContainerInfo getMatchingContainer(long size, String 
owner,
*/
   ContainerInfo getMatchingContainer(long size, String owner,
   Pipeline pipeline, List excludedContainerIDS);
+
+  /**
+   * Once after report processor handler completes, call this to notify
+   * container manager to increment metrics.
+   * @param isFullReport
+   * @param success
+   */
+  void notifyContainerReportProcessing(boolean isFullReport, boolean success);
 
 Review comment:
   From proto, there is no enum. But i see we have TypedEvent
   TypedEvent
 INCREMENTAL_CONTAINER_REPORT = new TypedEvent<>(
 IncrementalContainerReportFromDatanode.class,
 "Incremental_Container_Report")
   TypedEvent CONTAINER_REPORT =
 new TypedEvent<>(ContainerReportFromDatanode.class, 
"Container_Report");
   
   Do you mean we want to use them instead of boolean?  
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319783)
Time Spent: 0.5h  (was: 20m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Supratim Deka
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319782=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319782
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 19:59
Start Date: 27/Sep/19 19:59
Worklog Time Spent: 10m 
  Work Description: anuengineer commented on pull request #1541: HDDS-1146. 
Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541#discussion_r329229253
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
 ##
 @@ -179,4 +179,12 @@ ContainerInfo getMatchingContainer(long size, String 
owner,
*/
   ContainerInfo getMatchingContainer(long size, String owner,
   Pipeline pipeline, List excludedContainerIDS);
+
+  /**
+   * Once after report processor handler completes, call this to notify
+   * container manager to increment metrics.
+   * @param isFullReport
+   * @param success
+   */
+  void notifyContainerReportProcessing(boolean isFullReport, boolean success);
 
 Review comment:
   I am not sure, But I think we have a Enum that indicates if the report is 
full or partial. If there is already one, then we might want to use it. 
Otherwise, I am okay with the boolean.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319782)
Time Spent: 20m  (was: 10m)

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Supratim Deka
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDDS-1146) Adding container related metrics in SCM

2019-09-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDDS-1146?focusedWorklogId=319775=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319775
 ]

ASF GitHub Bot logged work on HDDS-1146:


Author: ASF GitHub Bot
Created on: 27/Sep/19 19:53
Start Date: 27/Sep/19 19:53
Worklog Time Spent: 10m 
  Work Description: bharatviswa504 commented on pull request #1541: 
HDDS-1146. Adding container related metrics in SCM.
URL: https://github.com/apache/hadoop/pull/1541
 
 
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 319775)
Remaining Estimate: 0h
Time Spent: 10m

> Adding container related metrics in SCM
> ---
>
> Key: HDDS-1146
> URL: https://issues.apache.org/jira/browse/HDDS-1146
> Project: Hadoop Distributed Data Store
>  Issue Type: Improvement
>  Components: SCM
>Reporter: Nanda kumar
>Assignee: Supratim Deka
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-1146.000.patch, HDDS-1146.001.patch, 
> HDDS-1146.002.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This jira aims to add more container related metrics to SCM.
>  Following metrics will be added as part of this jira:
>  * Number of containers
>  * Number of open containers
>  * Number of closed containers
>  * Number of quasi closed containers
>  * Number of closing containers
> Above are already handled in HDDS-918.
>  * Number of successful create container calls
>  * Number of failed create container calls
>  * Number of successful delete container calls
>  * Number of failed delete container calls
> Handled in HDDS-2193.
>  * Number of successful container report processing
>  * Number of failed container report processing
>  * Number of successful incremental container report processing
>  * Number of failed incremental container report processing
> These will be handled in this jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org