[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-07 Thread jgl...@cloudbees.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Jesse Glick updated  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Jenkins /  JENKINS-49931  
 
 
  Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
Change By: 
 Jesse Glick  
 
 
Status: 
 In Review Resolved  
 
 
Resolution: 
 Fixed  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-06 Thread esc...@gmail.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Emilio Escobar  commented on  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
 new https://github.com/jenkinsci/support-core-plugin/pull/135   
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-06 Thread scm_issue_l...@java.net (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 SCM/JIRA link daemon commented on  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
 Code changed in jenkins User: Emilio Escobar Path: src/main/java/com/cloudbees/jenkins/support/impl/HeapUsageHistogram.java http://jenkins-ci.org/commit/support-core-plugin/a3ff0c7e985b0ecb9a4460bab94932bc55a17b6a Log: [FIXED JENKINS-49931] DISABLED by default  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-06 Thread esc...@gmail.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Emilio Escobar  updated  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Jenkins /  JENKINS-49931  
 
 
  Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
Change By: 
 Emilio Escobar   
 
 
Status: 
 In  Progress  Review  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-06 Thread esc...@gmail.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Emilio Escobar  commented on  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
 https://github.com/jenkinsci/support-core-plugin/pull/134 already available for review!  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-06 Thread esc...@gmail.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Emilio Escobar  started work on  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
Change By: 
 Emilio Escobar   
 
 
Status: 
 Open In Progress  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-05 Thread svano...@cloudbees.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Sam Van Oort commented on  JENKINS-49931  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
 Assigned to Emilio Escobar  because I think he has a fix already almost done for it.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[JIRA] (JENKINS-49931) Heap Histogram Collection Destabilizes Masters

2018-03-05 Thread svano...@cloudbees.com (JIRA)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Sam Van Oort created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Jenkins /  JENKINS-49931  
 
 
  Heap Histogram Collection Destabilizes Masters   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Assignee: 
 Emilio Escobar   
 
 
Components: 
 support-core-plugin  
 
 
Created: 
 2018-03-05 19:08  
 
 
Priority: 
  Critical  
 
 
Reporter: 
 Sam Van Oort  
 

  
 
 
 
 

 
 When we added Heap Stats collection (https://issues.jenkins-ci.org/browse/JENKINS-22791) in v2.42 it appears that we inadvertently caused a major performance-stability regression if the histogram is collected regularly. How? Well, this gathers a live heap histogram. This appears to triggers a Full GC. This is visible in GC logs because they show the following cause: > [Full GC (Heap Inspection Initiated GC).  Now, because this is a FullGC and not a concurrent or young-gen GC, and we're generally using G1 GC, the slow Serial garbage collector is used for FullGC. This is a NON-concurrent GC mode, meaning the application is fully paused until it completes, and it is SINGLE-threaded, meaning rather than 1 GB/s per CPU of GC throughput, we get <1 GB/s total. It also cleans and compacts the entire heap rather than just part of it as with other modes.  So, with 15 GB of used heap that means a pause of up ~15s. This matches behavior observed in the wild.  I am rating this as critical because on larger-scale production masters a hang that long can cause job failures, visible UI hangs, HTTP request timeouts, and other issues – it should result in Surable Task failures for Pipelines, for example. Proposed solution: only gather the live heap histogram when a user is explicitly requesting a support bundle (disable it by default).