Hi Stefan, let's get constructive: what can we do about this? In general a second CPU sounds like a good idea, but under this circumstances it just doesn't make sense.
Franz -----Original Message----- From: Stefan Rossbach [mailto:srossb...@arcor.de] Sent: Dienstag, 18. September 2012 01:17 To: Zieris, Franz Cc: dpp-devel@lists.sourceforge.net Subject: Re: [Dpp-devel] Jenkins is acting ... strange Ok I read the JVM (Oracle Spec) JVM_DEFAULT_MAX_HEAP = Math.min(0.25 * totalMemory, 1 GB) What I do not understand, normally a JVM starts with approx 64 MB heap memory, increasing the heap as needed. Findbugs allocates much of memory. Since the Maxheap is set to 1GB it seems that the Garbage Collector (with a dual Core CPU this is normally an Concurrent Garbage Collector) most of the objects are put into the old generation although they should be garbaged. According to the top command, Findbugs starts with a working set of approx 100 MB and this working set is increased to almost 800 MB. And here comes the Unbuntu Memory Manager. It seems that it tries to hold as much swapable pages in real memory. You can reproduce this on Win32/64 also. Load as much applications to hit a 3/4 Memory usage. Now start an additional application that consumes about 3/4 of the available memory. Kernel Time and HDD usage will gone mad, stalling the system for severel seconds / minutes (depends on the speed of the underlaying HDD / SSD). Am 17.09.2012 17:41, schrieb Zieris, Franz: > OK, is this a pure internal issue, or are there any chances that the > hardware/external VM configuration is responsible for this? > > If the latter seems likely, could you briefly summarize your steps, so I can > file a ticket for our IT staff? > > -----Original Message----- > From: Stefan Rossbach [mailto:srossb...@arcor.de] > Sent: Montag, 17. September 2012 17:36 > To: Zieris, Franz > Cc: dpp-devel@lists.sourceforge.net > Subject: Re: [Dpp-devel] Jenkins is acting ... strange > > Just an update, > > swap space is working, but I have no clue why allocating 3 GB tooks about 5 > minutes. > > Am 17.09.2012 17:31, schrieb Stefan Rossbach: >> There seems to be a misconfiguration with the swap file. >> >> I am trying to allocate 3 GB in 1 MB chunks, swap memory is not used, >> >> CPU load = 100% (0% user, 100% kernel) >> >> >> Am 17.09.2012 17:11, schrieb Zieris, Franz: >>> OK, saros-build now running on 1 CPU. >>> >>>> I have to figure out how much heap memory is currently used by the JVM. >>> Please let me know, whether and how I may help. >>> >>> -----Original Message----- >>> From: Stefan Rossbach [mailto:srossb...@arcor.de] >>> Sent: Montag, 17. September 2012 17:02 >>> To: Zieris, Franz >>> Cc: dpp-devel@lists.sourceforge.net >>> Subject: Re: [Dpp-devel] Jenkins is acting ... strange >>> >>> It seems that Findbugs is causing that issue. I have to figure out how much >>> heap memory is currently used by the JVM. >>> >>> Am 17.09.2012 16:56, schrieb Zieris, Franz: >>>> I asked our IT staff to downgrade our VM to one CPU, in order to narrow >>>> down the possible sources of this behavior. >>>> >>>> I'll shutdown saros-build now. >>>> >>>> -----Original Message----- >>>> From: Stefan Rossbach [mailto:srossb...@arcor.de] >>>> Sent: Montag, 17. September 2012 16:42 >>>> To: Zieris, Franz >>>> Cc: dpp-devel@lists.sourceforge.net >>>> Subject: Re: [Dpp-devel] Jenkins is acting ... strange >>>> >>>> According to 'top' command: tomcat is using up to 200% cpu time (about >>>> 100% cpu time are used in the kernel / system). >>>> >>>> Am 17.09.2012 16:36, schrieb Zieris, Franz: >>>>> So, any ideas? >>>>> >>>>> -----Original Message----- >>>>> From: Stefan Rossbach [mailto:srossb...@arcor.de] >>>>> Sent: Montag, 17. September 2012 16:30 >>>>> To: Zieris, Franz >>>>> Cc: dpp-devel@lists.sourceforge.net >>>>> Subject: Re: [Dpp-devel] Jenkins is acting ... strange >>>>> >>>>> Mem: 4059844k total, 4020040k used, 39804k free, 197924k buffers >>>>> Swap: 2097144k total, 0k used, 2097144k free, 1754424k cached >>>>> >>>>> >>>>> WTF ?! >>>>> >>>>> Am 17.09.2012 16:25, schrieb Zieris, Franz: >>>>>> Dear Developers, >>>>>> >>>>>> I already read the tomcat logfiles, but didn't find anything promising. >>>>>> >>>>>> My current guess is, that there were some flawed build directories >>>>>> (build started, but did not finish) and the subsequent build tried to >>>>>> use the folder again. Currently I'm running a single job and put the >>>>>> Jenkins into "shutdown" mode, to prevent any further jobs from being >>>>>> started. We will see how far this idea gets. >>>>>> >>>>>> Franz >>>>>> >>>>>> -----Original Message----- >>>>>> From: >>>>>> se_p_agil-bounces~franz.zieris=fu-berlin...@lists.spline.inf.fu-b >>>>>> e >>>>>> rl >>>>>> i >>>>>> n >>>>>> .de >>>>>> [mailto:se_p_agil-bounces~franz.zieris=fu-berlin...@lists.spline.inf. >>>>>> f u-berlin.de] On Behalf Of Zieris, Franz >>>>>> Sent: Montag, 17. September 2012 11:44 >>>>>> To: dpp-devel@lists.sourceforge.net >>>>>> Cc: se_p_a...@lists.spline.inf.fu-berlin.de >>>>>> Subject: [SE_P_Agil] Jenkins is acting ... strange >>>>>> >>>>>> Dear Developers, >>>>>> >>>>>> Currently our Jenkins is not very responsive, the two CPUs are far below >>>>>> 10%, whereas Gerrit is working fast and fine. I already restarted >>>>>> tomcat6 (/etc/init.d/tomcat6 restart), but it didn't work out. Any ideas? >>>>>> >>>>>> Franz >>>>>> _______________________________________________ >>>>>> Mailingliste des Projekts "Agile Softwareenwicklung in einem >>>>>> Open-Source-Projekt" >>>>>> https://www.inf.fu-berlin.de/w/SE/SoftwaretechnikProjektAgil2012 >>>>>> >>>>>> Mails an: se_p_a...@lists.spline.inf.fu-berlin.de >>>>>> https://lists.spline.inf.fu-berlin.de/mailman/listinfo/se_p_agil >>>>>> >>>>>> ----------------------------------------------------------------- >>>>>> - >>>>>> -- >>>>>> - >>>>>> - >>>>>> -------- >>>>>> Live Security Virtual Conference >>>>>> Exclusive live event will cover all the ways today's security and >>>>>> threat landscape has changed and how IT managers can respond. >>>>>> Discussions will include endpoint security, mobile security and >>>>>> the latest in malware threats. >>>>>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>>>> _______________________________________________ >>>>>> Dpp-devel mailing list >>>>>> Dpp-devel@lists.sourceforge.net >>>>>> https://lists.sourceforge.net/lists/listinfo/dpp-devel >> --------------------------------------------------------------------- >> - >> -------- >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. >> Discussions will include endpoint security, mobile security and the >> latest in malware threats. >> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Dpp-devel mailing list >> Dpp-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dpp-devel ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Dpp-devel mailing list Dpp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dpp-devel
