Could this be a general issue?
The factor of 4 looks similar to what I see in hipp10 (Object Teams).
Some figures:
Two individual test suites A and B as part of the Object Teams build over time
(from some builds which I happened to keep):
up-to 2016-12-08:
suite A: around 30 min.
suite B: around 30 min.
2017-01-18:
suite A at 47 min.
suite B presumably timed out after 120 min (details no longer available)
2017-01-22:
suite A at 44 min.
suite B completed at 110 min
2017-01-27:
suite A at 60 min.
suite B presumably timed out after 120 min (details no longer available)
2017-02-04:
suite A timed out at 120 min
suite B died at 50 min (not sure if this could possibly be a time out)
Several intermediate builds timed out, providing no useful test results.
To check if this could be a regression in the software being tested, I
inspected logs from
local test runs over the given period, which, however, show no performance
regression.
When I looked up the various hipps on this hipp10, I couldn't see much info
about
build history, in fact only the platform hipp has data in the build history
view.
The same suite A is actually part of JDT/Core so we can do some comparison of
the
Object Teams HIPP with eclipse.jdt.core-Gerrit on Platform HIPP (same machine
hipp10!).
On the latter HIPP we observe alternating time outs but also test runs
completing
normally at the typical 30 min.
In this situation I have to run each build several times until I happen to get
complete test results. Of course these unnecessary test runs aggravate the
situation...
Short term work around: stop using hipp for testing, test only locally (saves
much time)
use hipp only for building / publishing - which will also take some load from
that machine.
Maybe we should all do this to retain good response times at least for the
actual
production builds?
best,
Stephan
On 02/02/2017 08:08 AM, Markus Knauer wrote:
I see similar problems on hipp8. Its reported load is between 1 and 2 at the
moment, sometimes less, i.e. not very high, but the EPP
package builds are extremely slow, much slower than usual. A few months ago in
October these builds used to take 1:20h [1], now we
are close to 6 hours [2] which is 4 times slower for the same build. In both
cases, builds were running on a day during Simultaneous
Release week which rules out this as a reason for the increased duration.
Maybe its worth to look at this host, too, as the increased build time doesn't
seem to be caused by a high load of the CPU.
And I highly appreciate your efforts to distribute CPU resources in a better
way!
Much thanks,
Markus
[1] https://hudson.eclipse.org/packaging/job/neon.epp-tycho-build/448/
[2] https://hudson.eclipse.org/packaging/job/oxygen.epp-tycho-build/183/
On 2 February 2017 at 01:22, Frederic Gurr <[email protected]
<mailto:[email protected]>> wrote:
Hi,
Unfortunately hipp9 has been experiencing a near-constant high load
during work hours. Due to a few projects hogging resources other
projects are severely constrained and have problems running their builds
in a timely manner.
To fix this, I have taken the following near term steps:
- Papyrus HIPP
- disabled execution of concurrent builds for "Papyrus-Gerrit-Oxygen"
- limited the number of executors to 2
- EMF Compare HIPP
- limited the number of executors to 2
I will also move the EMF Compare HIPP to a different hipp machine
tomorrow at 9am EST. Downtime will be less than one hour.
We will also investigate how CPU usage can be limited to ensure a fair
distribution of resources.
Regards,
Fred
_______________________________________________
cross-project-issues-dev mailing list
[email protected]
<mailto:[email protected]>
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
<https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
_______________________________________________
cross-project-issues-dev mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
