On 10.12.15 11:02 , Tomek Rekawek wrote:
I’ve noticed that the failed builds can be assigned to a few
machines. For instance, builds on ubuntu-6, ubuntu-5, H11 and H10
fails almost every time. On the other hand, all builds on the
jenkins-ubuntu-1404-4gb succeedes.
Good find! Are these machines slower in general (i.e. for all builds) or
only for the DocumentRDB builds? In the former case it might make sense
to just increase the time out.
For the further diagnostics it could be useful to get the
unittest.log file from the failed build. Right now, the only way to
do it is to wait until some build fails and grab the file from the
workspace. The workspace is purged every time the jobs starts, so it
requires a good timing. Maybe we can change the configuration, so the
target/unittest.log file will be archived if the job is failed?
This would be useful anyway but I think so far no one has figured out
how to do it. Unfortunately I can only grant committers access to that
Jenkins. Maybe Tommaso could help at this point as he was involved in
setting that instance up.
Michael
Best regards, Tomek
On 09/12/15 17:20, "Michael Dürig" <mdue...@apache.org> wrote:
On 9.12.15 12:02 , Michael Dürig wrote:
Do you have any suggestions?
Something must be way wrong as the build either takes ~30 minutes
or it times out after 90 minutes. There is quite a difference
here. Maybe we need to go through the log files and compare test
execution times of individual tests between timed out and
succeeded builds to get a clue.
Comparing test execution times for builds 593 and 595. The former
timed out and the latter passed. Both ran on Java 1.8.
Running org.apache.jackrabbit.oak.jcr.ValueFactoryTest Tests run:
2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 66.774 sec
vs.
Running org.apache.jackrabbit.oak.jcr.MoveRemoveTest Tests run: 12,
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.893 sec
Running org.apache.jackrabbit.oak.jcr.query.QueryTest Tests run:
27, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 541.876 sec
vs.
Running org.apache.jackrabbit.oak.jcr.query.QueryTest Tests run:
27, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 48.32 sec
So it seems that for some reason sometimes tests run 3 - 5 times
slower than normal, which with a usual build time of ~30 minutes
bumps us over the 90 minutes build time out.
Michael
See:
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/595/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/consoleFull
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/593/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
