Damn good catch, Peter.
Christian.
On 5-Sep-08, at 11:49 , Peter Janes wrote:
After having this happen a third time I took a closer look at the
database and saw that one of the entries in the BUILDRESULT table
had an END_TIME of 0. When I changed that field to a current time
and restarted the server Continuum started to work again.
I was able to reproduce the problem by setting the END_TIME of the
last BUILDRESULT to 0, starting the server, and waiting for that
project to be queued. It was resolved again when I reset END_TIME
to its original value.
I've reported this as http://jira.codehaus.org/browse/CONTINUUM-1871
and attached the thread dump.
Thanks,
Peter J.
Brett Porter wrote:
I certainly haven't seen something like this - perhaps something in
the
project change caused an infinite loop that might be a bug in the
Maven
project handling.
Are you able to get a thread dump from Continuum when it occurs?
Thanks,
Brett
2008/9/3 Peter Janes <[EMAIL PROTECTED]>
Since adding a set of projects to a Continuum 1.2 instance last
Thursday,
the server has stopped performing any builds on any projects. The
server
had performed hundreds of builds without issue over a period of
several
months.
The initial problem that appeared was that projects would be
queued but
never started; as soon as the first job entered the queue
Continuum would
start using 100% CPU and never start building anything. Even when
I removed
all of the projects from the queue it remained at 100% CPU.
Restarting the
server didn't help: as soon as a job was queued the CPU would
spike and no
further progress would be made.
Thinking that somehow the (MySQL) database might be corrupt, I
removed it
and started fresh. I added the projects that appeared to cause
the problem
and they built successfully, so I started re-adding the rest of the
projects. Unfortunately, at some point while I was doing this
builds
stopped being executed again: this time they get queued up and are
apparently started, but I don't see any build processes being
invoked. Even
the simplest pom-only job times out. The timeouts are shown in the
Continuum log but aren't being recorded in the database as
failures or
errors, and restarting doesn't make a difference, which suggests
to me that
perhaps the database has been corrupted again.
I don't see anything in Jira or the list archives; has anyone seen
behaviours like this? More importantly, has anyone been able to
solve them?
Environment: RHEL 5, Continuum 1.2 (built in May), Java 1.6.0_03,
MySQL
5.0.45, Maven 2.0.9
Peter J.
--
Sometimes the Universe needs a change of perspective.
--J. Michael Straczynski
--
Sometimes the Universe needs a change of perspective.
--J. Michael Straczynski
