Thank Craig and Ruben for the prompt response, comments inline...

> maybe we should brainstorm a bit about this,
> It would be rather easy to set a property 'PublishExceptions' or something,
> which will ignore source control problems, but this is back again hiding the
> problem underneath.

Agreed, I knew our build server was underpowered, but didn't realise
we were timing out, here's the stack trace if it helps:

ThoughtWorks.CruiseControl.Core.CruiseControlException: Source control
operation failed: svn: Can't read from connection: An existing
connection was forcibly closed by the remote host.
. Process command: C:/Program Files/Subversion/svn.exe log svn://...
   at 
ThoughtWorks.CruiseControl.Core.Sourcecontrol.ProcessSourceControl.Execute(ProcessInfo
processInfo)
   at 
ThoughtWorks.CruiseControl.Core.Sourcecontrol.Svn.GetModifications(IIntegrationResult
from, IIntegrationResult to)
   at 
ThoughtWorks.CruiseControl.Core.Sourcecontrol.QuietPeriod.GetModifications(ISourceControl
sourceControl, IIntegrationResult lastBuild, IIntegrationResult
thisBuild)
   at 
ThoughtWorks.CruiseControl.Core.IntegrationRunner.GetModifications(IIntegrationResult
from, IIntegrationResult to)
   at 
ThoughtWorks.CruiseControl.Core.IntegrationRunner.Integrate(IntegrationRequest
request)

> Some possibilities I see :
> ° adjust publishers (mainly email) so it is easy to set which people get
> mailed on what state

That doesn't solve the problem where builds are failed and left with
an "Exception" status, even though the system is able to query source
control again.

> ° hiding the exception state in cctray : show the previous build state, so
>    programmers only see that the build is broken when the compile does not
> work/tests fail
>    and not because the network was down

As per above.

>    Dashboard may not hide it, because it is also used to see the logs.
> ° build in a retry amount (let's say 3 times after a sleep of 5 seonds) to
> bypass real temp errors
>     (but log them in the ccnet log). Mainly meant for the schedule trigger.

This sounds good, and it would be nice if the retry is configurable.
But once again, I don't think this exception should fail a build,
since none of the expected conditions for a build have actually been
triggered.

I've watched this process three times now, pretty much all of our
intervalTrigger build projects are now in Exception status and have
multiple failed builds as a result. They fail about 2 minutes after a
large build in another project is triggered. Timing seems to indicate
this is when Subversion has finished it's checkout and MSBuild has
started compiling. To make things more confusing, the projects in
Exception status show the exception error as well as the results from
the last successful build - nunit results, etc.

It's also a little odd though that "svn log" is timing out, as I would
have thought a network saturation issue rather than a CPU or RAM issue
would be the culprit, since svn is very light with respect to its
usage of these resources. So aside from the obvious problem, there may
be another issue in 1.4.3?

thanks again,
si

Reply via email to