Hi Craig Sounds ok to me
only a question : when is it retried ? immediately, or is the project placed back in the Q? if it is immediately, what would be the sleep/wait time before the retry occurs? I like the idea of having 1 exception build, even if the maxSourceControlRetry value is set to 5 that is indeed a very great improvement over the current approach, where every attempt would run the publisher block. with kind regards Ruben Willems On Wed, Mar 25, 2009 at 6:59 PM, Craig Sutherland <[email protected]>wrote: > > Hello all, > > I've been looking into this issue and seeing how we can improve > things, since it obviously is a contentious issue. > > First, what we were trying to achieve: previously errors with source > control were ignored or they would just crash the server. Obviously > this state was not particularly good. The changes now include source > control operations errors as build exceptions. Unfortunately, this now > appears to have gone the other way and made CC.Net too sensitive to > issues with source control. > > So what I am planning on doing is adding some tolerance to the > process. There is already a project setting called > maxSourceControlRetries, which currently sets the allowed number of > source control failures before stopping the project. > > What I am going to do is modify the source control checking, so if get > modifications fails, it only logs the error (similar to pre-1.4.3). > However, it will count the number of failures - if the number of > failures reaches the value in maxSourceControlRetries (by default this > is five), then it will log an error, change the build status to > exception and stop the project. If this state occurs, it will also run > the publishers to send out any e-mails, notifications, etc. > > This way people now have some control over the sensitivity of source > control checks. If you want to completely ignore this feature, set > maxSourceControlRetries to some really high number (it's an Int32 > though, so not too high), or if your source control/network/etc. is > really, really good and you don't except any problems, set it to a low > number (e.g. 0). > > What do people think about this approach? > > > Craig > > On Mar 26, 2:33 am, Jon W <[email protected]> wrote: > > On Tue, Mar 24, 2009 at 9:19 PM, si <[email protected]> wrote: > > > > >>> I now have 7 projects in "Exception" status (which are actually > > >>> running) with failed builds and none of them have actually failed, > > >>> they only failed because our server wasn't able to query svn because > > >>> another build appears to have saturated available resources on the > > >>> server.. > > > > >> ...and that's where our opinions differ: I see the situation as an > actual > > >> failure - if you aren't able to guarantee a successful build in a > timely > > >> fashion, it's a failure (along the lines of a typical "enterprise" > SLA)... > > >> just a question of filtering different failure notifications to > different > > >> people based on the cause. But then my opinion is just one of many, > and > > >> everyone is entitled to their own ;-) > > > > > Of course, and choice is good! > > > > > Anyway, I took Ruben's advice about setting up a single queue for all > > > our projects, and that should resolve the problem. > > > > > I'm still not completely convinced that the underlying cause (svn log > > > timeout) was due to the server or network, it may well be, but if more > > > reports start coming in then there might be some other issue at work. > > > > I agree with Si. I would like the svn timeouts to be logged just as > > they were pre-1.4.3, but I don't think the builds should enter the > > failed state due to a 'checking for modifications' action. An > > administrative notification can be sent, but that should only notify > > the administrator of the build system, and not the 100 developers who > > only need to focus on code modifications. > > > > Thanks, > > Jon >
