Hi That is even better, automatic recovery ! +1 for me
with kind regards Ruben Willems On Thu, Mar 26, 2009 at 8:43 AM, Per-Jonny Käck <[email protected]> wrote: > I also really like the idea of having 1 exception build. > > But with Craigs proposed solution I'm not sure I see the point of stopping > the project. > If the network/source control system comes up again it feels wasted that > the project has already been stopped. > > I think I would be happier with the following scenario. > - if get modifications fails, it only logs the error (similar to > pre-1.4.3). > - it counts the number of successive failures > - if the number of successive failures EQUALS the value in > maxSourceControlRetries then it will log an error, change the build status > to exception and CONTINUE RUNNING the project. > - if the above state occurs, it will also run the publishers to send out > any e-mails, notifications, etc. > > What happens when the project continue running after the failed build due > to maxSourceControlRetries? > - if get modifications fails, it continue forever to log the error (similar > to pre-1.4.3) > - it counts the number of successive failures up to Int32.MaxValue > - it should NOT reach maxSourceControlRetries again and not trigger any new > publishers. > - as soon as the get modifications succeeds and a "real build" is started > the exception count is reset again > > Comments? > > //P-J > > 2009/3/26 Ruben Willems <[email protected]> > > Hi Craig >> >> Sounds ok to me >> >> only a question : >> when is it retried ? >> immediately, or is the project placed back in the Q? >> >> if it is immediately, what would be the sleep/wait time before the retry >> occurs? >> >> I like the idea of having 1 exception build, even if the >> maxSourceControlRetry value is set to 5 >> that is indeed a very great improvement over the current approach, where >> every attempt would run the publisher block. >> >> >> with kind regards >> Ruben Willems >> >> >> >> On Wed, Mar 25, 2009 at 6:59 PM, Craig Sutherland < >> [email protected]> wrote: >> >>> >>> Hello all, >>> >>> I've been looking into this issue and seeing how we can improve >>> things, since it obviously is a contentious issue. >>> >>> First, what we were trying to achieve: previously errors with source >>> control were ignored or they would just crash the server. Obviously >>> this state was not particularly good. The changes now include source >>> control operations errors as build exceptions. Unfortunately, this now >>> appears to have gone the other way and made CC.Net too sensitive to >>> issues with source control. >>> >>> So what I am planning on doing is adding some tolerance to the >>> process. There is already a project setting called >>> maxSourceControlRetries, which currently sets the allowed number of >>> source control failures before stopping the project. >>> >>> What I am going to do is modify the source control checking, so if get >>> modifications fails, it only logs the error (similar to pre-1.4.3). >>> However, it will count the number of failures - if the number of >>> failures reaches the value in maxSourceControlRetries (by default this >>> is five), then it will log an error, change the build status to >>> exception and stop the project. If this state occurs, it will also run >>> the publishers to send out any e-mails, notifications, etc. >>> >>> This way people now have some control over the sensitivity of source >>> control checks. If you want to completely ignore this feature, set >>> maxSourceControlRetries to some really high number (it's an Int32 >>> though, so not too high), or if your source control/network/etc. is >>> really, really good and you don't except any problems, set it to a low >>> number (e.g. 0). >>> >>> What do people think about this approach? >>> >>> >>> Craig >>> >>> On Mar 26, 2:33 am, Jon W <[email protected]> wrote: >>> > On Tue, Mar 24, 2009 at 9:19 PM, si <[email protected]> wrote: >>> > >>> > >>> I now have 7 projects in "Exception" status (which are actually >>> > >>> running) with failed builds and none of them have actually failed, >>> > >>> they only failed because our server wasn't able to query svn >>> because >>> > >>> another build appears to have saturated available resources on the >>> > >>> server.. >>> > >>> > >> ...and that's where our opinions differ: I see the situation as an >>> actual >>> > >> failure - if you aren't able to guarantee a successful build in a >>> timely >>> > >> fashion, it's a failure (along the lines of a typical "enterprise" >>> SLA)... >>> > >> just a question of filtering different failure notifications to >>> different >>> > >> people based on the cause. But then my opinion is just one of many, >>> and >>> > >> everyone is entitled to their own ;-) >>> > >>> > > Of course, and choice is good! >>> > >>> > > Anyway, I took Ruben's advice about setting up a single queue for all >>> > > our projects, and that should resolve the problem. >>> > >>> > > I'm still not completely convinced that the underlying cause (svn log >>> > > timeout) was due to the server or network, it may well be, but if >>> more >>> > > reports start coming in then there might be some other issue at work. >>> > >>> > I agree with Si. I would like the svn timeouts to be logged just as >>> > they were pre-1.4.3, but I don't think the builds should enter the >>> > failed state due to a 'checking for modifications' action. An >>> > administrative notification can be sent, but that should only notify >>> > the administrator of the build system, and not the 100 developers who >>> > only need to focus on code modifications. >>> > >>> > Thanks, >>> > Jon >>> >> >> >
