Does this mean that everything is in for the 1.4.4 release now? Unless someone finds any issues with these sticky exception changes?
//P-J 2009/3/31 Craig Sutherland <[email protected]> > > Hello all, > > I've made the basic changes that I described earlier - by default a > failure during the get modifications stage will return the build to > it's previous state and not run any publishers (i.e. similar > functionality to pre-1.4.3). Once the maximum number of retries is > hit, it will change the status of the build to Exception, run the > publishers and stop the project (i.e the 1.4.3 functionality). > > We'll look at refining this functionality over time, but hopefully > this will provide the basics for most people. These changes can be > downloaded from http://ccnetlive.thoughtworks.com/CCNet-builds/1.4.3/ > (choose one of the latest builds). > > Let us know if you find any issues, > > > Craig > > On Mar 26, 9:43 pm, Ruben Willems <[email protected]> wrote: > > Hi all > > > > just thought of something : > > It seems that we are focussing mainly on interval triggers, (which is ok) > > > > suppose there is a source control error with a schedule trigger (nightly > > build), > > what would the wanted behaviour be? > > > > this has mainly to do how the retry will be implemented : > > ° wait 5 seconds and retry > > or > > ° place project back in the Q and wait for next integration > > > > with the interval trigger, both approaches are more or less the same > > I prefer to place it back in the Q (basically it means ignore source > control > > exceptions until MaxSourceControlExceptions is reached) > > this means that the waiting period is longer (trigger interval), 30 > seconds > > * 5 = 2.5 minutes that source control has to recuperate > > > > if you have wait 5 seconds before retry, and it means source control has > 5 * > > 5 = 25 seconds to recuperate > > > > but with a schedule trigger, the next integration round may be : tomorrow > > any thoughts on this? > > > > or should we add MaxSourceControlExceptions on the trigger level ? (and > when > > not set, take it from the project level) > > ( just brainstorming here ... ) > > > > with kind regards > > Ruben Willems > > > > On Thu, Mar 26, 2009 at 8:54 AM, Ruben Willems <[email protected] > >wrote: > > > > > Hi > > > > > That is even better, automatic recovery ! > > > +1 for me > > > > > with kind regards > > > Ruben Willems > > > > > On Thu, Mar 26, 2009 at 8:43 AM, Per-Jonny Käck <[email protected]> > wrote: > > > > >> I also really like the idea of having 1 exception build. > > > > >> But with Craigs proposed solution I'm not sure I see the point of > stopping > > >> the project. > > >> If the network/source control system comes up again it feels wasted > that > > >> the project has already been stopped. > > > > >> I think I would be happier with the following scenario. > > >> - if get modifications fails, it only logs the error (similar to > > >> pre-1.4.3). > > >> - it counts the number of successive failures > > >> - if the number of successive failures EQUALS the value in > > >> maxSourceControlRetries then it will log an error, change the build > status > > >> to exception and CONTINUE RUNNING the project. > > >> - if the above state occurs, it will also run the publishers to send > out > > >> any e-mails, notifications, etc. > > > > >> What happens when the project continue running after the failed build > due > > >> to maxSourceControlRetries? > > >> - if get modifications fails, it continue forever to log the error > > >> (similar to pre-1.4.3) > > >> - it counts the number of successive failures up to Int32.MaxValue > > >> - it should NOT reach maxSourceControlRetries again and not trigger > any > > >> new publishers. > > >> - as soon as the get modifications succeeds and a "real build" is > started > > >> the exception count is reset again > > > > >> Comments? > > > > >> //P-J > > > > >> 2009/3/26 Ruben Willems <[email protected]> > > > > >> Hi Craig > > > > >>> Sounds ok to me > > > > >>> only a question : > > >>> when is it retried ? > > >>> immediately, or is the project placed back in the Q? > > > > >>> if it is immediately, what would be the sleep/wait time before the > retry > > >>> occurs? > > > > >>> I like the idea of having 1 exception build, even if the > > >>> maxSourceControlRetry value is set to 5 > > >>> that is indeed a very great improvement over the current approach, > where > > >>> every attempt would run the publisher block. > > > > >>> with kind regards > > >>> Ruben Willems > > > > >>> On Wed, Mar 25, 2009 at 6:59 PM, Craig Sutherland < > > >>> [email protected]> wrote: > > > > >>>> Hello all, > > > > >>>> I've been looking into this issue and seeing how we can improve > > >>>> things, since it obviously is a contentious issue. > > > > >>>> First, what we were trying to achieve: previously errors with source > > >>>> control were ignored or they would just crash the server. Obviously > > >>>> this state was not particularly good. The changes now include source > > >>>> control operations errors as build exceptions. Unfortunately, this > now > > >>>> appears to have gone the other way and made CC.Net too sensitive to > > >>>> issues with source control. > > > > >>>> So what I am planning on doing is adding some tolerance to the > > >>>> process. There is already a project setting called > > >>>> maxSourceControlRetries, which currently sets the allowed number of > > >>>> source control failures before stopping the project. > > > > >>>> What I am going to do is modify the source control checking, so if > get > > >>>> modifications fails, it only logs the error (similar to pre-1.4.3). > > >>>> However, it will count the number of failures - if the number of > > >>>> failures reaches the value in maxSourceControlRetries (by default > this > > >>>> is five), then it will log an error, change the build status to > > >>>> exception and stop the project. If this state occurs, it will also > run > > >>>> the publishers to send out any e-mails, notifications, etc. > > > > >>>> This way people now have some control over the sensitivity of source > > >>>> control checks. If you want to completely ignore this feature, set > > >>>> maxSourceControlRetries to some really high number (it's an Int32 > > >>>> though, so not too high), or if your source control/network/etc. is > > >>>> really, really good and you don't except any problems, set it to a > low > > >>>> number (e.g. 0). > > > > >>>> What do people think about this approach? > > > > >>>> Craig > > > > >>>> On Mar 26, 2:33 am, Jon W <[email protected]> wrote: > > >>>> > On Tue, Mar 24, 2009 at 9:19 PM, si <[email protected]> wrote: > > > > >>>> > >>> I now have 7 projects in "Exception" status (which are > actually > > >>>> > >>> running) with failed builds and none of them have actually > failed, > > >>>> > >>> they only failed because our server wasn't able to query svn > > >>>> because > > >>>> > >>> another build appears to have saturated available resources on > the > > >>>> > >>> server.. > > > > >>>> > >> ...and that's where our opinions differ: I see the situation as > an > > >>>> actual > > >>>> > >> failure - if you aren't able to guarantee a successful build in > a > > >>>> timely > > >>>> > >> fashion, it's a failure (along the lines of a typical > "enterprise" > > >>>> SLA)... > > >>>> > >> just a question of filtering different failure notifications to > > >>>> different > > >>>> > >> people based on the cause. But then my opinion is just one of > many, > > >>>> and > > >>>> > >> everyone is entitled to their own ;-) > > > > >>>> > > Of course, and choice is good! > > > > >>>> > > Anyway, I took Ruben's advice about setting up a single queue > for > > >>>> all > > >>>> > > our projects, and that should resolve the problem. > > > > >>>> > > I'm still not completely convinced that the underlying cause > (svn > > >>>> log > > >>>> > > timeout) was due to the server or network, it may well be, but > if > > >>>> more > > >>>> > > reports start coming in then there might be some other issue at > > >>>> work. > > > > >>>> > I agree with Si. I would like the svn timeouts to be logged just > as > > >>>> > they were pre-1.4.3, but I don't think the builds should enter the > > >>>> > failed state due to a 'checking for modifications' action. An > > >>>> > administrative notification can be sent, but that should only > notify > > >>>> > the administrator of the build system, and not the 100 developers > who > > >>>> > only need to focus on code modifications. > > > > >>>> > Thanks, > > >>>> > Jon >
