Hi all problem with ccnet live has been solved, so the latest builds allow you to configure the mentioned settings
remarks from testers are welcome ... with kind regards Ruben Willems On Tue, Mar 31, 2009 at 12:45 PM, Ruben Willems <[email protected]>wrote: > Hi > > > it is comitted to the trunk, but ccnet live has a problem which I can not > reproduce locally > it should work, so if someone will try it out > > > > with kind regards > Ruben Willems > > On Tue, Mar 31, 2009 at 10:49 AM, Ruben Willems > <[email protected]>wrote: > >> Hi all >> >> I created an issue in Jira >> http://jira.public.thoughtworks.org/browse/CCNET-1441 >> >> working on it now ... >> >> >> with kind regards >> Ruben Willems >> >> >> On Tue, Mar 31, 2009 at 8:37 AM, Ruben Willems >> <[email protected]>wrote: >> >>> Hi >>> >>> no, Craig did a change he mentioned before, this does not inlcude my >>> proposed changes yet >>> I hope to do that one of these days. >>> >>> >>> with kind regards >>> Ruben Willems >>> >>> On Tue, Mar 31, 2009 at 8:31 AM, Per-Jonny Käck <[email protected]>wrote: >>> >>>> Does this mean that everything is in for the 1.4.4 release now? >>>> Unless someone finds any issues with these sticky exception changes? >>>> >>>> //P-J >>>> >>>> 2009/3/31 Craig Sutherland <[email protected]> >>>> >>>> >>>>> Hello all, >>>>> >>>>> I've made the basic changes that I described earlier - by default a >>>>> failure during the get modifications stage will return the build to >>>>> it's previous state and not run any publishers (i.e. similar >>>>> functionality to pre-1.4.3). Once the maximum number of retries is >>>>> hit, it will change the status of the build to Exception, run the >>>>> publishers and stop the project (i.e the 1.4.3 functionality). >>>>> >>>>> We'll look at refining this functionality over time, but hopefully >>>>> this will provide the basics for most people. These changes can be >>>>> downloaded from http://ccnetlive.thoughtworks.com/CCNet-builds/1.4.3/ >>>>> (choose one of the latest builds). >>>>> >>>>> Let us know if you find any issues, >>>>> >>>>> >>>>> Craig >>>>> >>>>> On Mar 26, 9:43 pm, Ruben Willems <[email protected]> wrote: >>>>> > Hi all >>>>> > >>>>> > just thought of something : >>>>> > It seems that we are focussing mainly on interval triggers, (which is >>>>> ok) >>>>> > >>>>> > suppose there is a source control error with a schedule trigger >>>>> (nightly >>>>> > build), >>>>> > what would the wanted behaviour be? >>>>> > >>>>> > this has mainly to do how the retry will be implemented : >>>>> > ° wait 5 seconds and retry >>>>> > or >>>>> > ° place project back in the Q and wait for next integration >>>>> > >>>>> > with the interval trigger, both approaches are more or less the same >>>>> > I prefer to place it back in the Q (basically it means ignore source >>>>> control >>>>> > exceptions until MaxSourceControlExceptions is reached) >>>>> > this means that the waiting period is longer (trigger interval), 30 >>>>> seconds >>>>> > * 5 = 2.5 minutes that source control has to recuperate >>>>> > >>>>> > if you have wait 5 seconds before retry, and it means source control >>>>> has 5 * >>>>> > 5 = 25 seconds to recuperate >>>>> > >>>>> > but with a schedule trigger, the next integration round may be : >>>>> tomorrow >>>>> > any thoughts on this? >>>>> > >>>>> > or should we add MaxSourceControlExceptions on the trigger level ? >>>>> (and when >>>>> > not set, take it from the project level) >>>>> > ( just brainstorming here ... ) >>>>> > >>>>> > with kind regards >>>>> > Ruben Willems >>>>> > >>>>> > On Thu, Mar 26, 2009 at 8:54 AM, Ruben Willems < >>>>> [email protected]>wrote: >>>>> > >>>>> > > Hi >>>>> > >>>>> > > That is even better, automatic recovery ! >>>>> > > +1 for me >>>>> > >>>>> > > with kind regards >>>>> > > Ruben Willems >>>>> > >>>>> > > On Thu, Mar 26, 2009 at 8:43 AM, Per-Jonny Käck <[email protected]> >>>>> wrote: >>>>> > >>>>> > >> I also really like the idea of having 1 exception build. >>>>> > >>>>> > >> But with Craigs proposed solution I'm not sure I see the point of >>>>> stopping >>>>> > >> the project. >>>>> > >> If the network/source control system comes up again it feels >>>>> wasted that >>>>> > >> the project has already been stopped. >>>>> > >>>>> > >> I think I would be happier with the following scenario. >>>>> > >> - if get modifications fails, it only logs the error (similar to >>>>> > >> pre-1.4.3). >>>>> > >> - it counts the number of successive failures >>>>> > >> - if the number of successive failures EQUALS the value in >>>>> > >> maxSourceControlRetries then it will log an error, change the >>>>> build status >>>>> > >> to exception and CONTINUE RUNNING the project. >>>>> > >> - if the above state occurs, it will also run the publishers to >>>>> send out >>>>> > >> any e-mails, notifications, etc. >>>>> > >>>>> > >> What happens when the project continue running after the failed >>>>> build due >>>>> > >> to maxSourceControlRetries? >>>>> > >> - if get modifications fails, it continue forever to log the error >>>>> > >> (similar to pre-1.4.3) >>>>> > >> - it counts the number of successive failures up to Int32.MaxValue >>>>> > >> - it should NOT reach maxSourceControlRetries again and not >>>>> trigger any >>>>> > >> new publishers. >>>>> > >> - as soon as the get modifications succeeds and a "real build" is >>>>> started >>>>> > >> the exception count is reset again >>>>> > >>>>> > >> Comments? >>>>> > >>>>> > >> //P-J >>>>> > >>>>> > >> 2009/3/26 Ruben Willems <[email protected]> >>>>> > >>>>> > >> Hi Craig >>>>> > >>>>> > >>> Sounds ok to me >>>>> > >>>>> > >>> only a question : >>>>> > >>> when is it retried ? >>>>> > >>> immediately, or is the project placed back in the Q? >>>>> > >>>>> > >>> if it is immediately, what would be the sleep/wait time before >>>>> the retry >>>>> > >>> occurs? >>>>> > >>>>> > >>> I like the idea of having 1 exception build, even if the >>>>> > >>> maxSourceControlRetry value is set to 5 >>>>> > >>> that is indeed a very great improvement over the current >>>>> approach, where >>>>> > >>> every attempt would run the publisher block. >>>>> > >>>>> > >>> with kind regards >>>>> > >>> Ruben Willems >>>>> > >>>>> > >>> On Wed, Mar 25, 2009 at 6:59 PM, Craig Sutherland < >>>>> > >>> [email protected]> wrote: >>>>> > >>>>> > >>>> Hello all, >>>>> > >>>>> > >>>> I've been looking into this issue and seeing how we can improve >>>>> > >>>> things, since it obviously is a contentious issue. >>>>> > >>>>> > >>>> First, what we were trying to achieve: previously errors with >>>>> source >>>>> > >>>> control were ignored or they would just crash the server. >>>>> Obviously >>>>> > >>>> this state was not particularly good. The changes now include >>>>> source >>>>> > >>>> control operations errors as build exceptions. Unfortunately, >>>>> this now >>>>> > >>>> appears to have gone the other way and made CC.Net too sensitive >>>>> to >>>>> > >>>> issues with source control. >>>>> > >>>>> > >>>> So what I am planning on doing is adding some tolerance to the >>>>> > >>>> process. There is already a project setting called >>>>> > >>>> maxSourceControlRetries, which currently sets the allowed number >>>>> of >>>>> > >>>> source control failures before stopping the project. >>>>> > >>>>> > >>>> What I am going to do is modify the source control checking, so >>>>> if get >>>>> > >>>> modifications fails, it only logs the error (similar to >>>>> pre-1.4.3). >>>>> > >>>> However, it will count the number of failures - if the number of >>>>> > >>>> failures reaches the value in maxSourceControlRetries (by >>>>> default this >>>>> > >>>> is five), then it will log an error, change the build status to >>>>> > >>>> exception and stop the project. If this state occurs, it will >>>>> also run >>>>> > >>>> the publishers to send out any e-mails, notifications, etc. >>>>> > >>>>> > >>>> This way people now have some control over the sensitivity of >>>>> source >>>>> > >>>> control checks. If you want to completely ignore this feature, >>>>> set >>>>> > >>>> maxSourceControlRetries to some really high number (it's an >>>>> Int32 >>>>> > >>>> though, so not too high), or if your source control/network/etc. >>>>> is >>>>> > >>>> really, really good and you don't except any problems, set it to >>>>> a low >>>>> > >>>> number (e.g. 0). >>>>> > >>>>> > >>>> What do people think about this approach? >>>>> > >>>>> > >>>> Craig >>>>> > >>>>> > >>>> On Mar 26, 2:33 am, Jon W <[email protected]> wrote: >>>>> > >>>> > On Tue, Mar 24, 2009 at 9:19 PM, si <[email protected]> >>>>> wrote: >>>>> > >>>>> > >>>> > >>> I now have 7 projects in "Exception" status (which are >>>>> actually >>>>> > >>>> > >>> running) with failed builds and none of them have actually >>>>> failed, >>>>> > >>>> > >>> they only failed because our server wasn't able to query >>>>> svn >>>>> > >>>> because >>>>> > >>>> > >>> another build appears to have saturated available >>>>> resources on the >>>>> > >>>> > >>> server.. >>>>> > >>>>> > >>>> > >> ...and that's where our opinions differ: I see the >>>>> situation as an >>>>> > >>>> actual >>>>> > >>>> > >> failure - if you aren't able to guarantee a successful >>>>> build in a >>>>> > >>>> timely >>>>> > >>>> > >> fashion, it's a failure (along the lines of a typical >>>>> "enterprise" >>>>> > >>>> SLA)... >>>>> > >>>> > >> just a question of filtering different failure >>>>> notifications to >>>>> > >>>> different >>>>> > >>>> > >> people based on the cause. But then my opinion is just one >>>>> of many, >>>>> > >>>> and >>>>> > >>>> > >> everyone is entitled to their own ;-) >>>>> > >>>>> > >>>> > > Of course, and choice is good! >>>>> > >>>>> > >>>> > > Anyway, I took Ruben's advice about setting up a single >>>>> queue for >>>>> > >>>> all >>>>> > >>>> > > our projects, and that should resolve the problem. >>>>> > >>>>> > >>>> > > I'm still not completely convinced that the underlying cause >>>>> (svn >>>>> > >>>> log >>>>> > >>>> > > timeout) was due to the server or network, it may well be, >>>>> but if >>>>> > >>>> more >>>>> > >>>> > > reports start coming in then there might be some other issue >>>>> at >>>>> > >>>> work. >>>>> > >>>>> > >>>> > I agree with Si. I would like the svn timeouts to be logged >>>>> just as >>>>> > >>>> > they were pre-1.4.3, but I don't think the builds should enter >>>>> the >>>>> > >>>> > failed state due to a 'checking for modifications' action. An >>>>> > >>>> > administrative notification can be sent, but that should only >>>>> notify >>>>> > >>>> > the administrator of the build system, and not the 100 >>>>> developers who >>>>> > >>>> > only need to focus on code modifications. >>>>> > >>>>> > >>>> > Thanks, >>>>> > >>>> > Jon >>>>> >>>> >>>> >>> >> >
