Wyatt, I see the new options on TC, thanks for that. I still haven't thought about how I will go about capturing the failures exactly, but will give you a shout if I need some help with TC configuration just for those runs.
If you can reproduce any of those test failures locally, do you mind running them in VERBOSE mode (debug build without any other changes will do), and emailing the console output that you get? I might be too optimistic, but perhaps something there will stand out. Thanks again! Laimis On Sun, May 17, 2015 at 8:18 AM, Wyatt Barnett <[email protected]> wrote: > For TestSort_2 -- It appears to be passing based on data at > > http://teamcity.codebetter.com/project.html?projectId=LuceneNet&testNameId=-8365680837810961892&tab=testDetails > ; I am having locally reproducable problems on the others though. > > On Sun, May 17, 2015 at 7:55 AM, Wyatt Barnett <[email protected]> > wrote: > > > Done -- you should now see a run button when you visit > > http://teamcity.codebetter.com/project.html?projectId=LuceneNet > > > > On Sat, May 16, 2015 at 8:02 PM, Laimonas Simutis <[email protected]> > > wrote: > > > >> Wyatt, > >> > >> Could you add me to the lucene.net group on TC? I have a login there, > >> username: laimis. > >> > >> > >> Thanks! > >> > >> On Sat, May 16, 2015 at 6:15 PM, Wyatt Barnett <[email protected] > > > >> wrote: > >> > >> > Sounds good Laimis. You will need to setup a login to the CodeBetter > >> > teamcity server and get added to the lucene.net group if you haven't > >> > already. Let me know if you need help there too. > >> > > >> > On Sat, May 16, 2015 at 4:52 PM Laimonas Simutis <[email protected]> > >> wrote: > >> > > >> > > Wyatt, > >> > > > >> > > Sweet, I will let you know once I have a branch out with additional > >> > logging > >> > > and separate category for tests that you can configure to run. > >> > > > >> > > Re: release mode, tried that and was able to fix a few bugs after > >> > switching > >> > > to it. They were in that PR with debug.assert changes. Who knows, > the > >> > > remaining failures might still be related to that, but can't > >> reproduce it > >> > > locally. > >> > > > >> > > Laimis > >> > > On May 16, 2015 4:34 PM, "Wyatt Barnett" <[email protected]> > >> > wrote: > >> > > > >> > > > Sorry about the blank one -- getting used to google inbox here > and I > >> > > > misclicked. > >> > > > > >> > > > Anyhow, I have a repro or at least a rhyme and reason -- TeamCity > is > >> > > > running in release mode and I think we have difffering behavior > >> there. > >> > If > >> > > > you switch your copy of visual studio to release mode you will get > >> the > >> > > same > >> > > > failures we are seeing in TeamCity. Does that help narrow it down > a > >> > bit? > >> > > > > >> > > > On Sat, May 16, 2015 at 4:26 PM Wyatt Barnett < > >> [email protected] > >> > > > >> > > > wrote: > >> > > > > >> > > > > > >> > > > > > >> > > > > On Sat, May 16, 2015 at 3:22 PM Wyatt Barnett < > >> > [email protected] > >> > > > > >> > > > > wrote: > >> > > > > > >> > > > >> I agree with Itamar -- it feels environmental. I'll do some > >> digging > >> > > into > >> > > > >> the teamcity output but I think the plan of setting up some > extra > >> > > > verbose > >> > > > >> logging here would make sense. I can set you up with a separate > >> > build > >> > > > >> pointed at your fork if that helps -- it will keep the feedback > >> > cycle > >> > > > >> tighter. The other thing we could do is categorize the tests > and > >> > focus > >> > > > that > >> > > > >> build at running only that category so you don't need to wait > on > >> the > >> > > > whole > >> > > > >> suite to get responses. Let me know if you want me to proceed > >> there. > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> On Sat, May 16, 2015 at 3:10 PM, Itamar Syn-Hershko < > >> > > [email protected] > >> > > > > > >> > > > >> wrote: > >> > > > >> > >> > > > >>> Yes, that would be the best way to do this. On Java Lucene, > the > >> > > > >>> randomized > >> > > > >>> tests framework allows you to re-use the random seed > associated > >> > with > >> > > > the > >> > > > >>> failure, but we are not there yet. Either way, I suspect this > >> to be > >> > > an > >> > > > >>> environment issue rather than a code path one. > >> > > > >>> > >> > > > >>> -- > >> > > > >>> > >> > > > >>> Itamar Syn-Hershko > >> > > > >>> http://code972.com | @synhershko < > >> https://twitter.com/synhershko> > >> > > > >>> Freelance Developer & Consultant > >> > > > >>> Lucene.NET committer and PMC member > >> > > > >>> > >> > > > >>> On Sat, May 16, 2015 at 10:06 PM, Laimonas Simutis < > >> > [email protected] > >> > > > > >> > > > >>> wrote: > >> > > > >>> > >> > > > >>> > There are three tests that consistently fail on TC but no > >> matter > >> > > how > >> > > > >>> many > >> > > > >>> > times I try, I can't reproduce it locally. These tests are: > >> > > > >>> > > >> > > > >>> > TestFuzzyQuery.TestTieBreaker > >> > > > >>> > > >> > > > >>> > > >> > > > >>> > >> > > > > >> > > > >> > > >> > http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId-6371662534320583798 > >> > > > >>> > > >> > > > >>> > TestSimpleExplanations.TestDMQ8 > >> > > > >>> > > >> > > > >>> > > >> > > > >>> > >> > > > > >> > > > >> > > >> > http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId5725706748293106127 > >> > > > >>> > > >> > > > >>> > TestTopDocsMerge.TestSort_2 > >> > > > >>> > > >> > > > >>> > > >> > > > >>> > >> > > > > >> > > > >> > > >> > http://teamcity.codebetter.com/viewLog.html?buildId=191298&tab=buildResultsDiv&buildTypeId=LuceneNet_Core#testNameId-8365680837810961892 > >> > > > >>> > > >> > > > >>> > I would fix them if I could reproduce it -- and I am running > >> out > >> > of > >> > > > >>> ideas > >> > > > >>> > how to do it. Even if I put them in a loop running hundreds > of > >> > > > times, I > >> > > > >>> > can't trigger the failure. > >> > > > >>> > > >> > > > >>> > Anyone have any ideas how to go about reproducing it? I am > >> > thinking > >> > > > to > >> > > > >>> push > >> > > > >>> > very verbose code in a separate branch that logs the input > >> > values / > >> > > > >>> random > >> > > > >>> > values that are used and see what happens. Checking if > anyone > >> has > >> > > any > >> > > > >>> other > >> > > > >>> > suggestions. > >> > > > >>> > > >> > > > >>> > > >> > > > >>> > Thanks, > >> > > > >>> > > >> > > > >>> > Laimis > >> > > > >>> > > >> > > > >>> > >> > > > >> > >> > > > >> > >> > > > > >> > > > >> > > >> > > > > >
