So, after fighting a number of teething problems -- many of them self inflicted -- I've got a solid, repeatable test run that finishes in 38 minutes or so on our hardware.
I can certainly reprise this setup over on teamcity.codebetter.com without much effort. We will need to merge some changes into things before proceeding -- the current state of tests in the trunk will just hang if we tried to run them there. I'm guessing I'll need to sign a contributing agreement here as I don't believe I filed one. On Tue, Nov 18, 2014 at 5:19 PM, Wyatt Barnett <[email protected]> wrote: > Sounds good, we'll be [Explicit()] with reason. > > On Tue, Nov 18, 2014 at 5:06 PM, Itamar Syn-Hershko <[email protected]> > wrote: > >> Yes, but Explicit() will help us isolate those issues from other real bugs >> we can concentrate on solving in parallel. >> >> -- >> >> Itamar Syn-Hershko >> http://code972.com | @synhershko <https://twitter.com/synhershko> >> Freelance Developer & Consultant >> Author of RavenDB in Action <http://manning.com/synhershko/> >> >> On Wed, Nov 19, 2014 at 12:01 AM, Wyatt Barnett <[email protected]> >> wrote: >> >> > [Ignore()] is good too, I actually prefer a slightly nuanced version >> called >> > [Explicit()] as that lets you still fire off the test from resharper or >> > nunit gui. >> > >> > That said I proposed Assert.Fail() here because we can put the failure >> > point at the *exact* point where the folks should start debugging this >> from >> > versus having them start a at a whole test or test fixture. >> > >> > On Tue, Nov 18, 2014 at 4:56 PM, Itamar Syn-Hershko <[email protected] >> > >> > wrote: >> > >> > > Don't forget to stick a reason to the Ignore property tho! >> > > >> > > -- >> > > >> > > Itamar Syn-Hershko >> > > http://code972.com | @synhershko <https://twitter.com/synhershko> >> > > Freelance Developer & Consultant >> > > Author of RavenDB in Action <http://manning.com/synhershko/> >> > > >> > > On Tue, Nov 18, 2014 at 11:55 PM, Itamar Syn-Hershko < >> [email protected] >> > > >> > > wrote: >> > > >> > > > Or just Skip (or Ignore, however that's called in NUnit) :) >> > > > >> > > > -- >> > > > >> > > > Itamar Syn-Hershko >> > > > http://code972.com | @synhershko <https://twitter.com/synhershko> >> > > > Freelance Developer & Consultant >> > > > Author of RavenDB in Action <http://manning.com/synhershko/> >> > > > >> > > > On Tue, Nov 18, 2014 at 11:47 PM, Wyatt Barnett < >> > [email protected] >> > > > >> > > > wrote: >> > > > >> > > >> 5 minutes sounds like a reasonable going in proposition. That said >> > > >> depending on how many of these there are there might well be a >> *lot* >> > of >> > > 5 >> > > >> minute waits. >> > > >> >> > > >> Would it help if I just stuck Assert.Fails() where I run into these >> > > loops >> > > >> for the folks smarter than I to run down? >> > > >> >> > > >> On Tue, Nov 18, 2014 at 4:37 PM, Itamar Syn-Hershko < >> > [email protected] >> > > > >> > > >> wrote: >> > > >> >> > > >> > 5 mins max for one test maybe? >> > > >> > >> > > >> > To fix that infinite loop we need to revisit the original Java >> code, >> > > >> most >> > > >> > likely its porting of an iterator-style code that went wrong. I >> hope >> > > to >> > > >> > have time to look at it next week, please anyone else who feels >> like >> > > it >> > > >> > beat me to it.. >> > > >> > >> > > >> > -- >> > > >> > >> > > >> > Itamar Syn-Hershko >> > > >> > http://code972.com | @synhershko <https://twitter.com/synhershko >> > >> > > >> > Freelance Developer & Consultant >> > > >> > Author of RavenDB in Action <http://manning.com/synhershko/> >> > > >> > >> > > >> > On Tue, Nov 18, 2014 at 11:13 PM, Wyatt Barnett < >> > > >> [email protected]> >> > > >> > wrote: >> > > >> > >> > > >> > > So, after the 2nd day of watching the tests just spin I decided >> > > that a >> > > >> > > little visibility might make sense as something just was not >> > adding >> > > up >> > > >> > -- I >> > > >> > > expected some failing tests, and some long-running tests but >> there >> > > >> just >> > > >> > > ain't enough data here to keep something running for 8+ hours. >> > > >> > > >> > > >> > > Anyhow, I stood up VS and the debugger and started looking into >> > > things >> > > >> > and >> > > >> > > I found that I was hitting at least one infiinte loop dealing >> with >> > > >> > > randomized values -- specifically at >> > > >> > > >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> https://github.com/apache/lucene.net/blob/master/src/Lucene.Net.TestFramework/Index/BasePostingsFormatTestCase.cs#L394 >> > > >> > > ; >> > > >> > > >> > > >> > > Anyhow, I'm not sure how to proceed here as if we want test >> > > automation >> > > >> > the >> > > >> > > tests need to run through in a reasonable amount of time but I >> > don't >> > > >> know >> > > >> > > enough about the project to know what should run or not. One >> > thought >> > > >> > would >> > > >> > > be to use nunit timeout attributes ( >> > > >> > > http://www.nunit.org/index.php?p=timeout&r=2.5) to constrain >> > > things. >> > > >> If >> > > >> > so >> > > >> > > what is a reasonable timeout? >> > > >> > > >> > > >> > > I'm quite open to other, non kludgy thoughts too . . . >> > > >> > > >> > > >> > > On Tue, Nov 18, 2014 at 12:00 PM, Wyatt Barnett < >> > > >> [email protected] >> > > >> > > >> > > >> > > wrote: >> > > >> > > >> > > >> > > > I looked over GitVersion -- looks like a great fit for this >> > > project >> > > >> > > though >> > > >> > > > it will require a bit of forethought about branching >> strategies. >> > > >> > > > >> > > >> > > > I'll take a run at getting it integrated once I get through >> the >> > > test >> > > >> > > suite >> > > >> > > > running successfully. >> > > >> > > > >> > > >> > > > On Sat, Nov 15, 2014 at 11:59 AM, Itamar Syn-Hershko < >> > > >> > [email protected] >> > > >> > > > >> > > >> > > > wrote: >> > > >> > > > >> > > >> > > >> Inline >> > > >> > > >> >> > > >> > > >> -- >> > > >> > > >> >> > > >> > > >> Itamar Syn-Hershko >> > > >> > > >> http://code972.com | @synhershko < >> > https://twitter.com/synhershko >> > > > >> > > >> > > >> Freelance Developer & Consultant >> > > >> > > >> Author of RavenDB in Action <http://manning.com/synhershko/ >> > >> > > >> > > >> >> > > >> > > >> On Sat, Nov 15, 2014 at 6:53 PM, Wyatt Barnett < >> > > >> > [email protected] >> > > >> > > > >> > > >> > > >> wrote: >> > > >> > > >> >> > > >> > > >> > Thanks. I registered at CodeBetter.com under wwb. Is there >> > any >> > > >> > > >> > documentation on their TeamCity setup or someone who I can >> > > reach >> > > >> out >> > > >> > > to >> > > >> > > >> > regarding questions about the build environment? >> > > >> > > >> > >> > > >> > > >> >> > > >> > > >> I'm asking around, will let you know >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> > >> > > >> > > >> > Thanks for the rundown -- things sound pretty >> straightforward >> > > and >> > > >> > > >> doable. >> > > >> > > >> > One thing we'll need to think a bit a bout is how do we >> want >> > to >> > > >> > manage >> > > >> > > >> the >> > > >> > > >> > git branching strategy to best integrate with TeamCity and >> > best >> > > >> > > automate >> > > >> > > >> > the release cycle. Doing things like constantly building a >> > > >> "trunk" >> > > >> > and >> > > >> > > >> > releasing based on tags are very doable. >> > > >> > > >> > >> > > >> > > >> >> > > >> > > >> Are you familiar with >> > > https://github.com/ParticularLabs/GitVersion >> > > >> ? >> > > >> > > I'll >> > > >> > > >> be interested in adopting this to our process. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> > >> > > >> > > >> > I've grabbed the code, looks like things are pretty clean >> in >> > > >> terms >> > > >> > of >> > > >> > > >> being >> > > >> > > >> > able to build and run cleanly. One question -- I started >> > > running >> > > >> > the >> > > >> > > >> test >> > > >> > > >> > suite, it appears to execute about 80% successfully. I'm >> > > >> presuming >> > > >> > > this >> > > >> > > >> is >> > > >> > > >> > because we are still porting 4.8.0 here and is expected >> > > behavior. >> > > >> > > Please >> > > >> > > >> > confirm. >> > > >> > > >> > >> > > >> > > >> >> > > >> > > >> Yes, we still have some failing tests. The hope is to also >> > > utilize >> > > >> > > >> TeamCity's reports to measure the affects of internal >> changes >> > we >> > > >> make >> > > >> > > >> faster. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> > >> > > >> > > >> > I'll take a few stabs at a build cycle over the next few >> days >> > > and >> > > >> > see >> > > >> > > >> what >> > > >> > > >> > I can shake out. >> > > >> > > >> > >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> Thanks! >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> > >> > > >> > > >> > On Sat, Nov 15, 2014 at 11:06 AM, Itamar Syn-Hershko < >> > > >> > > >> [email protected]> >> > > >> > > >> > wrote: >> > > >> > > >> > >> > > >> > > >> > > Thanks Wyatt >> > > >> > > >> > > >> > > >> > > >> > > Can you please register at >> http://teamcity.codebetter.com/ >> > > and >> > > >> > send >> > > >> > > >> me >> > > >> > > >> > > your >> > > >> > > >> > > user name (privately if you prefer)? >> > > >> > > >> > > >> > > >> > > >> > > Please use the master branch of our Apache git git:// >> > > >> > > >> > > git.apache.org/lucene.net.git - or the mirror at >> > > >> > > >> > > https://github.com/apache/lucene.net (would rather you >> use >> > > the >> > > >> > > >> original >> > > >> > > >> > > one >> > > >> > > >> > > to avoid delays) >> > > >> > > >> > > >> > > >> > > >> > > The way I see it is this: >> > > >> > > >> > > * Lucene.Net.TestFramework compiles and generates a >> nuget >> > > >> package >> > > >> > > >> > > * Lucene.Net.Core and Lucene.Net.Tests compiles, taking >> > > >> dependency >> > > >> > > on >> > > >> > > >> > > Lucene.Net.TestFramework >> > > >> > > >> > > * Lucene.Net.Tests is run and if successful versions the >> > > build >> > > >> and >> > > >> > > >> > > generates nuget on the local feed (we also have a MyGet >> > > >> account to >> > > >> > > >> work >> > > >> > > >> > > with) >> > > >> > > >> > > * Same process to all sub-projects: Lucene.Net.Queries >> for >> > > >> > example: >> > > >> > > >> > > * Compile >> > > >> > > >> > > * Compile Lucene.Net.Tests.Queries >> > > >> > > >> > > * Run tests from Lucene.Net.Tests.Queries (take >> > dependency >> > > on >> > > >> > > >> > > Lucene.Net.TestFramework) >> > > >> > > >> > > * Create nuget for Lucene.Net.Queries >> > > >> > > >> > > * Every sub-project should be re-compiled and tests >> re-run >> > if >> > > >> the >> > > >> > > >> > projects >> > > >> > > >> > > it depends on have changed (you can probably separate >> the >> > > >> cases by >> > > >> > > >> > defining >> > > >> > > >> > > watch folders under src\) >> > > >> > > >> > > >> > > >> > > >> > > We can then iterate from there. >> > > >> > > >> > > >> > > >> > > >> > > I updated the README to explain the new structure >> > > >> > > >> > > >> > > >> https://github.com/apache/lucene.net/blob/master/README.md#files >> > > >> > - >> > > >> > > >> let >> > > >> > > >> > me >> > > >> > > >> > > know if you have any questions. >> > > >> > > >> > > >> > > >> > > >> > > -- >> > > >> > > >> > > >> > > >> > > >> > > Itamar Syn-Hershko >> > > >> > > >> > > http://code972.com | @synhershko < >> > > >> https://twitter.com/synhershko> >> > > >> > > >> > > Freelance Developer & Consultant >> > > >> > > >> > > Author of RavenDB in Action < >> > http://manning.com/synhershko/> >> > > >> > > >> > > >> > > >> > > >> > > On Sat, Nov 15, 2014 at 5:51 AM, Wyatt Barnett < >> > > >> > > >> [email protected]> >> > > >> > > >> > > wrote: >> > > >> > > >> > > >> > > >> > > >> > > > That would be great -- let me know what I need to do >> to >> > > help >> > > >> > make >> > > >> > > >> that >> > > >> > > >> > > > happen. >> > > >> > > >> > > > >> > > >> > > >> > > > In the meantime I've got a teamcity server to work >> with >> > > here, >> > > >> > > >> should I >> > > >> > > >> > be >> > > >> > > >> > > > looking at >> > > >> > > >> > > >> > > https://github.com/synhershko/lucene.net/tree/Lucene.Net_4.8.0 >> > > >> > > >> > > > as the project layout. >> > > >> > > >> > > > >> > > >> > > >> > > > Also, what is envisioned for the output of the build >> > > >> pipeline? >> > > >> > > Many >> > > >> > > >> > > things >> > > >> > > >> > > > are within the art of the possible. >> > > >> > > >> > > > >> > > >> > > >> > > > On Fri, Nov 14, 2014 at 9:08 PM, Prescott Nasser < >> > > >> > > >> > [email protected]> >> > > >> > > >> > > > wrote: >> > > >> > > >> > > > >> > > >> > > >> > > > > Would love the help setting this up - Itamar do you >> > know >> > > >> if we >> > > >> > > can >> > > >> > > >> > > > provide >> > > >> > > >> > > > > Wyatt access for this? >> > > >> > > >> > > > > ________________________________ >> > > >> > > >> > > > > From: Wyatt Barnett<mailto:[email protected]> >> > > >> > > >> > > > > Sent: 11/14/2014 4:26 PM >> > > >> > > >> > > > > To: [email protected]<mailto: >> > > >> [email protected]> >> > > >> > > >> > > > > Subject: Re: Setting up the CI pipeline >> > > >> > > >> > > > > >> > > >> > > >> > > > > I do a lot of CI and CD and I've got loads of seat >> time >> > > >> with >> > > >> > > >> > TeamCity, >> > > >> > > >> > > I >> > > >> > > >> > > > > would be happy to help the cause. >> > > >> > > >> > > > > >> > > >> > > >> > > > > On Fri, Nov 14, 2014 at 5:49 AM, Itamar Syn-Hershko >> < >> > > >> > > >> > > [email protected]> >> > > >> > > >> > > > > wrote: >> > > >> > > >> > > > > >> > > >> > > >> > > > > > Heya, >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > So JetBrains and CodeBetter have setup a TeamCity >> > > account >> > > >> > for >> > > >> > > >> us - >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > >> > > >> > > >> >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> http://teamcity.codebetter.com/project.html?projectId=LuceneNet&tab=projectOverview >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > I have asked them to add Prescott and Troy as >> > > >> collaborators, >> > > >> > > so >> > > >> > > >> we >> > > >> > > >> > 3 >> > > >> > > >> > > > have >> > > >> > > >> > > > > > access to change stuff there. >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > The idea is to have every sub-project (Core, >> Codecs, >> > > >> > Queries, >> > > >> > > >> > Facets, >> > > >> > > >> > > > > etc) >> > > >> > > >> > > > > > compile and available as a nuget package, and also >> > > >> > thoroughly >> > > >> > > >> > tested >> > > >> > > >> > > > via >> > > >> > > >> > > > > > the test agents on TeamCity. >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > Prescott, Troy - you said you can work on setting >> > this >> > > >> up, >> > > >> > > will >> > > >> > > >> be >> > > >> > > >> > > > happy >> > > >> > > >> > > > > > for you to go ahead and do this now. >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > If there's anyone else on this list who has >> > experience >> > > >> with >> > > >> > > >> > TeamCity >> > > >> > > >> > > > and >> > > >> > > >> > > > > > setting up CI pipelines, please drop us a line if >> you >> > > are >> > > >> > > >> willing >> > > >> > > >> > to >> > > >> > > >> > > > help >> > > >> > > >> > > > > > with this effort. >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > Cheers, >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > -- >> > > >> > > >> > > > > > >> > > >> > > >> > > > > > Itamar Syn-Hershko >> > > >> > > >> > > > > > http://code972.com | @synhershko < >> > > >> > > >> https://twitter.com/synhershko> >> > > >> > > >> > > > > > Freelance Developer & Consultant >> > > >> > > >> > > > > > Author of RavenDB in Action < >> > > >> http://manning.com/synhershko/ >> > > >> > > >> > > >> > > >> > > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > >> > > >> > > >> >> > > >> > > > >> > > >> > > > >> > > >> > > >> > > >> > >> > > >> >> > > > >> > > > >> > > >> > >> > >
