Better yet, how about: 1. Replacing all Console.WriteLine with logger.WriteLine - but set it up so output does show up on TeamCity's output AND VisualStudio's Output window? this way we can get far better logs on demand.
2. In the CI server, have the VERBOSE and DEBUG settings tunable for when we do want to run tests and get the logs for further inspections -- Itamar Syn-Hershko http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer & Consultant Author of RavenDB in Action <http://manning.com/synhershko/> On Tue, Nov 25, 2014 at 10:41 PM, Paul Irwin <[email protected]> wrote: > I think I found the issue with the tests taking forever to run. > TestDoubleMockGraphTokenFilterRandom was running seemingly indefinitely -- > I don't actually think now that it would run indefinitely, I just didn't > feel like waiting on it after a good number of minutes. > > I noticed a strange behavior when I was stepping through the test to see > what was going on -- the console output would catch up every few times I'd > step over a statement, as if there were a background thread going on. > > I do think there is some background thread issue, but it isn't with the > Lucene.net code (as far as I can tell) -- it appears to be with NUnit > itself or perhaps the VS test framework. But since it happened on the CI > server too, I'm leaning towards NUnit. > > What I actually think is happening is that the Console output is handled > asynchronously in some background thread, and there's just *so much* > logging > happening in the LookaheadTokenFilter et al that when DEBUG evaluates to > true and the Console.WriteLine calls stack up, it takes forever to get > through them all. Running the tests in Release mode (when the > Console.WriteLine is not called) or manually setting DEBUG and VERBOSE to > evaluate to false even in Debug configuration, causes the test to pass in > 15 seconds on my machine. > > Wyatt, can you try running the build and tests in Release mode and see if > it runs better for you? We might also be able to get through this by > reducing the logging by commenting out the Console.WriteLine calls. > > > Paul Irwin > Lead Software Engineer > feature[23] > > Email: [email protected] > Cell: 863-698-9294 > > On Mon, Nov 24, 2014 at 9:50 AM, Itamar Syn-Hershko <[email protected]> > wrote: > > > Please do. > > > > Send a ICLA to secretary@apache and send us a PR, I'll take care of > > merging > > it > > > > Thanks! > > > > -- > > > > Itamar Syn-Hershko > > http://code972.com | @synhershko <https://twitter.com/synhershko> > > Freelance Developer & Consultant > > Author of RavenDB in Action <http://manning.com/synhershko/> > > > > On Mon, Nov 24, 2014 at 4:48 PM, Wyatt Barnett <[email protected]> > > wrote: > > > > > So, after fighting a number of teething problems -- many of them self > > > inflicted -- I've got a solid, repeatable test run that finishes in 38 > > > minutes or so on our hardware. > > > > > > I can certainly reprise this setup over on teamcity.codebetter.com > > without > > > much effort. We will need to merge some changes into things before > > > proceeding -- the current state of tests in the trunk will just hang if > > we > > > tried to run them there. I'm guessing I'll need to sign a contributing > > > agreement here as I don't believe I filed one. > > > > > > On Tue, Nov 18, 2014 at 5:19 PM, Wyatt Barnett < > [email protected]> > > > wrote: > > > > > > > Sounds good, we'll be [Explicit()] with reason. > > > > > > > > On Tue, Nov 18, 2014 at 5:06 PM, Itamar Syn-Hershko < > > [email protected]> > > > > wrote: > > > > > > > >> Yes, but Explicit() will help us isolate those issues from other > real > > > bugs > > > >> we can concentrate on solving in parallel. > > > >> > > > >> -- > > > >> > > > >> Itamar Syn-Hershko > > > >> http://code972.com | @synhershko <https://twitter.com/synhershko> > > > >> Freelance Developer & Consultant > > > >> Author of RavenDB in Action <http://manning.com/synhershko/> > > > >> > > > >> On Wed, Nov 19, 2014 at 12:01 AM, Wyatt Barnett < > > > [email protected]> > > > >> wrote: > > > >> > > > >> > [Ignore()] is good too, I actually prefer a slightly nuanced > version > > > >> called > > > >> > [Explicit()] as that lets you still fire off the test from > resharper > > > or > > > >> > nunit gui. > > > >> > > > > >> > That said I proposed Assert.Fail() here because we can put the > > failure > > > >> > point at the *exact* point where the folks should start debugging > > this > > > >> from > > > >> > versus having them start a at a whole test or test fixture. > > > >> > > > > >> > On Tue, Nov 18, 2014 at 4:56 PM, Itamar Syn-Hershko < > > > [email protected] > > > >> > > > > >> > wrote: > > > >> > > > > >> > > Don't forget to stick a reason to the Ignore property tho! > > > >> > > > > > >> > > -- > > > >> > > > > > >> > > Itamar Syn-Hershko > > > >> > > http://code972.com | @synhershko < > https://twitter.com/synhershko> > > > >> > > Freelance Developer & Consultant > > > >> > > Author of RavenDB in Action <http://manning.com/synhershko/> > > > >> > > > > > >> > > On Tue, Nov 18, 2014 at 11:55 PM, Itamar Syn-Hershko < > > > >> [email protected] > > > >> > > > > > >> > > wrote: > > > >> > > > > > >> > > > Or just Skip (or Ignore, however that's called in NUnit) :) > > > >> > > > > > > >> > > > -- > > > >> > > > > > > >> > > > Itamar Syn-Hershko > > > >> > > > http://code972.com | @synhershko < > > https://twitter.com/synhershko> > > > >> > > > Freelance Developer & Consultant > > > >> > > > Author of RavenDB in Action <http://manning.com/synhershko/> > > > >> > > > > > > >> > > > On Tue, Nov 18, 2014 at 11:47 PM, Wyatt Barnett < > > > >> > [email protected] > > > >> > > > > > > >> > > > wrote: > > > >> > > > > > > >> > > >> 5 minutes sounds like a reasonable going in proposition. That > > > said > > > >> > > >> depending on how many of these there are there might well be > a > > > >> *lot* > > > >> > of > > > >> > > 5 > > > >> > > >> minute waits. > > > >> > > >> > > > >> > > >> Would it help if I just stuck Assert.Fails() where I run into > > > these > > > >> > > loops > > > >> > > >> for the folks smarter than I to run down? > > > >> > > >> > > > >> > > >> On Tue, Nov 18, 2014 at 4:37 PM, Itamar Syn-Hershko < > > > >> > [email protected] > > > >> > > > > > > >> > > >> wrote: > > > >> > > >> > > > >> > > >> > 5 mins max for one test maybe? > > > >> > > >> > > > > >> > > >> > To fix that infinite loop we need to revisit the original > > Java > > > >> code, > > > >> > > >> most > > > >> > > >> > likely its porting of an iterator-style code that went > > wrong. I > > > >> hope > > > >> > > to > > > >> > > >> > have time to look at it next week, please anyone else who > > feels > > > >> like > > > >> > > it > > > >> > > >> > beat me to it.. > > > >> > > >> > > > > >> > > >> > -- > > > >> > > >> > > > > >> > > >> > Itamar Syn-Hershko > > > >> > > >> > http://code972.com | @synhershko < > > > https://twitter.com/synhershko > > > >> > > > > >> > > >> > Freelance Developer & Consultant > > > >> > > >> > Author of RavenDB in Action < > http://manning.com/synhershko/> > > > >> > > >> > > > > >> > > >> > On Tue, Nov 18, 2014 at 11:13 PM, Wyatt Barnett < > > > >> > > >> [email protected]> > > > >> > > >> > wrote: > > > >> > > >> > > > > >> > > >> > > So, after the 2nd day of watching the tests just spin I > > > decided > > > >> > > that a > > > >> > > >> > > little visibility might make sense as something just was > > not > > > >> > adding > > > >> > > up > > > >> > > >> > -- I > > > >> > > >> > > expected some failing tests, and some long-running tests > > but > > > >> there > > > >> > > >> just > > > >> > > >> > > ain't enough data here to keep something running for 8+ > > > hours. > > > >> > > >> > > > > > >> > > >> > > Anyhow, I stood up VS and the debugger and started > looking > > > into > > > >> > > things > > > >> > > >> > and > > > >> > > >> > > I found that I was hitting at least one infiinte loop > > dealing > > > >> with > > > >> > > >> > > randomized values -- specifically at > > > >> > > >> > > > > > >> > > >> > > > > > >> > > >> > > > > >> > > >> > > > >> > > > > > >> > > > > >> > > > > > > https://github.com/apache/lucene.net/blob/master/src/Lucene.Net.TestFramework/Index/BasePostingsFormatTestCase.cs#L394 > > > >> > > >> > > ; > > > >> > > >> > > > > > >> > > >> > > Anyhow, I'm not sure how to proceed here as if we want > test > > > >> > > automation > > > >> > > >> > the > > > >> > > >> > > tests need to run through in a reasonable amount of time > > but > > > I > > > >> > don't > > > >> > > >> know > > > >> > > >> > > enough about the project to know what should run or not. > > One > > > >> > thought > > > >> > > >> > would > > > >> > > >> > > be to use nunit timeout attributes ( > > > >> > > >> > > http://www.nunit.org/index.php?p=timeout&r=2.5) to > > constrain > > > >> > > things. > > > >> > > >> If > > > >> > > >> > so > > > >> > > >> > > what is a reasonable timeout? > > > >> > > >> > > > > > >> > > >> > > I'm quite open to other, non kludgy thoughts too . . . > > > >> > > >> > > > > > >> > > >> > > On Tue, Nov 18, 2014 at 12:00 PM, Wyatt Barnett < > > > >> > > >> [email protected] > > > >> > > >> > > > > > >> > > >> > > wrote: > > > >> > > >> > > > > > >> > > >> > > > I looked over GitVersion -- looks like a great fit for > > this > > > >> > > project > > > >> > > >> > > though > > > >> > > >> > > > it will require a bit of forethought about branching > > > >> strategies. > > > >> > > >> > > > > > > >> > > >> > > > I'll take a run at getting it integrated once I get > > through > > > >> the > > > >> > > test > > > >> > > >> > > suite > > > >> > > >> > > > running successfully. > > > >> > > >> > > > > > > >> > > >> > > > On Sat, Nov 15, 2014 at 11:59 AM, Itamar Syn-Hershko < > > > >> > > >> > [email protected] > > > >> > > >> > > > > > > >> > > >> > > > wrote: > > > >> > > >> > > > > > > >> > > >> > > >> Inline > > > >> > > >> > > >> > > > >> > > >> > > >> -- > > > >> > > >> > > >> > > > >> > > >> > > >> Itamar Syn-Hershko > > > >> > > >> > > >> http://code972.com | @synhershko < > > > >> > https://twitter.com/synhershko > > > >> > > > > > > >> > > >> > > >> Freelance Developer & Consultant > > > >> > > >> > > >> Author of RavenDB in Action < > > > http://manning.com/synhershko/ > > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> On Sat, Nov 15, 2014 at 6:53 PM, Wyatt Barnett < > > > >> > > >> > [email protected] > > > >> > > >> > > > > > > >> > > >> > > >> wrote: > > > >> > > >> > > >> > > > >> > > >> > > >> > Thanks. I registered at CodeBetter.com under wwb. Is > > > there > > > >> > any > > > >> > > >> > > >> > documentation on their TeamCity setup or someone > who I > > > can > > > >> > > reach > > > >> > > >> out > > > >> > > >> > > to > > > >> > > >> > > >> > regarding questions about the build environment? > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> I'm asking around, will let you know > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > > >> > > >> > > >> > Thanks for the rundown -- things sound pretty > > > >> straightforward > > > >> > > and > > > >> > > >> > > >> doable. > > > >> > > >> > > >> > One thing we'll need to think a bit a bout is how do > > we > > > >> want > > > >> > to > > > >> > > >> > manage > > > >> > > >> > > >> the > > > >> > > >> > > >> > git branching strategy to best integrate with > TeamCity > > > and > > > >> > best > > > >> > > >> > > automate > > > >> > > >> > > >> > the release cycle. Doing things like constantly > > > building a > > > >> > > >> "trunk" > > > >> > > >> > and > > > >> > > >> > > >> > releasing based on tags are very doable. > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> Are you familiar with > > > >> > > https://github.com/ParticularLabs/GitVersion > > > >> > > >> ? > > > >> > > >> > > I'll > > > >> > > >> > > >> be interested in adopting this to our process. > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > > >> > > >> > > >> > I've grabbed the code, looks like things are pretty > > > clean > > > >> in > > > >> > > >> terms > > > >> > > >> > of > > > >> > > >> > > >> being > > > >> > > >> > > >> > able to build and run cleanly. One question -- I > > > started > > > >> > > running > > > >> > > >> > the > > > >> > > >> > > >> test > > > >> > > >> > > >> > suite, it appears to execute about 80% successfully. > > I'm > > > >> > > >> presuming > > > >> > > >> > > this > > > >> > > >> > > >> is > > > >> > > >> > > >> > because we are still porting 4.8.0 here and is > > expected > > > >> > > behavior. > > > >> > > >> > > Please > > > >> > > >> > > >> > confirm. > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> Yes, we still have some failing tests. The hope is to > > also > > > >> > > utilize > > > >> > > >> > > >> TeamCity's reports to measure the affects of internal > > > >> changes > > > >> > we > > > >> > > >> make > > > >> > > >> > > >> faster. > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > > >> > > >> > > >> > I'll take a few stabs at a build cycle over the next > > few > > > >> days > > > >> > > and > > > >> > > >> > see > > > >> > > >> > > >> what > > > >> > > >> > > >> > I can shake out. > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> Thanks! > > > >> > > >> > > >> > > > >> > > >> > > >> > > > >> > > >> > > >> > > > > >> > > >> > > >> > On Sat, Nov 15, 2014 at 11:06 AM, Itamar > Syn-Hershko < > > > >> > > >> > > >> [email protected]> > > > >> > > >> > > >> > wrote: > > > >> > > >> > > >> > > > > >> > > >> > > >> > > Thanks Wyatt > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > Can you please register at > > > >> http://teamcity.codebetter.com/ > > > >> > > and > > > >> > > >> > send > > > >> > > >> > > >> me > > > >> > > >> > > >> > > your > > > >> > > >> > > >> > > user name (privately if you prefer)? > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > Please use the master branch of our Apache git > > git:// > > > >> > > >> > > >> > > git.apache.org/lucene.net.git - or the mirror at > > > >> > > >> > > >> > > https://github.com/apache/lucene.net (would > rather > > > you > > > >> use > > > >> > > the > > > >> > > >> > > >> original > > > >> > > >> > > >> > > one > > > >> > > >> > > >> > > to avoid delays) > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > The way I see it is this: > > > >> > > >> > > >> > > * Lucene.Net.TestFramework compiles and generates > a > > > >> nuget > > > >> > > >> package > > > >> > > >> > > >> > > * Lucene.Net.Core and Lucene.Net.Tests compiles, > > > taking > > > >> > > >> dependency > > > >> > > >> > > on > > > >> > > >> > > >> > > Lucene.Net.TestFramework > > > >> > > >> > > >> > > * Lucene.Net.Tests is run and if successful > versions > > > the > > > >> > > build > > > >> > > >> and > > > >> > > >> > > >> > > generates nuget on the local feed (we also have a > > > MyGet > > > >> > > >> account to > > > >> > > >> > > >> work > > > >> > > >> > > >> > > with) > > > >> > > >> > > >> > > * Same process to all sub-projects: > > Lucene.Net.Queries > > > >> for > > > >> > > >> > example: > > > >> > > >> > > >> > > * Compile > > > >> > > >> > > >> > > * Compile Lucene.Net.Tests.Queries > > > >> > > >> > > >> > > * Run tests from Lucene.Net.Tests.Queries (take > > > >> > dependency > > > >> > > on > > > >> > > >> > > >> > > Lucene.Net.TestFramework) > > > >> > > >> > > >> > > * Create nuget for Lucene.Net.Queries > > > >> > > >> > > >> > > * Every sub-project should be re-compiled and > tests > > > >> re-run > > > >> > if > > > >> > > >> the > > > >> > > >> > > >> > projects > > > >> > > >> > > >> > > it depends on have changed (you can probably > > separate > > > >> the > > > >> > > >> cases by > > > >> > > >> > > >> > defining > > > >> > > >> > > >> > > watch folders under src\) > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > We can then iterate from there. > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > I updated the README to explain the new structure > > > >> > > >> > > >> > > > > > >> > > >> > > https://github.com/apache/lucene.net/blob/master/README.md#files > > > >> > > >> > - > > > >> > > >> > > >> let > > > >> > > >> > > >> > me > > > >> > > >> > > >> > > know if you have any questions. > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > -- > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > Itamar Syn-Hershko > > > >> > > >> > > >> > > http://code972.com | @synhershko < > > > >> > > >> https://twitter.com/synhershko> > > > >> > > >> > > >> > > Freelance Developer & Consultant > > > >> > > >> > > >> > > Author of RavenDB in Action < > > > >> > http://manning.com/synhershko/> > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > On Sat, Nov 15, 2014 at 5:51 AM, Wyatt Barnett < > > > >> > > >> > > >> [email protected]> > > > >> > > >> > > >> > > wrote: > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > > That would be great -- let me know what I need > to > > do > > > >> to > > > >> > > help > > > >> > > >> > make > > > >> > > >> > > >> that > > > >> > > >> > > >> > > > happen. > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > In the meantime I've got a teamcity server to > work > > > >> with > > > >> > > here, > > > >> > > >> > > >> should I > > > >> > > >> > > >> > be > > > >> > > >> > > >> > > > looking at > > > >> > > >> > > >> > > > > > >> > > https://github.com/synhershko/lucene.net/tree/Lucene.Net_4.8.0 > > > >> > > >> > > >> > > > as the project layout. > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > Also, what is envisioned for the output of the > > build > > > >> > > >> pipeline? > > > >> > > >> > > Many > > > >> > > >> > > >> > > things > > > >> > > >> > > >> > > > are within the art of the possible. > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > On Fri, Nov 14, 2014 at 9:08 PM, Prescott > Nasser < > > > >> > > >> > > >> > [email protected]> > > > >> > > >> > > >> > > > wrote: > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > > Would love the help setting this up - Itamar > do > > > you > > > >> > know > > > >> > > >> if we > > > >> > > >> > > can > > > >> > > >> > > >> > > > provide > > > >> > > >> > > >> > > > > Wyatt access for this? > > > >> > > >> > > >> > > > > ________________________________ > > > >> > > >> > > >> > > > > From: Wyatt Barnett<mailto: > > > [email protected]> > > > >> > > >> > > >> > > > > Sent: 11/14/2014 4:26 PM > > > >> > > >> > > >> > > > > To: [email protected]<mailto: > > > >> > > >> [email protected]> > > > >> > > >> > > >> > > > > Subject: Re: Setting up the CI pipeline > > > >> > > >> > > >> > > > > > > > >> > > >> > > >> > > > > I do a lot of CI and CD and I've got loads of > > seat > > > >> time > > > >> > > >> with > > > >> > > >> > > >> > TeamCity, > > > >> > > >> > > >> > > I > > > >> > > >> > > >> > > > > would be happy to help the cause. > > > >> > > >> > > >> > > > > > > > >> > > >> > > >> > > > > On Fri, Nov 14, 2014 at 5:49 AM, Itamar > > > Syn-Hershko > > > >> < > > > >> > > >> > > >> > > [email protected]> > > > >> > > >> > > >> > > > > wrote: > > > >> > > >> > > >> > > > > > > > >> > > >> > > >> > > > > > Heya, > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > So JetBrains and CodeBetter have setup a > > > TeamCity > > > >> > > account > > > >> > > >> > for > > > >> > > >> > > >> us - > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > > > > >> > > >> > > > > >> > > >> > > > >> > > > > > >> > > > > >> > > > > > > http://teamcity.codebetter.com/project.html?projectId=LuceneNet&tab=projectOverview > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > I have asked them to add Prescott and Troy > as > > > >> > > >> collaborators, > > > >> > > >> > > so > > > >> > > >> > > >> we > > > >> > > >> > > >> > 3 > > > >> > > >> > > >> > > > have > > > >> > > >> > > >> > > > > > access to change stuff there. > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > The idea is to have every sub-project (Core, > > > >> Codecs, > > > >> > > >> > Queries, > > > >> > > >> > > >> > Facets, > > > >> > > >> > > >> > > > > etc) > > > >> > > >> > > >> > > > > > compile and available as a nuget package, > and > > > also > > > >> > > >> > thoroughly > > > >> > > >> > > >> > tested > > > >> > > >> > > >> > > > via > > > >> > > >> > > >> > > > > > the test agents on TeamCity. > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > Prescott, Troy - you said you can work on > > > setting > > > >> > this > > > >> > > >> up, > > > >> > > >> > > will > > > >> > > >> > > >> be > > > >> > > >> > > >> > > > happy > > > >> > > >> > > >> > > > > > for you to go ahead and do this now. > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > If there's anyone else on this list who has > > > >> > experience > > > >> > > >> with > > > >> > > >> > > >> > TeamCity > > > >> > > >> > > >> > > > and > > > >> > > >> > > >> > > > > > setting up CI pipelines, please drop us a > line > > > if > > > >> you > > > >> > > are > > > >> > > >> > > >> willing > > > >> > > >> > > >> > to > > > >> > > >> > > >> > > > help > > > >> > > >> > > >> > > > > > with this effort. > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > Cheers, > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > -- > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > Itamar Syn-Hershko > > > >> > > >> > > >> > > > > > http://code972.com | @synhershko < > > > >> > > >> > > >> https://twitter.com/synhershko> > > > >> > > >> > > >> > > > > > Freelance Developer & Consultant > > > >> > > >> > > >> > > > > > Author of RavenDB in Action < > > > >> > > >> http://manning.com/synhershko/ > > > >> > > >> > > > > > >> > > >> > > >> > > > > > > > > >> > > >> > > >> > > > > > > > >> > > >> > > >> > > > > > > >> > > >> > > >> > > > > > >> > > >> > > >> > > > > >> > > >> > > >> > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > >> > > >> > > > > >> > > >> > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > >
