I believe the statics are remnants of the Lucene Java port, and mostly readonly configurations. Are any of those statics mutable?
Adding on the timeout thing - this indeed was a precaution. We should probably move from a attribute-based timeout to throwing a TimeoutException from the main test method, having the main loop check for the time passed since start and throw if it's above the timeout configured? -- Itamar Syn-Hershko http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer & Consultant Lucene.NET committer and PMC member On Fri, Jun 24, 2016 at 10:12 PM, Connie Yau <[email protected]> wrote: > Hi, > > Thank you for the input! I will add a Trait for the tests that have > Timeouts for now so they can be fixed to be more stable. > > One more question... While migrating the code to xUnit, I noticed that > there are a lot of `static` variables in LuceneTestCase and which would > make it hard to run in parallel because some test classes modify the static > values. Is there a reason for keeping them static? > > Thanks! > Connie > > -----Original Message----- > From: Laimonas Simutis [mailto:[email protected]] > Sent: Wednesday, June 22, 2016 2:54 PM > To: [email protected] > Subject: Re: Lucene.NET to .NET Core > > Wyatt is right on. Certain tests under random circumstances could get into > a bad state and run "forever", potentially generate gigs of data on disk, > etc and so timeout was added to abort the test earlier in case that > happened. > > Our plan was to make a pass at those long running tests and fix the issues > that cause them to run so long once we are fully complete with the porting. > > Perhaps you can add a category / trait on such tests in order to avoid > them until they can be properly investigated and underlying issues fixed? > > > > On Wed, Jun 22, 2016 at 9:26 AM, Wyatt Barnett <[email protected]> > wrote: > > > I don't know the history on why the timeouts were added but I suspect > > they were self-defensive in nature. I think in most cases the timeouts > > are really back-stopping things so that if there is a race condition > > the test suite bails and fails rather than runs forever. Is there a > > way to backstop stuff like that with xunit? > > > > > > On Tue, Jun 21, 2016 at 8:24 PM Connie Yau <[email protected]> > wrote: > > > > > Hi, > > > > > > I've been looking into migrating existing tests from NUnit to xUnit > > > and I think it's doable with the suggestions that were proposed. > > > The only question I need answered from the community is: > > > > > > "How important are timeouts in your test cases?" > > > - xUnit runs tests asynchronously and parallel, so they removed the > > > ability to have timeout because the ability to time these tests are > > > not reliable. > > > - There are 4 timeouts that I see in the cases. > > > - int.MaxValue > > > - 5 minutes > > > - 2.5 minutes > > > - 40000 milliseconds > > > My current approach to dealing with timeouts is removing them > > > because the tests run concurrently, so we aren’t waiting for a single > test to finish. > > > > > > Thanks! > > > Connie > > > > > > -----Original Message----- > > > From: Elizabeth Maher (NEWMAN) > > > [mailto:[email protected]] > > > Sent: Monday, June 20, 2016 12:26 PM > > > To: [email protected] > > > Subject: RE: Lucene.NET to .NET Core > > > > > > Our goal with the tests was to keep it working with the existing > > > system with a few changes as possible. We thought there would be > > > more changes than the community was comfortable with. However, it > > > sounds like the interest in xUnit is more that we realized. > > > > > > There are three items that concerned us in the xUnit move. > > > 1. No categories - Traits can be used instead. Hopefully just some > > > simple cut and paste. > > > 2. No timeout for test cases - This is the biggest issue as tc > > > timeout > > is > > > frequently used in the test cases I saw. We are not sure the effect > > > of this or how to overcome any issues related to this. > > > 3. TeamCity integration - Looks like Wyatt found a good article > > > explaining how to do this. > > > > > > Perhaps Connie can look into the first two issues and Wyatt can look > > > into the last one? Connie will respond with an update by tomorrow > > > evening and then we can decide together. > > > > > > On a sad note, I've been pulled off onto another project. Connie > > > will take point on this project and finish up necessary work. > > > > > > Thanks, > > > Elizabeth > > > > > > > > > -----Original Message----- > > > From: Wyatt Barnett [mailto:[email protected]] > > > Sent: Monday, June 20, 2016 8:12 AM > > > To: [email protected] > > > Subject: Re: Lucene.NET to .NET Core > > > > > > Sorry for the delayed response, I was off at the beach last weekend. > > > > > > Not a whole lot of seat-time w/ xunit here. In terms of running it, > > > there is a plugin, we can see if CodeBetter would be willing to > > > install it. The hand-plumbed option doesn't seem to daunting either. > > > In terms of > > categories > > > that is a better question but it seems like Xunit has a [Traits] > > attribute > > > that could be used -- see > > > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fstacko > > verflow.com%2fquestions%2f21791739%2fexecute-tests-based-on-xunit-filt > > ered-by-traits-in-teamcity&data=01%7c01%7cElizabeth.Maher%40microsoft. > > com%7c93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011d > > b47%7c1&sdata=SaJ%2b%2b5r2mjXE6s550uotyCpNxB9eTP5EH577DmSU8to%3d > > > for > > > an example. > > > > > > On Sun, Jun 19, 2016 at 5:49 AM Itamar Syn-Hershko > > > <[email protected]> > > > wrote: > > > > > > > "All these issues together lead us to the conclusion that we would > > > > be doing a disservice to just blindly port to xUnit." - why do you > > > > say that? if a move to xUnit benefits us on multiple occasions > > > > (better API, .NET Core support, faster runs) this one-tine effort > > > > would be > > worth > > > it - no? > > > > > > > > Wyatt, can you chime in perhaps - you have the most experience > > > > getting the build system to run, and test categorisation > > > > > > > > Thanks > > > > > > > > -- > > > > > > > > Itamar Syn-Hershko > > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fco > > > > de97 > > > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c93aa1f13228 > > > > a414 > > > > 2f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=Upe > > > > iUeT A6iiP7fV7dZ9Z9bKiMjgqo990K6xyt4D6LTs%3d | @synhershko > > > > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f > > > > twit > > > > ter.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.co > > > > m%7c > > > > 93aa1f13228a4142f26108d3991d5513%7c72f988bf86f141af91ab2d7cd011db4 > > > > 7%7c 1&sdata=kc%2fH3fEbR68ajKlN3pZ7vISmdPb%2bIRTHWrDTZ4FdplY%3d> > > > > Freelance Developer & Consultant > > > > Lucene.NET committer and PMC member > > > > > > > > On Fri, Jun 17, 2016 at 7:06 AM, Elizabeth Maher (NEWMAN) < > > > > [email protected]> wrote: > > > > > > > > > Hi all. I have a quick update on our progress. > > > > > > > > > > We did a test of what it would be like to port Lucene.NET to use > > xUnit. > > > > > As Itamer guessed, it is quite a bit of work. The basic work of > > > > > renaming attributes and changing out package names and the like > > > > > is a little > > > > tedious, > > > > > but it is doable. The issues comes in because of the use of > > > > > categories > > > > and > > > > > timeout attributes. Creating categories in xUnit is possible, > > > > > but rather complicated. The bigger issue is that my research > > > > > indicated that > > > > TeamCity > > > > > only has basic integration with xUnit and you have to run all > > > > > the > > > tests. > > > > > Running only categories requires extra scripting. The bigger > > > > > issue was that there is no TimeOut attribute in xUnit so longer > > > > > running tests are > > > > an > > > > > issue. We would also have to re-arrange some the test fixture > > > > > setup and teardown code into class constructors and dispose > > > > > methods, but that work was doable. All these issues together > > > > > lead us to the conclusion that we would be doing a disservice to > > > > > just blindly port > > to > > > xUnit. > > > > > > > > > > nUnit has just release an alpha package that runs on .NET Core RC2. > > > > > Connie is looking into getting working this week or next to so > > > > > we can > > > > keep > > > > > the use of such features as testcase timeouts and categories. > > > > > We'll send an update when we've tried out the alpha package. > > > > > > > > > > Thanks, > > > > > Elizabeth > > > > > > > > > > -----Original Message----- > > > > > From: Elizabeth Maher (NEWMAN) > > > > > [mailto:[email protected]] > > > > > Sent: Monday, May 23, 2016 4:40 PM > > > > > To: [email protected] > > > > > Subject: RE: Lucene.NET to .NET Core > > > > > > > > > > Thank you everyone for the feedback. > > > > > > > > > > It's sounds everyone is favor of the move to xUnit. I've talked > > > > > to a couple of my co-workers and they said the conversion can be > > > > > a little tedious, but often worth the effort. Give me a couple > > > > > days to see if > > > > it’s > > > > > a rabbits hole. I'll respond back if its more effort that I can > > > expend. > > > > > > > > > > Itamar, > > > > > Regarding your last question. There are no more know tasks > > > > > other than what I initially outlined. Once I merge Connie's > > > > > change in from master > > > > we > > > > > will have a building/ported assembly. I just need to get the > > > > > tests > > > > running > > > > > to verify everything works as expected. > > > > > > > > > > Thanks, > > > > > Elizabeth > > > > > > > > > > -----Original Message----- > > > > > From: [email protected] > > > > > [mailto:[email protected]] > > > > On > > > > > Behalf Of Itamar Syn-Hershko > > > > > Sent: Friday, May 20, 2016 6:46 AM > > > > > To: [email protected] > > > > > Subject: Re: Lucene.NET to .NET Core > > > > > > > > > > Hey there, > > > > > > > > > > Thank you so much again for your great work here. Lots of good > > > > > contributions we could definitely use. > > > > > > > > > > Moving to xUnit.Net was actually on my list of things to do - > > > > > basically also because it's API is much better and more flexible > > > > > (and less buggy!) and my experience with it was much better than > > > > > with NUnit (which we use > > > > an > > > > > outdated version of). Some people claim xUnit is also faster, so > > > > > hey > > > > > - > > > > why > > > > > not. > > > > > > > > > > Are you sure you can "easily move all tests to xUnit"? From what > > > > > I can tell this will be a rather rigorous copy-paste job? While > > > > > the JavaCompatibility bits we have ( > > > > > > > > > > > > > > https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.Tes > > > > tFra > > > > mework/JavaCompatibility > > > > > ) > > > > > do contain centralized function like assertTrue() etc that save > > > > > loads of time and effort porting tests, so that could be changed > > > > > easily to use xUnit, there are many places where asserts etc > > > > > where ported in full to > > > > the > > > > > NUnit API. > > > > > > > > > > I am definitely pro this change, but let's make sure we don't go > > > > > down > > > > into > > > > > a rabbit hole first. Let me know what you think. > > > > > > > > > > With regards to the status update: > > > > > > > > > > 1. I made some last minor comments on the merge scheduler PR - > > > > > we should be able to merge it within a few days now. Great work > > > > > there > > > > > > > > > > 2. What is keeping us from merging your ICU etc changes? please > > > > > note some subprojects are absolutely necessary for Lucene.NET to > > > > > run on Core CLR as well - even though they are "sub" projects > > > > > and not part of the Lucene > > > > core > > > > > itself. Queries and Analysis are probably the biggest ones > > > > > (latter not fully ported yet, and it's the one with the ICU dep). > > > > > > > > > > 3. Other than what you posted - are there any other pending > > > > > items to get this running on .NET Core? > > > > > > > > > > -- > > > > > > > > > > Itamar Syn-Hershko > > > > > > > > > > > > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fco > > > > de97 > > > > 2.com&data=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152df > > > > d451 > > > > ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=HxE > > > > 52%2 bFAC6FJAN%2fifQ5V8v27tZZALzBuDw2yhbw%2bkc8%3d > > > > > | @synhershko < > > > > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ft > > > > witt > > > > er.com%2fsynhershko&data=01%7c01%7cElizabeth.Maher%40microsoft.com > > > > %7c4 > > > > 43f03152dfd451ebb1c08d380b51eba%7c72f988bf86f141af91ab2d7cd011db47 > > > > %7c1 &sdata=7iqRqwQwlcNbmwkckUjcS1MWjUPJi3cxLFfRdLc3BnQ%3d > > > > > > > > > > > Freelance Developer & Consultant Lucene.NET committer and PMC > > > > > member > > > > > > > > > > On Fri, May 20, 2016 at 3:17 AM, Elizabeth Maher (NEWMAN) < > > > > > [email protected]> wrote: > > > > > > > > > > > Lucene.NET Community, > > > > > > > > > > > > Connie and I continue to make progress on moving Lucene.NET to > > > > > > .NET > > > > Core. > > > > > > The last challenge we face is to get the tests to successfully > > > > > > run on .NET Core. Currently the unit tests use NUnit. There > > > > > > is a .NET Core version called NUnitLite. However, the issue > > > > > > is there is some manual steps to get NUnitLite tests to work on > .NET Core. > > > > > > We would have to create console applications for each of the > > > > > > test library, call the appropriate APIs to create a test > > > > > > results file and then work with the CI system to manually upload > the results. > > > > > > Also, there is not Test Runner integration for NUnitLite. > > > > > > While, this is all possible to do, I would like to propose a > > > > > > different change. I could easily move all the test to use > > > > > > xUnit. The would benefit Lucene.NET in the long run as there > > > > > > are both full fx< > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f > > > > > > %2fx > > > > > > unit > > > > > > .github.io%2fdocs%2fgetting-started-desktop.html&data=01%7c01% > > > > > > 7cEl > > > > > > izab > > > > > > eth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51eba%7 > > > > > > c72f > > > > > > 988b > > > > > > f86f141af91ab2d7cd011db47%7c1&sdata=2orwADQfb%2fkpCykXkySekBp0 > > > > > > Dxan > > > > > > w0WR > > > > > > rUupy%2fzRFTY%3d> and core fx< > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f > > > > > > %2fx > > > > > > unit > > > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html&data=01%7 > > > > > > c01% > > > > > > 7cEl > > > > > > izabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d380b51e > > > > > > ba%7 > > > > > > c72f > > > > > > 988bf86f141af91ab2d7cd011db47%7c1&sdata=UGKx5nz6%2foaV2vt2LQDT > > > > > > qkpn > > > > > > 0g%2 bu2XXHzE4pDZ0K%2fhU%3d> versions of xUnit, there is a > > > > > > TeamCity plugin that automatically works< > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f > > > > > > %2fx > > > > > > unit > > > > > > .github.io%2fdocs%2fgetting-test-results-in-teamcity.html&data > > > > > > =01% > > > > > > 7c01 > > > > > > %7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451ebb1c08d38 > > > > > > 0b51 > > > > > > eba% > > > > > > 7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=an0p3MTPFYe24wQiP > > > > > > q3Ss F3Eb v3gwJwrr88TjoQ4DnM%3d>, as well as the ability to > > > > > > run tests in Visual Studio.< > > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f > > > > > > %2fx > > > > > > unit > > > > > > .github.io%2fdocs%2fgetting-started-dotnet-core.html%23run-tes > > > > > > ts-v > > > > > > s&da > > > > > > ta=01%7c01%7cElizabeth.Maher%40microsoft.com%7c443f03152dfd451 > > > > > > ebb1 > > > > > > c08d > > > > > > 380b51eba%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=BN%2f3X > > > > > > EmrU yKQu 8tdZtoghxSskpFBPP1N1YGxT7e45Vc%3d.> > > > > > > I believe the long-term maintenance cost for both full and > > > > > > core fx tests would be lower using xUnit. Is this an > > > > > > acceptable direction for me to take the unit tests? Please let > me know. > > > > > > > > > > > > Below is the progress of our migration work, for the curious. > > > > > > > > > > > > 1. Remove use of SharpZipLib dependency - completed > > > > > > > > > > > > 2. Update ICU4NET dependency for .NET Core - completed in > dev > > > > > branch > > > > > > > > > > > > a. In https://github.com/conniey/lucenenet/tree/move2dnx. > > > > > > > > > > > > 3. Remove Appache.NMS dependency - completed > > > > > > > > > > > > 4. Move Scheduler to use Tasks - PR submitted > > > > > > > > > > > > a. Pull request at > > > https://github.com/apache/lucenenet/pull/171 > > > > . > > > > > > > > > > > > 5. Make portable libraries to build .NET Core binaries - > > > > completed > > > > > > in dev branch > > > > > > > > > > > > a. In https://github.com/conniey/lucenenet/tree/move2dnx. > > > > > > > > > > > > 6. Verify tests pass on both full framework and .Net Core. > - > > In > > > > > > progress > > > > > > > > > > > > a. Need to get tests to run on .NET Core. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Elizabeth > > > > > > > > > > > > > > > > > > > > >
