Vincent (and/or anyone else willing to help out),

I have managed to get the projects upgraded to the new .csproj format and 
Visual Studio 2017, but there have been many challenges getting the tests to 
run all the way through without crashing, and I have finally managed to locate 
the tests that were causing the crashes (in Release mode, anyway) and manually 
fail them. This was a temporary fix to get by long enough to setup TeamCity so 
it works with the new configuration, which is now in place.

During my search for an answer as to why the test runner crashes, I discovered 
two causes:


1.       On .NET Standard 2.0, there is a bug that crashes the test runner when 
Debug.Assert evaluates to false. See: 
https://github.com/Microsoft/vstest/issues/1022

2.       Some of the concurrency tests have been converted literally from Java 
and in those tests there are either throw statements or NUnit asserts that are 
causing uncaught exceptions on background threads.

The second issue wasn't registering as a "problem" because of Microsoft's 
decision to make the default behavior in .NET 4.5 ignore uncaught exceptions 
(https://blogs.msdn.microsoft.com/pfxteam/2011/09/28/task-exception-handling-in-net-4-5/).
 Essentially, because in .NET 4.5 the ThrowUnobservedTaskExceptions setting is 
false on our test framework, these exceptions are being ignored. In Java, these 
would have been marshalled to the main thread causing a failure, meaning we are 
getting false positives. In .NET Core 2.0, these exceptions are now causing the 
foreground thread to crash (as one might expect), so we can now no longer 
ignore these issues.

However, there is another snag. We are getting more test failures with the new 
tooling than we did with the old tooling. And these tests do not fail in Visual 
Studio with the new tooling. Although ultimately I would like to get us onto 
the new tooling across .NET Framework and .NET Core, for the time being I have 
reverted the .NET Framework side back to using the NUnit 3 Console (as we had 
previously) and sure enough those additional test failures went away again. I 
suspect the tests (and perhaps even the software) have more concurrency bugs 
that are now being revealed by the new .NET Core 2.0 SDK tooling because of a 
different approach to concurrency.
See the comparison between 4.8.0-ci0000001123, 4.8.0-ci0000001126, 
4.8.0-ci0000001127 that had a lot of failures on the .NET Core 2.0 and 1.0.4 
SDKs (dotnet test), and 4.8.0-ci0000001134 & 4.8.0-ci0000001136 on NUnit 3 
Console: 
https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnNet451
 (note you can login as a guest to see these)

We are also seeing higher numbers of failures on .NET Core 1.0 and 2.0:

https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnDotNetCore10
https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnDotNetCore20

Anyway, back to the tests that I "fixed" to make the test runner not crash, 
which need some work:

https://github.com/apache/lucenenet/commit/40c42ced8127d61a9e066548ceb54d03868e54a0
https://github.com/apache/lucenenet/commit/62333ec568e3989e73dee65464e89d85c44acb82
https://github.com/apache/lucenenet/commit/b7ab123bed3ce084c1fa0e871f44045c2d7416af
https://github.com/apache/lucenenet/commit/fa94509d3f5b2daea6f2d812112b905e66a60595
https://github.com/apache/lucenenet/commit/60faa9cdcbc9b0e37a2472f5d3cc5af76c96c82a

The first three are the most concerning. I am not so worried about having to 
remove Debug.Assert to get them to work for the moment, as in production they 
won't be there anyway.

Also of concern are the "file locked by another process" errors we are now 
getting pretty much anywhere IndexWriter is used. I will need to confirm, but I 
don't believe we were getting these errors in the last release and these don't 
appear to be an issue with the tests. It would be preferable to have a fix 
rather than having to revert something in order to eliminate these errors.

I have added some instructions to the README 
(https://github.com/apache/lucenenet#visual-studio) on how to run tests and 
switch between target frameworks within Visual Studio 2017.

Do note that changing target frameworks doesn't go very smoothly and often 
requires a restart of Visual Studio 2017 (either the tests are not discovered, 
or there is a "build failed" error that doesn't show any corresponding errors 
in the list). I am hoping the tooling in Visual Studio 2017 will improve 
eventually and allow switching the framework to test on in the UI as well as 
perform better, but this is what we have for the time being.

Anyway, I would appreciate some assistance getting the test failures under 
control so we can finally release all of these new features, patches, and 
performance improvements that have been contributed by the team.

Thanks,
Shad Storhaug (NightOwl888)

P.S. Vincent - you will be pleased to know that the [Debuggable] attribute has 
now been removed and your advice of using 
[MethodImpl(MethodImplOptions.NoInlining)] worked great for the tests that 
require method names in the stack trace 
(https://github.com/apache/lucenenet/commit/4abbd4be4bf65c83562cac5870c802b6e31abf64).

Reply via email to