Thanks Edward. Just the info is enough for now. It sounds like this will go away when the build gets revamped.
Regards, David On 24/10/15 13:12, Edward Ned Harvey (bouncycastle) wrote: >> From: David Hook [mailto:d...@autochthonous.org] >> >> I'm using 2015, Community Edition. Would you tell me which tests they >> are? I'll see if I can reproduce the problem, or at least work out what >> makes them different. > I didn't mean to imply this needs attention. I was just setting dennis's > expectations. If tests are failing for me, it doesn't matter, because I'm not > actually developing anything. If everything's working fine for you, this > doesn't necessarily need any attention. > > But since you ask, here are my details, step by step: > > I do a fresh checkout, then create a branch because I'm going to do the .sln > conversion. Then open the .sln and let it do the conversion. Close and reopen > VS. > > (Tangent) Is there any reason we need to keep the old .sln file around? I > think it's safe to say by now, that everyone has a modern IDE. Evidence: > You're using 2015 Community Edition. I'd actually really like to reconfigure > things to use a modern .sln file and NUnit so it can be built and tested on > both Visual Studio and Xamarin Studio. (End Tangent) > > Build all, in Debug mode. > Go to Test Explorer. > Click Run All > > After 11 minutes, Visual Studio mysteriously aborted. No error message or > anything. 474 tests passed, 267 Not Run. While it was running, the "Run All" > button had changed to "Cancel" but now after the mysterious abort, the > "Cancel" button changed back to "Run All" even though not all the tests had > been run yet. I don't know which test caused the mysterious abort. I made > some efforts to figure it out, and I did not successfully figure it out. > > Now I click "Run Not Run Tests" > Got a failure on PerformTest7. > 25 minutes later, all complete. The only failed test was PerformTest7. If I > run PerformTest7 again and again, it always fails consistently. > > Half an hour seems really excessively long, so I notice, 20 minutes are spent > on TestSumOfTwoMultiplies and TestMultiply. So I just comment out these two > tests: > crypto\test\src\math\ec\test\ECPointPerformanceTest.cs TestMultiply() > crypto\test\src\math\ec\test\ECAlgorithmsTest.cs TestSumOfTwoMultiplies() > > Now I clean solution, rebuild, and run all again. This time it takes 8 > minutes and there is one failure, TestCreationECDSA, and a mysterious abort, > with 2 tests still not run. So I Run Not Run Tests, in order to finish up > those last 2 tests. Now there are 2 failed tests: PerformTest7 and > TestCreationECDSA. If I run either of these tests again and again, they > always fails consistently. > > Clearly something is wrong, but it's not clear what's wrong. If you disable > one test, it shouldn't have any impact on the success of some other test. And > the mysterious abort is weird. > > I hope the 25 minute tests don't actually need to run for 25 minutes in order > to do their job. Notice it's a PerformanceTest. Does this really belong in > unit testing? > > Before anything, let me ask this: > > David, the tests in my environment have been broken a long time. I've been > ignoring them, assuming it doesn't matter if my tests work, because I'm not > doing any real development. If these are real problems that affect other > people too, I can put some more effort into figuring it out. I don't want to > put effort in if it's pointless. Basically just tell me if you see similar > problems, and if it's useful to work on it. > > Like I said, I'd really like to improve the .sln file and convert Micrsoft > unit tests to NUnit, and make it all work cross-platform. That would be > pretty cool. :-)
