rclabo commented on issue #406: URL: https://github.com/apache/lucenenet/issues/406#issuecomment-771772631
Thanks @NightOwl888, great info. I've done many hours of research on this and here is what I have found. I'd really value your thoughts. ## Console.WriteLine IS going somewhere Almost by accident, I discovered that at least for failing tests the output for `Console.WriteLine` does actually go someplace useful. When looking at the failing test, notice the error message column in the screenshot below  At first glance it seems like it's just telling that there was a certain type of exception. BUT if you right click that failing test and click "Copy Details" it will place the WHOLE message on the clipboard. By pasting that in some editor, I came to realize that the exception message is crazy long, and in fact does contain the `Console.WriteLine` output:  While this does not send the `Console.WriteLine` to the output window, it does make the output available when investigating a failing test which is probably one of it's most likely use cases. ## Other API Options I have tried all of theses: ``` TestContext.Progress.WriteLine TestContext.Out.WriteLine TestContext.WriteLine Console.WriteLine Console.Error.WriteLine ``` In the version of NUnit that Lucene.Net is using, none of these go to the output window. Although, `System.Diagnostics.Debug.WriteLine` can be routed in VS to the immediate window, which is not a totally bad option BUT it does not get included in the exception message which is an important consideration. ## Parallel Unit tests One of the reason that this whole issue cropped up in NUnit3 which did not exist in NUnit2 is that in NUnit3 there was a concern that when running unit tests in parallel, the output from something like `Console.WriteLine` was a bit problematic given that the output from several different tests running at the same time would be intermixed. One could certainly see how that could be a problem. This is why NUnit introduced the methods on the `TextContext` class for writing output in version 3. The idea was to have some methods on that class write immediately (somewhere?!) and to have some be buffered at the unit test level and only output with the test results when that unit test completes. This in theory sounds pretty smart to me, as it gets away from the whole problem of having output intermixed for multiple tests but still provides the ability to theoretically output in real time if desired. In practice, the actual implementation seems wonky, however. Here is what my tests show based on examining the exception message content, and remember I can get none of these to go to the output window (or intermediate window) currently: `TestContext.Progress.WriteLine` - I can't find this output anywhere, not in the exception message, not in the output window or immediate window. I don't know where it goes... crazy! `TestContext.Out.WriteLine` - writes immediately even though the [docs](https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html ) say it's buffered. `TestContext.WriteLine` - writes immediately. `Console.WriteLine` - writes immediately. `Console.Error.WriteLine` - buffered till end of unit test then written. ## In NUnit 3.17 it's now possible to send Console.WriteLine to Output Window Apparently this is a recognized NUnit3 issue, [issue 343](https://github.com/nunit/nunit3-vs-adapter/issues/343). The issue was first placed in June 2017 and got plenty of community banter. The issue was resolved with the release of NUnit3TestAdapter ver 3.17 which is the current production release of NUnit. This is to say that being able to have `Console.WriteLine` go to the output window was only recently solved by NUnit. Currently Lucene.Net uses NUnit3TestAdapter ver 3.16.1. ## Wrap-up and Call for Feedback So it seems tempting to recommend that we upgraded Lucene.net from NUnit3.16.1 to NUnit 3.17 but I know in a project of this size, and with the number of unit test it has, there may be hidden issues I'm unaware of. The good news is whether we upgrade NUnit or now, we now know how to see the output from `Console.WriteLine` in the exception message of failed tests. And frankly that's probably the time we most want to see the output. So maybe it's good enough to just know that's possible. As an aside we now have a way to make the random seed for the test visible when a test fails. We can just use `Console.WriteLine` to write the seed and it will show up in the exception message. That's cool to know. But we do still have to figure out if upgrading to NUnit 3.17 is worth while. Given that I don't personally use console messages that much in my own approach to debugging, I don't have a strong opinion either way, especially since I now know how to see the output in the exception messages. What are your thoughts? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
