I think it will buy you less than you think, but it’s probably harmless.
I’ve got north of 20 years experience writing NetBeans’ code, and never
felt the need for one.
And I have to say, in my current gig I deal with a codebase with a lot of
existing mock based tests. They are mostly useless other than their one
purpose: To generate artificially high code coverage numbers management
likes to see, without having to do the work it takes to actually test that
the software does the right thing. It is trivial to write broken code that
all of the unit tests pass for. But the thing those tests DO test is call
sequences - what called what in what order. And they are incredibly fragile
to changes in that - you wind up with tests of the implementation details
of things, but not of whether or not they work.
So you have to run a bunch of harder to set up integration tests if you
actually want to know if you broke something. The unit tests are a lovely
Potemkin village that show fabulous “coverage” and tell you nothing about
whether the software works. They are useless.
You know what an integration tests really is? A unit test developers don’t
like running. Usually because it’s slow or needs databases or other
plumbing. That’s the actual bug, and the place that deserves investment -
making those things easy, so developers exercise as much of the code as
possible in every build. You can do that for most anything.
The one thing mocks shine for is if you depend on libraries that routinely
make incompatible changes - if the mocking library generates the
implementations, changes in things you don’t call don’t matter.
And that particular problem is one that NetBeans just doesn’t have - API
contracts are taken seriously.
So, do it if you’re excited about it, but consider whether you’ve exhausted
the paths to writing a real test instead. Or submitting a patch to the
thing you feel the need to mock to make what you want to do easier.
There are only two purpose tests serve:
- Keeping the investment made in code by not breaking it
- Giving developers rapid feedback when they do
Now, will investing in mocks actually improve either of those things? Or is
a better investment making it easier to test the thing you’re having
trouble with, so your tests run against the actual code they need to work
against in real life?
Coverage reports are a scourge. The dopamine hit of making that number go
up is too tempting, as is the urge to compete with everyone else’s numbers,
and you wind up writing tests of things like toString{} methods that are
used in logging if at all. Coverage > 80% is a strong indication that
developers are wasting time.
-Tim
On Sat, Oct 9, 2021 at 1:26 PM Karl-Philipp Richter <[email protected]>
wrote:
> Hi,
> I wanted to contribute some unit tests to the project. As far I
> understand there's no mocking framework in use currently looking at the
> test classes and answers to my recent "Getting Started" post.
>
> I think this is odd and a framework should be introduced quickly. I
> recommend Mockito. If you don't believe it's useful try writing a unit
> test which passes a reference to
> org.netbeans.modules.parsing.spi.indexing.Indexable to a unit under
> test. In under two minutes. This should take seconds.
>
> So far, I don't know how you measure coverage and what your requirements
> for minimal coverage are for new code and bugfixes is. High coverage
> isn't a guarantee for maintainable code and a high quality product,
> however low coverage in a safe indicator for the opposite. Having a
> mocking framework available and promoting it for new code in reviews
> will increase your development speed and reduce bugs severely.
>
> I'm opening a different thread from my "Getting Started" post since it's
> a more concrete issue.
>
> -Kalle
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
> --
http://timboudreau.com
