Re: Mocking framework
Well, yes, but it doesn’t invent an entire implementation for you that returns null for everything, including things that can’t be. Mock services is fine - because you still wind up writing mocks, you just use that to plumb them into the default lookup. Different species of thing. -Tim On Tue, Oct 12, 2021 at 2:58 AM Jaroslav Tulach wrote: > so 9. 10. 2021 v 19:27 odesílatel Karl-Philipp Richter > > napsal: > > > Hi, > > I wanted to contribute some unit tests to the project. As far I > > understand there's no mocking framework in use currently looking > > > > That's not entirely true. There is `MockServices` class: > > https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/MockServices.html > > It may not be as comfortable as Mockito & co., but it is enough to test > everything as NetBeans architecture is based on "injectable singletons" > pattern: > > http://wiki.apidesign.org/wiki/Injectable_Singleton > > -jt > -- http://timboudreau.com
Re: Mocking framework
so 9. 10. 2021 v 19:27 odesílatel Karl-Philipp Richter napsal: > Hi, > I wanted to contribute some unit tests to the project. As far I > understand there's no mocking framework in use currently looking > That's not entirely true. There is `MockServices` class: https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/MockServices.html It may not be as comfortable as Mockito & co., but it is enough to test everything as NetBeans architecture is based on "injectable singletons" pattern: http://wiki.apidesign.org/wiki/Injectable_Singleton -jt
Re: Mocking framework
On 10/11/21 07:43, Neil C Smith wrote: On Sun, 10 Oct 2021 at 03:33, Tim Boudreau wrote: So you have to run a bunch of harder to set up integration tests if you actually want to know if you broke something. The unit tests are a lovely Potemkin village that show fabulous “coverage” and tell you nothing about whether the software works. They are useless. Thanks, Tim, for the whole post, and particularly this point - spot on! I enjoyed that, too. "a lovely Potemkin village", heh. -- Glenn Holmer (Linux registered user #16682) "After the vintage season came the aftermath -- and Cenbe."null - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Mocking framework
On Sun, 10 Oct 2021 at 03:33, Tim Boudreau wrote: > So you have to run a bunch of harder to set up integration tests if you > actually want to know if you broke something. The unit tests are a lovely > Potemkin village that show fabulous “coverage” and tell you nothing about > whether the software works. They are useless. Thanks, Tim, for the whole post, and particularly this point - spot on! Neil - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Mocking framework
I think it will buy you less than you think, but it’s probably harmless.
I’ve got north of 20 years experience writing NetBeans’ code, and never
felt the need for one.
And I have to say, in my current gig I deal with a codebase with a lot of
existing mock based tests. They are mostly useless other than their one
purpose: To generate artificially high code coverage numbers management
likes to see, without having to do the work it takes to actually test that
the software does the right thing. It is trivial to write broken code that
all of the unit tests pass for. But the thing those tests DO test is call
sequences - what called what in what order. And they are incredibly fragile
to changes in that - you wind up with tests of the implementation details
of things, but not of whether or not they work.
So you have to run a bunch of harder to set up integration tests if you
actually want to know if you broke something. The unit tests are a lovely
Potemkin village that show fabulous “coverage” and tell you nothing about
whether the software works. They are useless.
You know what an integration tests really is? A unit test developers don’t
like running. Usually because it’s slow or needs databases or other
plumbing. That’s the actual bug, and the place that deserves investment -
making those things easy, so developers exercise as much of the code as
possible in every build. You can do that for most anything.
The one thing mocks shine for is if you depend on libraries that routinely
make incompatible changes - if the mocking library generates the
implementations, changes in things you don’t call don’t matter.
And that particular problem is one that NetBeans just doesn’t have - API
contracts are taken seriously.
So, do it if you’re excited about it, but consider whether you’ve exhausted
the paths to writing a real test instead. Or submitting a patch to the
thing you feel the need to mock to make what you want to do easier.
There are only two purpose tests serve:
- Keeping the investment made in code by not breaking it
- Giving developers rapid feedback when they do
Now, will investing in mocks actually improve either of those things? Or is
a better investment making it easier to test the thing you’re having
trouble with, so your tests run against the actual code they need to work
against in real life?
Coverage reports are a scourge. The dopamine hit of making that number go
up is too tempting, as is the urge to compete with everyone else’s numbers,
and you wind up writing tests of things like toString{} methods that are
used in logging if at all. Coverage > 80% is a strong indication that
developers are wasting time.
-Tim
On Sat, Oct 9, 2021 at 1:26 PM Karl-Philipp Richter
wrote:
> Hi,
> I wanted to contribute some unit tests to the project. As far I
> understand there's no mocking framework in use currently looking at the
> test classes and answers to my recent "Getting Started" post.
>
> I think this is odd and a framework should be introduced quickly. I
> recommend Mockito. If you don't believe it's useful try writing a unit
> test which passes a reference to
> org.netbeans.modules.parsing.spi.indexing.Indexable to a unit under
> test. In under two minutes. This should take seconds.
>
> So far, I don't know how you measure coverage and what your requirements
> for minimal coverage are for new code and bugfixes is. High coverage
> isn't a guarantee for maintainable code and a high quality product,
> however low coverage in a safe indicator for the opposite. Having a
> mocking framework available and promoting it for new code in reviews
> will increase your development speed and reduce bugs severely.
>
> I'm opening a different thread from my "Getting Started" post since it's
> a more concrete issue.
>
> -Kalle
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org
> For additional commands, e-mail: dev-h...@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
> --
http://timboudreau.com
Re: Mocking framework
On Sat, Oct 9, 2021 at 7:26 PM Karl-Philipp Richter
wrote:
> Hi,
> I wanted to contribute some unit tests to the project. As far I
> understand there's no mocking framework in use currently looking at the
> test classes and answers to my recent "Getting Started" post.
>
> I think this is odd and a framework should be introduced quickly. I
> recommend Mockito. If you don't believe it's useful try writing a unit
> test which passes a reference to
> org.netbeans.modules.parsing.spi.indexing.Indexable to a unit under
> test. In under two minutes. This should take seconds.
>
I guess something like this could work:
org.netbeans.modules.parsing.impl.indexing.SPIAccessor.getInstance().create();
e.g.:
org.netbeans.modules.parsing.impl.indexing.SPIAccessor.getInstance().create(new
IndexableImpl() {...});
or possibly more useful in many cases:
org.netbeans.modules.parsing.impl.indexing.SPIAccessor.getInstance().create(new
FileObjectIndexable(, ));
I have nothing against mocking libraries, but I wonder what exactly is the
test for which an instance of Indexable is created. There are some tests
that use the SPIAccessor...create(new FileObjectIndexable(...)) path[1],
but the setup they do is much more complex than just creating an instance
of Indexable.
Thanks,
Jan
[1]
https://github.com/apache/netbeans/blob/f2ca686da8852fa06b3eb7521cf1ef727e4182c7/java/java.source.base/test/unit/src/org/netbeans/modules/java/source/indexing/CompileWorkerTestBase.java#L214
> So far, I don't know how you measure coverage and what your requirements
> for minimal coverage are for new code and bugfixes is. High coverage
> isn't a guarantee for maintainable code and a high quality product,
> however low coverage in a safe indicator for the opposite. Having a
> mocking framework available and promoting it for new code in reviews
> will increase your development speed and reduce bugs severely.
>
> I'm opening a different thread from my "Getting Started" post since it's
> a more concrete issue.
>
> -Kalle
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org
> For additional commands, e-mail: dev-h...@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>
Re: Mocking framework
You are welcome to introduce any framework you like. On the other hand, I would pause to consider that NetBeans has existed since around 1995 without such a framework — introducing this may be a lot of work and might be less beneficial than might br assumed. Also note Mockito would have to be integrated into an Ant-based build system. Gj On Sat, 9 Oct 2021 at 19:26, Karl-Philipp Richter wrote: > Hi, > I wanted to contribute some unit tests to the project. As far I > understand there's no mocking framework in use currently looking at the > test classes and answers to my recent "Getting Started" post. > > I think this is odd and a framework should be introduced quickly. I > recommend Mockito. If you don't believe it's useful try writing a unit > test which passes a reference to > org.netbeans.modules.parsing.spi.indexing.Indexable to a unit under > test. In under two minutes. This should take seconds. > > So far, I don't know how you measure coverage and what your requirements > for minimal coverage are for new code and bugfixes is. High coverage > isn't a guarantee for maintainable code and a high quality product, > however low coverage in a safe indicator for the opposite. Having a > mocking framework available and promoting it for new code in reviews > will increase your development speed and reduce bugs severely. > > I'm opening a different thread from my "Getting Started" post since it's > a more concrete issue. > > -Kalle > > - > To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org > For additional commands, e-mail: dev-h...@netbeans.apache.org > > For further information about the NetBeans mailing lists, visit: > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists > > > >
Mocking framework
Hi, I wanted to contribute some unit tests to the project. As far I understand there's no mocking framework in use currently looking at the test classes and answers to my recent "Getting Started" post. I think this is odd and a framework should be introduced quickly. I recommend Mockito. If you don't believe it's useful try writing a unit test which passes a reference to org.netbeans.modules.parsing.spi.indexing.Indexable to a unit under test. In under two minutes. This should take seconds. So far, I don't know how you measure coverage and what your requirements for minimal coverage are for new code and bugfixes is. High coverage isn't a guarantee for maintainable code and a high quality product, however low coverage in a safe indicator for the opposite. Having a mocking framework available and promoting it for new code in reviews will increase your development speed and reduce bugs severely. I'm opening a different thread from my "Getting Started" post since it's a more concrete issue. -Kalle - To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
