Re: Ah, simple solution to unittests inside templates
On Sunday, 18 September 2016 at 12:16:54 UTC, Andrei Alexandrescu
wrote:
I don't see that as much of a hurdle seeing as any template
written has a few "obvious" types it'll work with. To
encapsulate that if needed:
struct Awesome(A, B, C)
{
private enum ut = is(A == int) && is(B == int) && is(C ==
int);
unittest { alias Awe = Awesome!(int, int, int); }
static if (ut) unittest
{
...
}
}
I like how the non-static unittest ensures that any instantiation
of Awesome triggers the static unittest. I've encapsulated this
pattern into a template:
https://github.com/ntrel/stuff/blob/master/testinstanceflag.d#L34
Awesome would then just need one line instead of two before the
'static if' line:
private enum ut = testInstanceFlag!(Awesome, int, int, int);
(testInstanceFlag currently only works with one argument though).
The nice thing is that people reading the source can look up the
docs for testInstanceFlag to learn about it.
Re: Ah, simple solution to unittests inside templates
On Mon, Sep 19, 2016 at 09:02:05AM -0700, H. S. Teoh via Digitalmars-d wrote: > On Sat, Sep 17, 2016 at 01:22:52PM -0400, Andrei Alexandrescu via > Digitalmars-d wrote: > > Recall the discussion a few days ago about unittests inside templates > > being instantiated with the template. Often that's desirable, but > > sometimes not - for example when you want to generate nice ddoc > > unittests and avoid bloating. For those cases, here's a simple > > solution that I don't think has been mentioned: > > I've already mentioned this exact idea recently in the d-learn forum. > Nobody responded then. [...] Correction: it was on this very same forum: http://forum.dlang.org/post/mailman.96.1472754654.2965.digitalmar...@puremagic.com T -- May you live all the days of your life. -- Jonathan Swift
Re: Ah, simple solution to unittests inside templates
On Sat, Sep 17, 2016 at 01:22:52PM -0400, Andrei Alexandrescu via Digitalmars-d
wrote:
> Recall the discussion a few days ago about unittests inside templates
> being instantiated with the template. Often that's desirable, but
> sometimes not - for example when you want to generate nice ddoc
> unittests and avoid bloating. For those cases, here's a simple
> solution that I don't think has been mentioned:
I've already mentioned this exact idea recently in the d-learn forum.
Nobody responded then.
> /** Awesome struct */
> struct Awesome(T)
> {
> /** Awesome function. */
> void awesome() {}
>
> ///
> static if (is(T == int)) unittest
> {
> Awesome awesome;
> awesome.awesome;
> }
> }
>
> The unittest documentation is nicely generated. The unittest code
> itself is only generated for one instantiation.
[...]
And you also have to make sure Awesome!int is actually instantiated,
otherwise the unittest won't actually run!
T
--
In theory, there is no difference between theory and practice.
Re: Ah, simple solution to unittests inside templates
On 9/18/16 8:14 AM, Andrei Alexandrescu wrote:
On 9/18/16 6:00 AM, Jonathan M Davis via Digitalmars-d wrote:
Yes. That's DIP 82:
http://wiki.dlang.org/DIP82
I need to go over it again and then introduce it into the new DIP
process.
But I really think that that's where we should go to fix this problem.
Just a thought: things that we can't do have high priority. Things that
we can do with a modest cost are much less attractive. Consider:
struct Awesome(A, B, C)
{
private enum ut = is(A == int) && is(B == int) && is(C == int);
static if (ut) unittest
{
...
}
}
You're looking at an overhead with a small fixed cost plus a few
characters ("if (ut)") per unittest.
This is exactly how I did RedBlackTree unit tests:
https://github.com/dlang/phobos/blob/master/std/container/rbtree.d#L748
https://github.com/dlang/phobos/blob/master/std/container/rbtree.d#L815
This is still less than ideal, and has caused some real problems over
the years. e.g.:
https://issues.dlang.org/show_bug.cgi?id=12246
https://issues.dlang.org/show_bug.cgi?id=14082
The best solution IMO is not to run templated unit tests unless
specifically requested. Perhaps only run them when instantiated inside a
unittest block?
All that being said, running unit tests on EVERY integral type caught
about 3 bugs in the compiler when I was creating dcollections, and found
several corner-case bugs in my code as well. It was well worth the
"overhead" IMO.
-Steve
Re: Ah, simple solution to unittests inside templates
On Sunday, September 18, 2016 20:11:56 Andrei Alexandrescu via Digitalmars-d wrote: > Understood. I don't want anyone to get furious later on account of my > sugarcoating things, so let me say this: I'm likely to oppose such a > proposal. Walter and I have similar design sensibilities so he's likely > to oppose it too. I don't think this is an important issue that should > be addressed. The ease of implementation argument is, as discussed > before, fallacious. What you call an ugly workaround I call business as > usual using the nice facilities of the D language. I suggest you work on > ideas with more impact. -- Andrei The ease of implementation argument is not fallacious when you're arguing that it shouldn't be done, because it's not the most critical thing on the list of things to be done. Certainly, we shouldn't add such a feature just because it's easy. It being easy to implement would just mean that the fact that there are other critical things taking up Walter's time doesn't mean that we shouldn't do it. The decision should be made on the merits of the feature. When it gets done is then affected by how easy it is and what else there is to do which is more critical. Regardless, your workaround doesn't even fully work around the problem, because the unittest block still ends up in user code - just less user code, because it's just the one instantiation that gets it. So, it's an improvement, but it's not a fix. That problem can be fixed by defining special version idefintiers for your code that you put the unittest blocks within templates inside of and that no one would define in their own projects, but then you end up with quite a bit of extra plumbing just so that you can have your unittest blocks next to your functions inside of templated types without them ending up in user code or being compiled and run multiple times when testing your own code. Having an attribute take care of this (be it static or something else) would be far cleaner and would make it far more likely that folks would actually do it, so you wouldn't end up with someone else's tests in your code, just because they had a templated type that you're using that was unit tested. But I don't think that there's much that I can do to convince you if you think that adding a bunch of extra version blocks and static ifs just so that you can have the functions inside of templated types have their unit tests next to them without impacting the code of users or resulting in the tests being compiled and run an excess number of times is not ugly. - Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On 09/18/2016 04:29 PM, Jonathan M Davis via Digitalmars-d wrote: On Sunday, September 18, 2016 08:14:47 Andrei Alexandrescu via Digitalmars-d wrote: On 9/18/16 6:00 AM, Jonathan M Davis via Digitalmars-d wrote: Yes. That's DIP 82: http://wiki.dlang.org/DIP82 I need to go over it again and then introduce it into the new DIP process. But I really think that that's where we should go to fix this problem. Just a thought: things that we can't do have high priority. Things that we can do with a modest cost are much less attractive. Consider: Is this the biggest issue we have? No. But that doesn't mean that it isn't important and that it shouldn't be addressed. It just means that it's not the highest priority and therefore probably not what happens next. But also consider that implementing this is probably straightforward enough that someone other than Walter could implement it, and it wouldn't necessarily have to take away from something more critical like DIP1000. Aside from approval, I wouldn't expect that this would require Walter. If it were approved by Walter sometime in the nearish future, then after that, it could be implemented by any of the compiler devs whenever they could fit it in, whether that's sooner or later. I'm fairly certain that in this case, the main problem is the approval and not the implementation. I fully intend to update the DIP for this and submit it for approval with the new process. If that needs to wait for some reason, then that's not the end of the world. But I definitely think that this is a problem that we should solve and not just write off because we have an ugly workaround. Understood. I don't want anyone to get furious later on account of my sugarcoating things, so let me say this: I'm likely to oppose such a proposal. Walter and I have similar design sensibilities so he's likely to oppose it too. I don't think this is an important issue that should be addressed. The ease of implementation argument is, as discussed before, fallacious. What you call an ugly workaround I call business as usual using the nice facilities of the D language. I suggest you work on ideas with more impact. -- Andrei
Re: Ah, simple solution to unittests inside templates
On Monday, September 19, 2016 01:55:58 Timon Gehr via Digitalmars-d wrote: > On 18.09.2016 22:10, Jonathan M Davis via Digitalmars-d wrote: > > In > > this case, a non-static unittest block would be compiled into each > > template > > intsantiation, whereas a static one would be compiled once per template > > and > > would not require that the template even be instantiated. > > Yes, but those are actually not "pretty much the same". > > The second feature does not exist for any kind of declaration, and it is > not 'static'. If it is introduced, why limit it to unittests? The entire point of DIP 82 is to make it so that you can lexically put the unittest block right after the symbol inside of the template so that it can be ddoc-ed and so that the test can live right next to what it's testing without getting compiled into the template. It also makes it so that you don't have to put unittest blocks outside of the template to ensure that the template actually gets instantiated for testing. What use case is there for anything else living inside a template but not actually being part of the template? For other declarations, you'd just declare them outside of the template. unittest blocks are a total oddball here, because they need to be right after the function to work with ddoc, and it's a maintenance problem for them not to be right next to what they're testing. I don't know of anything else with similar problems. - Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On 18.09.2016 22:52, Jonathan M Davis via Digitalmars-d wrote: On Sunday, September 18, 2016 13:10:36 Jonathan M Davis via Digitalmars-d wrote: On Sunday, September 18, 2016 08:02:47 Andrei Alexandrescu via Digitalmars-d wrote: On 9/17/16 5:23 PM, Andrej Mitrovic wrote: I think at some point someone suggested we could implement explicit support for such unittests via `static unittest`: That suggests the unittest shall be evaluated during compilation. -- Andrei How so? At this point, static as a keyword pretty much never means that something is compile-time specific. Actually, static does mean compile-time in the case of static assert, so there is at least once case where it does, but most uses of static mean something else, and you have to know the context to know what the static keyword means. I selected static, because this use case fit reasonably well with how it was used with constructors, and it didn't require a new keyword or attribute. Currently: - static in front of a statement is a declaration running the statement at compile time in an appropriate sense. - static in front of a declaration means that the declaration does not use a context pointer. static unittest does not fit this pattern. But the word static itself isn't the important part. It's the feature, and something else could be used. Yup.
Re: Ah, simple solution to unittests inside templates
On 18.09.2016 22:10, Jonathan M Davis via Digitalmars-d wrote: On Sunday, September 18, 2016 08:02:47 Andrei Alexandrescu via Digitalmars-d wrote: > On 9/17/16 5:23 PM, Andrej Mitrovic wrote: > > I think at some point someone suggested we could implement explicit > > support for such unittests via `static unittest`: > That suggests the unittest shall be evaluated during compilation. -- Andrei How so? At this point, static as a keyword pretty much never means that something is compile-time specific. This is using static in pretty much the same sense that static constructors do. No. A normal constructor goes with each instance of a class or struct whereas a static one goes with the type. Yes, this is consistent with all other usages of static declarations. In this case, a non-static unittest block would be compiled into each template intsantiation, whereas a static one would be compiled once per template and would not require that the template even be instantiated. Yes, but those are actually not "pretty much the same". The second feature does not exist for any kind of declaration, and it is not 'static'. If it is introduced, why limit it to unittests?
Re: Ah, simple solution to unittests inside templates
On Sunday, September 18, 2016 13:10:36 Jonathan M Davis via Digitalmars-d wrote: > On Sunday, September 18, 2016 08:02:47 Andrei Alexandrescu via Digitalmars-d > wrote: > > On 9/17/16 5:23 PM, Andrej Mitrovic wrote: > > > I think at some point someone suggested we could implement explicit > > > > > support for such unittests via `static unittest`: > > That suggests the unittest shall be evaluated during compilation. -- > > Andrei > > How so? At this point, static as a keyword pretty much never means that > something is compile-time specific. Actually, static does mean compile-time in the case of static assert, so there is at least once case where it does, but most uses of static mean something else, and you have to know the context to know what the static keyword means. I selected static, because this use case fit reasonably well with how it was used with constructors, and it didn't require a new keyword or attribute. But the word static itself isn't the important part. It's the feature, and something else could be used. static just seemed like a good fit. That can be discussed with the DIP though whenever it gets resubmitted and reviewed. - Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On Sunday, September 18, 2016 08:14:47 Andrei Alexandrescu via Digitalmars-d wrote: > On 9/18/16 6:00 AM, Jonathan M Davis via Digitalmars-d wrote: > > Yes. That's DIP 82: > > > > http://wiki.dlang.org/DIP82 > > > > I need to go over it again and then introduce it into the new DIP process. > > But I really think that that's where we should go to fix this problem. > > Just a thought: things that we can't do have high priority. Things that > we can do with a modest cost are much less attractive. Consider: Is this the biggest issue we have? No. But that doesn't mean that it isn't important and that it shouldn't be addressed. It just means that it's not the highest priority and therefore probably not what happens next. But also consider that implementing this is probably straightforward enough that someone other than Walter could implement it, and it wouldn't necessarily have to take away from something more critical like DIP1000. Aside from approval, I wouldn't expect that this would require Walter. If it were approved by Walter sometime in the nearish future, then after that, it could be implemented by any of the compiler devs whenever they could fit it in, whether that's sooner or later. I'm fairly certain that in this case, the main problem is the approval and not the implementation. I fully intend to update the DIP for this and submit it for approval with the new process. If that needs to wait for some reason, then that's not the end of the world. But I definitely think that this is a problem that we should solve and not just write off because we have an ugly workaround. - Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On Sunday, September 18, 2016 08:02:47 Andrei Alexandrescu via Digitalmars-d wrote: > On 9/17/16 5:23 PM, Andrej Mitrovic wrote: > > I think at some point someone suggested we could implement explicit > > support for such unittests via `static unittest`: > That suggests the unittest shall be evaluated during compilation. -- Andrei How so? At this point, static as a keyword pretty much never means that something is compile-time specific. This is using static in pretty much the same sense that static constructors do. A normal constructor goes with each instance of a class or struct whereas a static one goes with the type. In this case, a non-static unittest block would be compiled into each template intsantiation, whereas a static one would be compiled once per template and would not require that the template even be instantiated. - Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On Sunday, 18 September 2016 at 12:02:47 UTC, Andrei Alexandrescu wrote: That suggests the unittest shall be evaluated during compilation. -- Andrei Nah, static is already quite overloaded and needs context to understand what it does. For example module constructors. Anyway compiler issues aside it's probably best to use working solutions like yours for now, and later we can devise something better via a DIP (one was posted here). As the saying goes, don't let great be enemy of the good.
Re: Ah, simple solution to unittests inside templates
On 09/18/2016 09:01 AM, John Colvin wrote: What would be really good would be to have a way to make ddoc associate a unittest with a particular symbol, regardless of location. Though there are many ways in which we could be better, I really think we're good. See e.g. https://github.com/dlang/phobos/pull/4043 where I have to pull all the unittests out of the template in order to get the win 32 tester to pass (fails saying "too many symbols" otherwise), but that ruins the documentation. Would the Awesome thing work? Andrei
Re: Ah, simple solution to unittests inside templates
On Sunday, 18 September 2016 at 12:02:47 UTC, Andrei Alexandrescu wrote: On 9/17/16 5:23 PM, Andrej Mitrovic wrote: I think at some point someone suggested we could implement explicit support for such unittests via `static unittest`: That suggests the unittest shall be evaluated during compilation. -- Andrei static as in static function. Not much better imo. Your solution with static if is fine (ish) , but ugly and a pain for some templates that only accept relatively complicated types. Also requires an external unittest to be kept in sync to ensure that the template is ever instantiated with a those arguments. What would be really good would be to have a way to make ddoc associate a unittest with a particular symbol, regardless of location. See e.g. https://github.com/dlang/phobos/pull/4043 where I have to pull all the unittests out of the template in order to get the win 32 tester to pass (fails saying "too many symbols" otherwise), but that ruins the documentation.
Re: Ah, simple solution to unittests inside templates
On 9/18/16 6:23 AM, Nick Treleaven wrote:
Besides the other comments, we still have to instantiate Awesome!int
somewhere for the tests to run, which could be forgotten or improperly
done, failing silently. (Also int is arbitrary, unhelpful for the
uninitiated).
I don't see that as much of a hurdle seeing as any template written has
a few "obvious" types it'll work with. To encapsulate that if needed:
struct Awesome(A, B, C)
{
private enum ut = is(A == int) && is(B == int) && is(C == int);
unittest { alias Awe = Awesome!(int, int, int); }
static if (ut) unittest
{
...
}
}
Andrei
Re: Ah, simple solution to unittests inside templates
On 9/18/16 6:00 AM, Jonathan M Davis via Digitalmars-d wrote:
Yes. That's DIP 82:
http://wiki.dlang.org/DIP82
I need to go over it again and then introduce it into the new DIP process.
But I really think that that's where we should go to fix this problem.
Just a thought: things that we can't do have high priority. Things that
we can do with a modest cost are much less attractive. Consider:
struct Awesome(A, B, C)
{
private enum ut = is(A == int) && is(B == int) && is(C == int);
static if (ut) unittest
{
...
}
}
You're looking at an overhead with a small fixed cost plus a few
characters ("if (ut)") per unittest.
Andrei
Re: Ah, simple solution to unittests inside templates
On 9/17/16 5:23 PM, Andrej Mitrovic wrote: I think at some point someone suggested we could implement explicit support for such unittests via `static unittest`: That suggests the unittest shall be evaluated during compilation. -- Andrei
Re: Ah, simple solution to unittests inside templates
On Sunday, 18 September 2016 at 11:10:22 UTC, ZombineDev wrote: On Saturday, 17 September 2016 at 17:22:52 UTC, Andrei Alexandrescu wrote: [...] This solution is used extensively by ndslice [1] and I agree that it's quite flexible. [1]: http://forum.dlang.org/post/mailman.166.1472923003.2965.digitalmar...@puremagic.com#post-psrgjdlvsiukkuhrekoo:40forum.dlang.org I meant: http://forum.dlang.org/post/psrgjdlvsiukkuhre...@forum.dlang.org
Re: Ah, simple solution to unittests inside templates
On Saturday, 17 September 2016 at 17:22:52 UTC, Andrei
Alexandrescu wrote:
Recall the discussion a few days ago about unittests inside
templates being instantiated with the template. Often that's
desirable, but sometimes not - for example when you want to
generate nice ddoc unittests and avoid bloating. For those
cases, here's a simple solution that I don't think has been
mentioned:
/** Awesome struct */
struct Awesome(T)
{
/** Awesome function. */
void awesome() {}
///
static if (is(T == int)) unittest
{
Awesome awesome;
awesome.awesome;
}
}
The unittest documentation is nicely generated. The unittest
code itself is only generated for one instantiation.
Andrei
This solution is used extensively by ndslice [1] and I agree that
it's quite flexible.
[1]
http://forum.dlang.org/post/mailman.166.1472923003.2965.digitalmar...@puremagic.com#post-psrgjdlvsiukkuhrekoo:40forum.dlang.org
Re: Ah, simple solution to unittests inside templates
On Saturday, 17 September 2016 at 17:22:52 UTC, Andrei
Alexandrescu wrote:
///
static if (is(T == int)) unittest
{
Awesome awesome;
awesome.awesome;
}
}
The unittest documentation is nicely generated. The unittest
code itself is only generated for one instantiation.
Besides the other comments, we still have to instantiate
Awesome!int somewhere for the tests to run, which could be
forgotten or improperly done, failing silently. (Also int is
arbitrary, unhelpful for the uninitiated).
Re: Ah, simple solution to unittests inside templates
On Saturday, September 17, 2016 21:23:58 Andrej Mitrovic via Digitalmars-d
wrote:
> On Saturday, 17 September 2016 at 17:22:52 UTC, Andrei
>
> Alexandrescu wrote:
> > The unittest documentation is nicely generated. The unittest
> > code itself is only generated for one instantiation.
>
> I had a similar thought, but I didn't really like verbosity of
> the static if.
It also has the downside that the unit tests still end up in the code of
anyone using the template. It's now only when they use the template with
that particular instantiation, so it's definitely not as bad as just putting
the unittest block inside the templated type like you would if it weren't
templated, but it's still definitely worse than what you get with a
non-templated type.
> I think at some point someone suggested we could implement
> explicit support for such unittests via `static unittest`:
>
> /** Awesome struct */
> struct Awesome(T)
> {
> /** Awesome function. */
> void awesome() {}
>
> ///
> static unittest
> {
> Awesome!int awesome;
> awesome.awesome;
> }
> }
>
> A static unittest would not have implicit access to anything
> internal in the aggregation (this is different to your solution
> where the unittest has access to all fields of the Awesome!int
> instance).
>
> Note the difference inside the unittest: it uses `Awesome!int
> awesome` instead of the previous `Awesome awesome`. In your case
> it implicitly refers to the Awesome!int instantiation, however
> you're actually presenting uncompilable code to the user! The
> user might copy-paste it into a main function but it would fail
> to build.
>
> So the feature might have some merit as it would avoid generating
> docs with code which may not even work.
Yes. That's DIP 82:
http://wiki.dlang.org/DIP82
I need to go over it again and then introduce it into the new DIP process.
But I really think that that's where we should go to fix this problem. It
makes for a clean solution that allows you to easily choose whether a
unittest block inside of a template is treated as part of the template or
whether it's just syntactically inside the template so that it can be used
for documentation and so that it can be next to what it's testing, but it's
not actually part of the template semantically.
- Jonathan M Davis
Re: Ah, simple solution to unittests inside templates
On Saturday, 17 September 2016 at 17:22:52 UTC, Andrei
Alexandrescu wrote:
The unittest documentation is nicely generated. The unittest
code itself is only generated for one instantiation.
I had a similar thought, but I didn't really like verbosity of
the static if.
I think at some point someone suggested we could implement
explicit support for such unittests via `static unittest`:
/** Awesome struct */
struct Awesome(T)
{
/** Awesome function. */
void awesome() {}
///
static unittest
{
Awesome!int awesome;
awesome.awesome;
}
}
A static unittest would not have implicit access to anything
internal in the aggregation (this is different to your solution
where the unittest has access to all fields of the Awesome!int
instance).
Note the difference inside the unittest: it uses `Awesome!int
awesome` instead of the previous `Awesome awesome`. In your case
it implicitly refers to the Awesome!int instantiation, however
you're actually presenting uncompilable code to the user! The
user might copy-paste it into a main function but it would fail
to build.
So the feature might have some merit as it would avoid generating
docs with code which may not even work.
