We've figured out a way to do groups in a simple, flexible way, that allows
for probably everyone's use-cases. It requires a change to the spec and to
how runners work.
Here's an implementation similar to speclj: https://www.refheap.com/15744
And this is how clojure.test would be translated:
+1 to that!
It would be great if there was some kind of common infrastructure from a
tooling perspective, regardless of how the tests actually get written /
expressed.
If I may add one more thing into the mix, it would be great if we could
have some good way to integrate with Java testing
I don't fully understand how clojure.test's fixtures plays
into its ability to call test functions from within other
test functions.
They don't. Being able to call test functions within other
test functions is probably the least-designed and least-used
feature of clojure.test.
-S
--
--
You
There's been some discussion about how nesting should work. It should
probably be part of the spec, in a flexible and non-intrusive way. Then we
would consider a flat suite of tests to be just nested suite that's only a
1-level deep.
The two things we want nesting for are defining fixture-like
You pass not the Discoverer's results to the Runner, but the Discoverer
itself, which the Runner then invokes at need, possibly more than once.
On Tue, Jun 11, 2013 at 1:35 AM, Steven Degutis sbdegu...@gmail.com wrote:
Originally we had Runner split into Discoverer and Runner, but I had to
Beautiful!
On Tue, Jun 11, 2013 at 1:57 AM, Cedric Greevey cgree...@gmail.com wrote:
You pass not the Discoverer's results to the Runner, but the Discoverer
itself, which the Runner then invokes at need, possibly more than once.
On Tue, Jun 11, 2013 at 1:35 AM, Steven Degutis
Timothy, Brandon, Cedric, et al.:
Separating out the Discoverer from the Runner in the SPEC is a bad idea.
The main benefit mentioned so far for such a separation is so we can have
different definitions of what constitutes a test. For example,
test.generative can generate multiple tests from
You know, if some day all the test libs / frameworks can be run, test
results exploited, etc. through the same abstraction, this will
greatly ease the pain of tools developers !
I would better write once the integration between Eclipse way of
running and displaying tests, than once per library !
On Tuesday, June 11, 2013 12:39:59 AM UTC-4, Steven Degutis wrote:
It's pretty frustrating that I, a regular old Clojure user who likes
writing tests, can't mix and match tools from existing testing libraries.
Seriously, there's 4 major ones (clojure.test, speclj, midje, expectations)
and
I'd like to support Laurent on this and the Test2 effort, having a spec of
tests in terms of abstractions useable from library to library, possibly
having tests as data, (since I have learned that it's all about data) would
really be really useful. The ring-spec like idea for tests really has
Jay,
Imagine there is no Micah or Marick or you, there's only Bob and Alice, two
Clojure geeks who have a really big test suite.
If they wrote their whole test suite using clojure.test, they can't use an
autorunner unless they rewrite it in one of the other libs.
If they wrote their whole test
On Tuesday, June 11, 2013 11:11:23 AM UTC-4, Steven Degutis wrote:
Jay,
[elided]
That's the issue I'm trying to solve. Maybe that's not what everyone sees
in this. But this is the big win I see in it.
I think that's a good goal, I think you should stick to that, instead of
continuing to
You're right. I'm sorry for doing that. From now on I'll make a bigger
point of it to be more correct.
On Tue, Jun 11, 2013 at 10:18 AM, Jay Fields j...@jayfields.com wrote:
On Tuesday, June 11, 2013 11:11:23 AM UTC-4, Steven Degutis wrote:
Jay,
[elided]
That's the issue I'm trying to
Interesting; these arguments sound oddly like those surrounding Common
Lisp's loop macro. When reading Midje's docs, I got the weird impression
Brian was aware of the history of non-lispy macros.
Taken straight from
http://www.gigamonkeys.com/book/loop-for-black-belts.html :
(loop repeat 5
One of the goals is to create extensions that mimic each existing test-lib,
so that migrating to test2 just means changing your :requires.
But there are some tricky spots:
1. I don't fully understand how clojure.test's fixtures plays into its
ability to call test functions from within other test
2 cents, I don't see the value in mixing and matching lower-level
constructs between test frameworks, which also sounds hard. I see a lot of
value in what the SPEC provides, standardized test-reporting, metadata and
runner infra. This makes tooling's life easier, and would ease the burden
of
Good idea. Thanks guys, I'll do that.
On Tue, Jun 11, 2013 at 3:18 PM, Brandon Bloom brandon.d.bl...@gmail.comwrote:
Maybe it makes sense to separate out the 'common testing interop' effort
from the 'another test framework' effort, so it can can get off the
ground?
I agree with this.
Maybe it makes sense to separate out the 'common testing interop' effort
from the 'another test framework' effort, so it can can get off the ground?
I agree with this. It's easier to solve fewer problems at once. Maybe
you should reduce the scope to just the common result reporting
schema? At
That was what I was suggesting the other day... I see more value in
providing a standardizing test result format and better reporting
tools / integration with IDEs etc than in YATF (Yet Another testing
Framework).
On Tue, Jun 11, 2013 at 1:18 PM, Brandon Bloom
brandon.d.bl...@gmail.com wrote:
As of this moment, it's usable for writing test suites.
The readme is a lot cleaner now: https://github.com/evanescence/test2
The spec is a lot shoerter now:
https://github.com/evanescence/test2/blob/master/SPEC.md
There's a working auto-runner extension:
Thanks for that link Stu. It's quite a lot to take in but it's very useful.
As for *assertion-results*, it seems this has to be in the implementation,
because both Asserters and Runners need a concrete but detached way to
communicate.
Right now they both point to
On 11 June 2013 14:42, Steven Degutis sbdegu...@gmail.com wrote:
Timothy, Brandon, Cedric, et al.:
Separating out the Discoverer from the Runner in the SPEC is a bad idea.
The main benefit mentioned so far for such a separation is so we can have
different definitions of what constitutes a
Ah, you're right. Thanks for helping me understand.
Now, the runner and discoverer logic are separated.
Here's the function that calls the runner, passing in a finder:
https://github.com/evanescence/test2/blob/master/src/test2/run.clj
Here's the default runner, and how it uses the finder:
On 12 June 2013 00:08, Steven Degutis sbdegu...@gmail.com wrote:
Right now they both point to test2.api.asserters/*assertion-results*. But
I'd really like to get rid of the concreteness of this if possible. But even
throwing exceptions to indicate assertion results won't do, because the
On Jun 11, 2013, at 10:02 AM, JeremyS jschoffen@gmail.com wrote:
Now, opinion time (more opinionated actually). Midje might be full of macros
and abstractions that I don't understand when I read the code. Hell it's full
of them. But I honestly never had to read it to write my tests.
On Jun 11, 2013, at 2:08 PM, Tj Gabbour t...@pentaside.org wrote:
Interesting; these arguments sound oddly like those surrounding Common Lisp's
loop macro. When reading Midje's docs, I got the weird impression Brian was
aware of the history of non-lispy macros.
I was a Common Lisp user and
I sense a misunderstanding of transients in this message.
Am Sonntag, 9. Juni 2013 18:51:00 UTC+2 schrieb Steven Degutis:
Thanks for the feedback. I'l look into #1. Regarding #2, we just wanted a
side-effecty (mutable) way of adding assertion-results within a test. I
suppose I could use
On Jun 10, 2013, at 12:22 AM, julianrz julia...@yahoo.com wrote:
This may be a little off topic, but does this, or any other framework, solve
some testing inconveniences that exist in Clojure and probably other
functional languages:
1) testing recursive functions. I want to test what a
1) testing recursive functions. I want to test what a recursion STEP
does, not the whole function. Can I mock 'recur'?
You shouldn't need to, pull the body of the loop out as as separate
function, then test that function.
2) testing sequence, esp. lazy
For this, I often create a generator
On Monday, June 10, 2013 9:20:31 AM UTC-5, tbc++ wrote:
1) testing recursive functions. I want to test what a recursion STEP
does, not the whole function. Can I mock 'recur'?
You shouldn't need to, pull the body of the loop out as as separate
function, then test that function.
2)
You can certainly use with-redefs with any testing library/framework, or
you can pass dependencies into your production-code functions. I think
perhaps I'm missing the detail you're seeing on how using a particular test
framework encourages using global vars - can you elaborate?
You're right, my
Hi guys,
This might not be a good idea but if the goal is to have a minimal testing
utilities (or testing abstractions might I say ?) do you plan
to port them to ClojureScript ? The idea of a minimal testing utility that
would be extensible to suit everyone's taste and style of testing would
On Jun 10, 2013, at 9:20 AM, Timothy Baldridge tbaldri...@gmail.com wrote:
Midje on the other hand, is a massive pile of macros and DSLs that so
complicate your code that advanced tests are insanely hard to debug. ... And
I can't tell you how many dozens of hours I've lost trying to figure
There are currently 4 roles defined: Definer, Asserter, Runner, and
Reporter.
It looks like the Runner does finding, filtering, and execution. I think
you could further break the Runner down into Discoverer and Executor. I
might want to just ask What tests do I have? without actually running
It might surprise you to know that there are actual human beings with
feelings who write the software you slam.
You are right. And I apologize for my strong words. For the goals that
Midje strives for, it is an excellent library.
My reaction is more against the ideas behind Midje (from the
It's pretty frustrating that I, a regular old Clojure user who likes
writing tests, can't mix and match tools from existing testing libraries.
Seriously, there's 4 major ones (clojure.test, speclj, midje, expectations)
and they each do mostly the same things but vary slightly in some areas. I
Originally we had Runner split into Discoverer and Runner, but I had to
combine them both in Runner again so that we can have an autorunner.
Imagine that you've started your autorunner at the command line, and you
create a new test in your existing file and save it. The discoverer has
already
Hi Steven,
A few thoughts:
1. You may want to look at
https://github.com/clojure/test.generative/blob/master/data-model.org.
2. I don't think you want a ref for *assertion-results* -- I am not aware
of any use cases that would need transactions. In any case the choice of
reference type probably
I'd like to mention that expectations* has 0 open pull requests, 0 open
issues, and is very actively maintained**. Steven, I don't want to
discourage you from creating your own testing framework, I think everyone
should, it's a very educational experience.
I just wanted to be clear that no one
Thanks for the feedback. I'l look into #1. Regarding #2, we just wanted a
side-effecty (mutable) way of adding assertion-results within a test. I
suppose I could use (trans []) and let users use conj! although the fact
that transient keeps saying alpha, use at your own risk concerns me a bit.
Jay (and others),
First of all, you must understand where test2 came from. It started as a
bunch of people** in #clojure discussing what we'd change about
clojure.test if we could.
We realized we can't change clojure.test because (1) this would break
backwards compatibility, and (2) clojure.test
There's a few things that I know the SPEC needs to address.
- Pending tests
- Some flexible concept of around-all and around-each that allow separating
Definers from Runners (currently, to implement clojure.test's
once-fixtures, it requires a custom Definer *and* a custom Runner)
- Whether
On Jun 9, 2013, at 1:07 PM, Steven Degutis sbdegu...@gmail.com wrote:
I think we all agree that it's extremely important to discuss the SPEC as a
community. In fact, since this is a pre-ANN, let's consider this thread the
perfect place for such a discussion.
I suggest that surveying users
On Sun, Jun 9, 2013 at 11:07 AM, Steven Degutis sbdegu...@gmail.com wrote:
We realized we can't change clojure.test because (1) this would break
backwards compatibility, and (2) clojure.test is really slow-moving since it
lives inside Clojure.
Are there any JIRA tickets open against
FWIW, about the only thing about clojure.test that I miss occasionally
when using Expectations is 'each' fixtures for a subset of tests but
the work involved in wrapping an expression in a try/finally with the
resource setup and tear down I need is usually so minimal that's it's
not even worth
Are there any JIRA tickets open against clojure.test? That would seem
to be a good place to start.
Good idea, that helps for gathering data about use-cases.
If someone (with a signed CA on file) wants to step up and maintain
clojure.test, even tho' it's part of core Clojure right
On Sun, Jun 9, 2013 at 5:50 PM, Steven Degutis sbdegu...@gmail.com wrote:
Changing clojure.test seems like the wrong way to go. Being attached to a CA
makes it hard to contribute to.
It's a one-off action. Sign it, send it in. Then you can contribute to
Clojure or any of its contrib libraries
On Sunday, June 9, 2013 8:50:46 PM UTC-4, Steven Degutis wrote:
But that's what I meant, that he's proposing we start with his lib and add
extensibility in the places we want it. So my response to that still
applies.
That's not at all what I said, proposed, alluded to, or anything of the
Is this still current? http://clojure.github.io/clojure-contrib/
On Sunday, June 9, 2013 8:19:15 PM UTC-5, Sean Corfield wrote:
On Sun, Jun 9, 2013 at 5:50 PM, Steven Degutis
sbde...@gmail.comjavascript:
wrote:
Changing clojure.test seems like the wrong way to go. Being attached to
a
No. Read http://dev.clojure.org/display/design/Where+Did+Clojure.Contrib+Go
On Sun, Jun 9, 2013 at 6:41 PM, Steven Degutis sbdegu...@gmail.com wrote:
Is this still current? http://clojure.github.io/clojure-contrib/
On Sunday, June 9, 2013 8:19:15 PM UTC-5, Sean Corfield wrote:
On Sun, Jun 9,
Then I apologize. I must have conflated what you said with what someone
else said. My mistake.
On Sunday, June 9, 2013 8:38:22 PM UTC-5, Jay Fields wrote:
On Sunday, June 9, 2013 8:50:46 PM UTC-4, Steven Degutis wrote:
But that's what I meant, that he's proposing we start with his lib and
This may be a little off topic, but does this, or any other framework,
solve some testing inconveniences that exist in Clojure and probably other
functional languages:
1) testing recursive functions. I want to test what a recursion STEP does,
not the whole function. Can I mock 'recur'?
2)
Ooh, okay a little off topic here, but Expectations looks fantastic. Thanks
a lot Jay!
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated -
Test2 is a new testing lib for Clojure, where the power is its simplicity,
extensibility, and a
SPEChttps://github.com/evanescence/test2/blob/master/SPEC.md much
like Ring's.
Github: https://github.com/evanescence/test2
Some background: It came out of
54 matches
Mail list logo
