Re: test.check slow shrinking
Short answer: Use a string generator that is much more likely to have collisions, and thus provoke your failure. Here's an example: (def small-strings (gen/sized (fn [s] (gen/resize (min s 2) (gen/not-empty gen/string-ascii) Longer answer: When using gen/bind, you create a nested shrink tree. When a test fails, and the shrinking process begins, the 'outer' generator is shrunk first. In your case, this means creating smaller matrices. After the smaller matrix is created, the inner generated is called again to create new random values with this smaller matrix. Since your test is unlikely to fail, it's quite likely that this shrunk test succeeds, and smaller matrices are not considered anymore. Since shrinking could not successfully shrink the outer generator, it begins to shrink the 'inner' generator. This is why you end up with a failing test case with a large matrix. Reid On Thursday, November 13, 2014 2:29:34 AM UTC-6, Lucas Bradstreet wrote: My thought process with that suggestion is that a shrink of the matrix elements (not the size), will not require the samples and probes to be regenerated. At some point I'll probably have to look into how the shrinking works. On 13 Nov 2014, at 06:18, Brian Craft craft...@gmail.com javascript: wrote: I tried your idea of generating the size first, then passing it to the matrix vector generators. This does seem to work better. The shrunk cases that return are actually worse, but so far it hasn't wedged itself, which is a great improvement. They all return within a few seconds. I don't yet understand the shrinking well enough to know why it's better. Example shrunk state: :samples [0 0 0 0 0 0 0 0 0], :probes [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] (def gen-matrix-size (gen/tuple gen/s-pos-int gen/s-pos-int)) (def gen-mostly-ints (gen/frequency [[9 gen/int] [1 (gen/return Double/NaN)]])) (defn gen-matrix2 [x y] (gen/vector (gen/vector gen-mostly-ints x) y)) (defn gen-names [n] (gen/vector (gen/such-that not-empty gen/string-alpha-numeric) n)) (def gen-tsv2 (gen/bind gen-matrix-size (fn [[x y]] (gen/hash-map :probes (gen-names x) :samples (gen-names y) :matrix (gen-matrix2 x y) On Wednesday, November 12, 2014 12:47:37 PM UTC-8, Lucas Bradstreet wrote: I'm pretty sure I did encounter the performance problem you're talking about, but I killed it and re-ran until I hit cases that shrink quickly. I'm afraid I'm not much help with those, although I agree that the bad shrinking is probably related to the performance issues. On 13 Nov 2014, at 04:22, Brian Craft craft...@gmail.com wrote: Interesting that you don't see a performance problem. What version did you try? I'm using 0.5.9. I just re-ran this example to make sure I wasn't imagining it. On the 11th run, it wedged, not returning, and burning a lot of cpu, presumably trying to shrink. It's a larger problem with the real test cases. One test ran for about eight hours yesterday before I killed it. Hard to tell what it's doing, and not sure what sort of times I should expect for shrinking larger test cases. On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote: I've also had some tricky shrinking type issues with recursive generators using bind. I had a play with your generators, using such-that to reduce the row/column name length and also preventing some generator shrinking by using no-shrink, but I didn't have much luck improving the resulting shrinks (though the shrinks did finish in a timely fashion). You might have more luck if you generated an m x n matrix size independently of the matrix itself and feed the size into the matrix generator and the samples and probes generators. This may allow the matrix size to be shrunk down more easily, while also shrinking the samples and probes vectors without them having to be regenerated for ever new matrix shrink (i.e. where the matrix size is maintained but the gen/int values within it are shrunk). However, this is all guess work and conjecture, as I don't understand the shrinking algorithm well enough, and I haven't tried my suggestion. Also, modifying your generators in this way may make your model less general if you wish to use it in other ways later. Cheers Lucas On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com wrote: Using test.check, I'm finding the shrinking to be very, very slow. Running a hundred cases takes a few seconds, unless it hits an error, in which case it takes 40-60 minutes to shrink, and the shrinking is not very effective (resulting test case is much larger than necessary). Sometimes the shrinking is much faster. It behaves a bit like it's occasionally getting into a pathological state, or a difficult shrinking
Re: test.check slow shrinking
My thought process with that suggestion is that a shrink of the matrix elements (not the size), will not require the samples and probes to be regenerated. At some point I'll probably have to look into how the shrinking works. On 13 Nov 2014, at 06:18, Brian Craft craft.br...@gmail.com wrote: I tried your idea of generating the size first, then passing it to the matrix vector generators. This does seem to work better. The shrunk cases that return are actually worse, but so far it hasn't wedged itself, which is a great improvement. They all return within a few seconds. I don't yet understand the shrinking well enough to know why it's better. Example shrunk state: :samples [0 0 0 0 0 0 0 0 0], :probes [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] (def gen-matrix-size (gen/tuple gen/s-pos-int gen/s-pos-int)) (def gen-mostly-ints (gen/frequency [[9 gen/int] [1 (gen/return Double/NaN)]])) (defn gen-matrix2 [x y] (gen/vector (gen/vector gen-mostly-ints x) y)) (defn gen-names [n] (gen/vector (gen/such-that not-empty gen/string-alpha-numeric) n)) (def gen-tsv2 (gen/bind gen-matrix-size (fn [[x y]] (gen/hash-map :probes (gen-names x) :samples (gen-names y) :matrix (gen-matrix2 x y) On Wednesday, November 12, 2014 12:47:37 PM UTC-8, Lucas Bradstreet wrote: I'm pretty sure I did encounter the performance problem you're talking about, but I killed it and re-ran until I hit cases that shrink quickly. I'm afraid I'm not much help with those, although I agree that the bad shrinking is probably related to the performance issues. On 13 Nov 2014, at 04:22, Brian Craft craft...@gmail.com wrote: Interesting that you don't see a performance problem. What version did you try? I'm using 0.5.9. I just re-ran this example to make sure I wasn't imagining it. On the 11th run, it wedged, not returning, and burning a lot of cpu, presumably trying to shrink. It's a larger problem with the real test cases. One test ran for about eight hours yesterday before I killed it. Hard to tell what it's doing, and not sure what sort of times I should expect for shrinking larger test cases. On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote: I've also had some tricky shrinking type issues with recursive generators using bind. I had a play with your generators, using such-that to reduce the row/column name length and also preventing some generator shrinking by using no-shrink, but I didn't have much luck improving the resulting shrinks (though the shrinks did finish in a timely fashion). You might have more luck if you generated an m x n matrix size independently of the matrix itself and feed the size into the matrix generator and the samples and probes generators. This may allow the matrix size to be shrunk down more easily, while also shrinking the samples and probes vectors without them having to be regenerated for ever new matrix shrink (i.e. where the matrix size is maintained but the gen/int values within it are shrunk). However, this is all guess work and conjecture, as I don't understand the shrinking algorithm well enough, and I haven't tried my suggestion. Also, modifying your generators in this way may make your model less general if you wish to use it in other ways later. Cheers Lucas On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com wrote: Using test.check, I'm finding the shrinking to be very, very slow. Running a hundred cases takes a few seconds, unless it hits an error, in which case it takes 40-60 minutes to shrink, and the shrinking is not very effective (resulting test case is much larger than necessary). Sometimes the shrinking is much faster. It behaves a bit like it's occasionally getting into a pathological state, or a difficult shrinking scenario. Are there any docs on generators or the shrinking algorithm that would help build tests that shrink more effectively? The problematic generator builds a randomly-sized matrix of integers, with randomly assigned names for the rows and columns. The failure case is when either a column or row name is repeated. I expect the slow shrinking has something to do with it being rare for the generator to emit the same name twice. ; Generator of randomly sized matrices of random numbers. (def gen-matrix (gen/bind gen/s-pos-int (fn [x] (gen/bind gen/s-pos-int (fn [y] (gen/vector (gen/vector gen/int x) y)) ; Generator of matrix with probe and sample ids. (def gen-tsv (gen/bind gen-matrix (fn [m] (gen/hash-map :probes (gen/vector (gen/such-that not-empty gen/string-alpha-numeric) (count m)) :samples (gen/vector
Re: test.check slow shrinking
I've also had some tricky shrinking type issues with recursive generators using
bind. I had a play with your generators, using such-that to reduce the
row/column name length and also preventing some generator shrinking by using
no-shrink, but I didn't have much luck improving the resulting shrinks (though
the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size independently of
the matrix itself and feed the size into the matrix generator and the samples
and probes generators. This may allow the matrix size to be shrunk down more
easily, while also shrinking the samples and probes vectors without them having
to be regenerated for ever new matrix shrink (i.e. where the matrix size is
maintained but the gen/int values within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if you
wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft.br...@gmail.com wrote:
Using test.check, I'm finding the shrinking to be very, very slow. Running a
hundred cases takes a few seconds, unless it hits an error, in which case it
takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the shrinking
is much faster. It behaves a bit like it's occasionally getting into a
pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would help
build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]], :samples
[0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding, sometimes
failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: test.check slow shrinking
Interesting that you don't see a performance problem. What version did you
try? I'm using 0.5.9.
I just re-ran this example to make sure I wasn't imagining it. On the 11th
run, it wedged, not returning, and burning a lot of cpu, presumably trying
to shrink.
It's a larger problem with the real test cases. One test ran for about
eight hours yesterday before I killed it. Hard to tell what it's doing, and
not sure what sort of times I should expect for shrinking larger test cases.
On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
I've also had some tricky shrinking type issues with recursive generators
using bind. I had a play with your generators, using such-that to reduce
the row/column name length and also preventing some generator shrinking by
using no-shrink, but I didn't have much luck improving the resulting
shrinks (though the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size
independently of the matrix itself and feed the size into the matrix
generator and the samples and probes generators. This may allow the matrix
size to be shrunk down more easily, while also shrinking the samples and
probes vectors without them having to be regenerated for ever new matrix
shrink (i.e. where the matrix size is maintained but the gen/int values
within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if
you wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com javascript:
wrote:
Using test.check, I'm finding the shrinking to be very, very slow. Running
a hundred cases takes a few seconds, unless it hits an error, in which case
it takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the
shrinking is much faster. It behaves a bit like it's occasionally getting
into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would
help build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding,
sometimes failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clo...@googlegroups.com javascript:
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com javascript:
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+u...@googlegroups.com javascript:.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe
Re: test.check slow shrinking
I'm pretty sure I did encounter the performance problem you're talking about,
but I killed it and re-ran until I hit cases that shrink quickly. I'm afraid
I'm not much help with those, although I agree that the bad shrinking is
probably related to the performance issues.
On 13 Nov 2014, at 04:22, Brian Craft craft.br...@gmail.com wrote:
Interesting that you don't see a performance problem. What version did you
try? I'm using 0.5.9.
I just re-ran this example to make sure I wasn't imagining it. On the 11th
run, it wedged, not returning, and burning a lot of cpu, presumably trying to
shrink.
It's a larger problem with the real test cases. One test ran for about eight
hours yesterday before I killed it. Hard to tell what it's doing, and not
sure what sort of times I should expect for shrinking larger test cases.
On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
I've also had some tricky shrinking type issues with recursive generators
using bind. I had a play with your generators, using such-that to reduce the
row/column name length and also preventing some generator shrinking by using
no-shrink, but I didn't have much luck improving the resulting shrinks
(though the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size independently
of the matrix itself and feed the size into the matrix generator and the
samples and probes generators. This may allow the matrix size to be shrunk
down more easily, while also shrinking the samples and probes vectors
without them having to be regenerated for ever new matrix shrink (i.e. where
the matrix size is maintained but the gen/int values within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if
you wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com wrote:
Using test.check, I'm finding the shrinking to be very, very slow. Running
a hundred cases takes a few seconds, unless it hits an error, in which case
it takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the
shrinking is much faster. It behaves a bit like it's occasionally getting
into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would help
build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding,
sometimes failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are
Re: test.check slow shrinking
I tried your idea of generating the size first, then passing it to the
matrix vector generators. This does seem to work better. The shrunk cases
that return are actually worse, but so far it hasn't wedged itself, which
is a great improvement. They all return within a few seconds. I don't yet
understand the shrinking well enough to know why it's better.
Example shrunk state:
:samples [0 0 0 0 0 0 0 0 0], :probes [0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0]
(def gen-matrix-size
(gen/tuple gen/s-pos-int gen/s-pos-int))
(def gen-mostly-ints
(gen/frequency [[9 gen/int] [1 (gen/return Double/NaN)]]))
(defn gen-matrix2 [x y]
(gen/vector (gen/vector gen-mostly-ints x) y))
(defn gen-names [n]
(gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
n))
(def gen-tsv2
(gen/bind
gen-matrix-size
(fn [[x y]]
(gen/hash-map
:probes (gen-names x)
:samples (gen-names y)
:matrix (gen-matrix2 x y)
On Wednesday, November 12, 2014 12:47:37 PM UTC-8, Lucas Bradstreet wrote:
I'm pretty sure I did encounter the performance problem you're talking
about, but I killed it and re-ran until I hit cases that shrink quickly.
I'm afraid I'm not much help with those, although I agree that the bad
shrinking is probably related to the performance issues.
On 13 Nov 2014, at 04:22, Brian Craft craft...@gmail.com javascript:
wrote:
Interesting that you don't see a performance problem. What version did you
try? I'm using 0.5.9.
I just re-ran this example to make sure I wasn't imagining it. On the 11th
run, it wedged, not returning, and burning a lot of cpu, presumably trying
to shrink.
It's a larger problem with the real test cases. One test ran for about
eight hours yesterday before I killed it. Hard to tell what it's doing, and
not sure what sort of times I should expect for shrinking larger test cases.
On Wednesday, November 12, 2014 9:05:40 AM UTC-8, Lucas Bradstreet wrote:
I've also had some tricky shrinking type issues with recursive generators
using bind. I had a play with your generators, using such-that to reduce
the row/column name length and also preventing some generator shrinking by
using no-shrink, but I didn't have much luck improving the resulting
shrinks (though the shrinks did finish in a timely fashion).
You might have more luck if you generated an m x n matrix size
independently of the matrix itself and feed the size into the matrix
generator and the samples and probes generators. This may allow the matrix
size to be shrunk down more easily, while also shrinking the samples and
probes vectors without them having to be regenerated for ever new matrix
shrink (i.e. where the matrix size is maintained but the gen/int values
within it are shrunk).
However, this is all guess work and conjecture, as I don't understand the
shrinking algorithm well enough, and I haven't tried my suggestion. Also,
modifying your generators in this way may make your model less general if
you wish to use it in other ways later.
Cheers
Lucas
On 12 Nov 2014, at 02:43, Brian Craft craft...@gmail.com wrote:
Using test.check, I'm finding the shrinking to be very, very slow.
Running a hundred cases takes a few seconds, unless it hits an error, in
which case it takes 40-60 minutes to shrink, and the shrinking is not very
effective (resulting test case is much larger than necessary). Sometimes
the shrinking is much faster. It behaves a bit like it's occasionally
getting into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would
help build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers,
with randomly assigned names for the rows and columns. The failure case is
when either a column or row name is repeated. I expect the slow shrinking
has something to do with it being rare for the generator to emit the same
name twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The
test.check slow shrinking
Using test.check, I'm finding the shrinking to be very, very slow. Running
a hundred cases takes a few seconds, unless it hits an error, in which case
it takes 40-60 minutes to shrink, and the shrinking is not very effective
(resulting test case is much larger than necessary). Sometimes the
shrinking is much faster. It behaves a bit like it's occasionally getting
into a pathological state, or a difficult shrinking scenario.
Are there any docs on generators or the shrinking algorithm that would help
build tests that shrink more effectively?
The problematic generator builds a randomly-sized matrix of integers, with
randomly assigned names for the rows and columns. The failure case is when
either a column or row name is repeated. I expect the slow shrinking has
something to do with it being rare for the generator to emit the same name
twice.
; Generator of randomly sized matrices of random numbers.
(def gen-matrix
(gen/bind
gen/s-pos-int
(fn [x] (gen/bind
gen/s-pos-int
(fn [y] (gen/vector (gen/vector gen/int x) y))
; Generator of matrix with probe and sample ids.
(def gen-tsv
(gen/bind
gen-matrix
(fn [m]
(gen/hash-map
:probes (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count m))
:samples (gen/vector
(gen/such-that not-empty gen/string-alpha-numeric)
(count (first m)))
:matrix (gen/return m)
Shrinking will result in a case like
{:matrix [[1 4 -3] [-4 -3 -5] [-5 2 3] [4 -5 -5] [1 -2 3] [1 4 1]],
:samples [0 0 0], :probes [0 0 0 0 0 0]}
where :samples [0 0] :probes[0] would do.
The following test will exhibit the behavior, sometimes succeeding,
sometimes failing quickly, sometimes shrinking for a very long time:
(tc/quick-check 100 (prop/for-all [tsv gen-tsv] (= (count (set (:probes
tsv))) (count (:probes tsv)
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
