Issue #10296 has been updated by Sean Millichamp.
I can confirm Justin's findings.
If I take the original spec code from master (before the diffs I posted) and
replace all instances of:
<pre>
@schedule.should be_match
</pre>
with
<pre>
@schedule.match?.should be_true
</pre>
or, if they have a parameter passed, replacing all instances of:
<pre>
@schedule.should be_match(previous)
</pre>
with:
<pre>
@schedule.match?(previous).should be_true
</pre>
Then I get one failure:
<pre>
Failures:
1) Puppet::Type::Schedule Puppet::Type::Schedule when matching ranges should
match the upper array of ranges
Failure/Error: @schedule.match?.should be_true
expected false to be true
# ./schedule_spec.rb:116
Finished in 0.5196 seconds
41 examples, 1 failure, 1 pending
</pre>
I believe that this is correctly failing now. I could not get arrays of ranges
to work in my prior interactive testing of schedules and this failure supports
my experience that they do not function.
What should the next course of action be? I could submit a patch that would
adjust all of the tests to the "@schedule.match?.should" style but I feel like
leaving the "@schedule.should be_match" style unfixed would be asking for this
to happen again and likely should be addressed by someone who knows rspec.
----------------------------------------
Bug #10296: rspec unit tests are not reporting failure when they should be
https://projects.puppetlabs.com/issues/10296
Author: Sean Millichamp
Status: Accepted
Priority: High
Assignee:
Category: testing
Target version:
Affected Puppet version: development
Keywords: rspec schedule testing
Branch: master
I was working on a patch to the schedule type and as part of this I was writing
some additional unit tests into the spec/unit/type/schedule_spec.rb file
I had written a number of tests which I fully expected to fail and then ran
them. Much to my surprise they all showed as passing.
After some investigation, every call like "@schedule.should be_match" always
reports passing, even if it should have failed. After dropping some debugging
statements in schedule's match? method it seems that it is never even being
invoked.
All tests of the form "@schedule.should_not be_match" do invoke match? in
schedule and pass/fail appropriately.
I don't know enough about how rspec works to troubleshoot it further, but it
seems like a considerable issue to me. In fact, I believe that if this was
working properly you would see at least one test failure relating to arrays of
ranges.
A simple change that can be made to the unit test to show this behavior is:
<pre>
diff --git a/spec/unit/type/schedule_spec.rb b/spec/unit/type/schedule_spec.rb
index b302d95..7a04abc 100755
--- a/spec/unit/type/schedule_spec.rb
+++ b/spec/unit/type/schedule_spec.rb
@@ -81,12 +81,12 @@ describe Puppet::Type.type(:schedule) do
end
it "should match when the start time is before the current time and the
end time is after the current time" do
- @schedule[:range] = "10:59:50 - 11:00:10"
+ @schedule[:range] = "10:59:50 - 10:59:55"
@schedule.should be_match
end
it "should not match when the start time is after the current time" do
- @schedule[:range] = "11:00:05 - 11:00:10"
+ @schedule[:range] = "10:59:05 - 11:00:10"
@schedule.should_not be_match
end
</pre>
Before this change, both show success (as you would expect). After this
change, This first one should report failure, and doesn't. The second one
should report failure and does. Dropping a debugging print in schedule's
match? method will show that "should"s never invoke match?. This is the single
failure run after the above changes:
<pre>
Failures:
1) Puppet::Type::Schedule Puppet::Type::Schedule when matching ranges should
not match when the start time is after the current time
Failure/Error: @schedule.should_not be_match
expected match? to return false, got true
# ./schedule_spec.rb:90
Finished in 0.52018 seconds
41 examples, 1 failure, 1 pending
</pre>
Thank you.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
