Issue #10296 has been updated by James Turnbull. Category set to testing Status changed from Unreviewed to Needs More Information Assignee set to Sean Millichamp
Good catch. Sean - what Puppet version? ---------------------------------------- Bug #10296: rspec unit tests are not reporting failure when they should be https://projects.puppetlabs.com/issues/10296 Author: Sean Millichamp Status: Needs More Information Priority: High Assignee: Sean Millichamp Category: testing Target version: Affected Puppet version: Keywords: rspec schedule testing Branch: I was working on a patch to the schedule type and as part of this I was writing some additional unit tests into the spec/unit/type/schedule_spec.rb file I had written a number of tests which I fully expected to fail and then ran them. Much to my surprise they all showed as passing. After some investigation, every call like "@schedule.should be_match" always reports passing, even if it should have failed. After dropping some debugging statements in schedule's match? method it seems that it is never even being invoked. All tests of the form "@schedule.should_not be_match" do invoke match? in schedule and pass/fail appropriately. I don't know enough about how rspec works to troubleshoot it further, but it seems like a considerable issue to me. In fact, I believe that if this was working properly you would see at least one test failure relating to arrays of ranges. A simple change that can be made to the unit test to show this behavior is: <pre> diff --git a/spec/unit/type/schedule_spec.rb b/spec/unit/type/schedule_spec.rb index b302d95..7a04abc 100755 --- a/spec/unit/type/schedule_spec.rb +++ b/spec/unit/type/schedule_spec.rb @@ -81,12 +81,12 @@ describe Puppet::Type.type(:schedule) do end it "should match when the start time is before the current time and the end time is after the current time" do - @schedule[:range] = "10:59:50 - 11:00:10" + @schedule[:range] = "10:59:50 - 10:59:55" @schedule.should be_match end it "should not match when the start time is after the current time" do - @schedule[:range] = "11:00:05 - 11:00:10" + @schedule[:range] = "10:59:05 - 11:00:10" @schedule.should_not be_match end </pre> Before this change, both show success (as you would expect). After this change, This first one should report failure, and doesn't. The second one should report failure and does. Dropping a debugging print in schedule's match? method will show that "should"s never invoke match?. This is the single failure run after the above changes: <pre> Failures: 1) Puppet::Type::Schedule Puppet::Type::Schedule when matching ranges should not match when the start time is after the current time Failure/Error: @schedule.should_not be_match expected match? to return false, got true # ./schedule_spec.rb:90 Finished in 0.52018 seconds 41 examples, 1 failure, 1 pending </pre> Thank you. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
