[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #20 from Željko Filipin zfili...@wikimedia.org --- Thanks Dan, I have tested this and it works great! :) -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #18 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 160024 merged by jenkins-bot: Stricter pending behavior for falsely passing steps https://gerrit.wikimedia.org/r/160024 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 Dan Duvall dduv...@wikimedia.org changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED Resolution|--- |FIXED --- Comment #19 from Dan Duvall dduv...@wikimedia.org --- Released in 0.4.0. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #16 from Željko Filipin zfili...@wikimedia.org --- (In reply to Dan Duvall from comment #15) I think I'd still rather just reimplement pending for our own Cucumber World (see comment 9) over monkey patching Cucumber::Pending (#1) or adding another dependency (#2). Sounds good to me! :) -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #17 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 160024 had a related patch set uploaded by Dduvall: Stricter pending behavior for falsely passing steps https://gerrit.wikimedia.org/r/160024 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 Gerrit Notification Bot gerritad...@wikimedia.org changed: What|Removed |Added Status|NEW |PATCH_TO_REVIEW -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #15 from Dan Duvall dduv...@wikimedia.org --- I think I'd still rather just reimplement pending for our own Cucumber World (see comment 9) over monkey patching Cucumber::Pending (#1) or adding another dependency (#2). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #14 from Željko Filipin zfili...@wikimedia.org --- I see, that was what I misunderstood. Thanks. :) Ok then, so we could do three things: #1 patch Cucumber::Pending #2 require RSpec::Core::Pending and use it #3 report a bug for Cucumber::Pending saying it does not behave as RSpec::Core::Pending, which is confusing -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #12 from Željko Filipin zfili...@wikimedia.org --- As far as I understand it, Cucumber does use rspec, that is why we explicitly use rspec-expectations[2]. 1: https://github.com/wikimedia/mediawiki-selenium/blob/master/mediawiki_selenium.gemspec#L26 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #13 from Dan Duvall dduv...@wikimedia.org --- Right, but rspec-expectations is separate from rspec-core which is what implements `describe`, `it`, `pending`, etc. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243
--- Comment #10 from Željko Filipin zfili...@wikimedia.org ---
I think just my Ruby-fu is failing me. See comments 6 and 7. RSpec::Pending
already does what I want, but looks like I am calling Cucumber::Pending here:
pending('bug #56081') do
true.should be(true)
end
How do I make sure RSpec::Pending is called. See comment 7 on what I have
tried.
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #11 from Dan Duvall dduv...@wikimedia.org --- I'm not sure that cucumber even uses `rspec-core`. It might just use the matchers. Either way, I think implementing our own helper is more inline with the Cucumber Way—not that I think the Cucumber Way is great, but it's better not to wrestle with it too much. If you don't want to override the behavior of `#pending` globally, then we can use a different name (`#pending_or_fail`, `#temporarily_pending`, etc.). -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #9 from Dan Duvall dduv...@wikimedia.org --- We could reimplement `#pending` in a world helper module and rescue/re-raise the `Cucumber::Pending` exception as something else. module MediawikiSelenium::Pending class Failure StandardError; end def pending(*args) super rescue Cucumber::Pending = e if block_given? raise Failure, e.message else raise e end end end World(MediawikiSelenium::Pending) -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 Željko Filipin zfili...@wikimedia.org changed: What|Removed |Added CC||dduv...@wikimedia.org Whiteboard||zfilipin --- Comment #8 from Željko Filipin zfili...@wikimedia.org --- Dan, any idea on how to make this happen? -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending RSpec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 Željko Filipin zfili...@wikimedia.org changed: What|Removed |Added Summary|Cucumber step should fail |Cucumber step should fail |if pending rspec|if pending RSpec |expectation no longer fails |expectation no longer fails -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 Željko Filipin zfili...@wikimedia.org changed: What|Removed |Added Summary|Mark a scenario as failed |Cucumber step should fail |if pending rspec|if pending rspec |expectation passes |expectation no longer fails -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #1 from Željko Filipin zfili...@wikimedia.org --- The documentation says I am expecting the correct behavior: https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/pending.rb?source=cc#L50-L57 -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #2 from Željko Filipin zfili...@wikimedia.org --- Cucumber output if the step is not marked pending: $ bundle exec cucumber features/font_selection.feature:23 Using the default profile... F- (::) failed steps (::) expected: sans-serif got: OpenDyslexic,sans-serif (using ==) (RSpec::Expectations::ExpectationNotMetError) ./features/step_definitions/font_selection_steps.rb:15:in `/^the active content font must be the same as font prior to the preview$/' features/font_selection.feature:26:in `Then the active content font must be the same as font prior to the preview' Failing Scenarios: cucumber features/font_selection.feature:23 # Scenario: Discarding live preview of content font 1 scenario (1 failed) 9 steps (1 failed, 1 skipped, 7 passed) 1m12.158s -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #4 from Željko Filipin zfili...@wikimedia.org --- But formatter called pretty gives more information (... 'bug #56081 (Cucumber::Pending)' ...): $ bundle exec cucumber -f pretty features/font_selection.feature:23 Using the default profile... @commons.wikimedia.beta.wmflabs.org @login @reset-preferences-after Feature: Font selection In order to have better using experience, As a reader and writer, I want to change or disable the fonts for interface and content. In addition the user is provided live preview feature: changes are applied immediately when selection is made. Changes can either be applied or discarded for easy testing. Background:# features/font_selection.feature:12 Given I am logged in # features/step_definitions/common_steps.rb:8 And I set German as the interface language # features/step_definitions/common_steps.rb:12 And I open ULS # features/step_definitions/font_selection_steps.rb:1 And I open display settings # features/step_definitions/font_selection_steps.rb:5 When I open fonts panel of language settings # features/step_definitions/font_selection_steps.rb:9 Scenario: Discarding live preview of content font # features/font_selection.feature:23 When I select OpenDyslexic font for the content language for the live preview # features/step_definitions/panel_steps.rb:38 And I close the panel to discard the changes # features/step_definitions/panel_steps.rb:43 Then the active content font must be the same as font prior to the preview # features/step_definitions/font_selection_steps.rb:13 bug #56081 (Cucumber::Pending) ./features/step_definitions/font_selection_steps.rb:14:in `/^the active content font must be the same as font prior to the preview$/' features/font_selection.feature:26:in `Then the active content font must be the same as font prior to the preview' And the selected content font must be system # features/step_definitions/font_selection_steps.rb:19 1 scenario (1 pending) 9 steps (1 skipped, 1 pending, 7 passed) 1m3.638s -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #3 from Željko Filipin zfili...@wikimedia.org --- If the step is marked as pending, the default Cucumber formatter (progress) does not give a lot of information: $ bundle exec cucumber features/font_selection.feature:23 Using the default profile... P- (::) pending steps (::) features/font_selection.feature:26:in `Then the active content font must be the same as font prior to the preview' 1 scenario (1 pending) 9 steps (1 skipped, 1 pending, 7 passed) 0m55.161s -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243
--- Comment #5 from Željko Filipin zfili...@wikimedia.org ---
But if change the expectation to the example from rspec documentation (so the
expectation inside the pending block no longer fails:
pending('bug #56081') do
true.should be(true)
end
I was expecting the scenario to fail, but it did not fail:
$ bundle exec cucumber -f pretty features/font_selection.feature:23
Using the default profile...
@commons.wikimedia.beta.wmflabs.org @login @reset-preferences-after
Feature: Font selection
In order to have better using experience,
As a reader and writer,
I want to change or disable the fonts for interface and content.
In addition the user is provided live preview feature: changes are applied
immediately when selection is made. Changes can either be applied or
discarded
for easy testing.
Background:#
features/font_selection.feature:12
Given I am logged in #
features/step_definitions/common_steps.rb:8
And I set German as the interface language #
features/step_definitions/common_steps.rb:12
And I open ULS #
features/step_definitions/font_selection_steps.rb:1
And I open display settings #
features/step_definitions/font_selection_steps.rb:5
When I open fonts panel of language settings #
features/step_definitions/font_selection_steps.rb:9
Scenario: Discarding live preview of content font
# features/font_selection.feature:23
When I select OpenDyslexic font for the content language for the live
preview # features/step_definitions/panel_steps.rb:38
And I close the panel to discard the changes
# features/step_definitions/panel_steps.rb:43
Then the active content font must be the same as font prior to the preview
# features/step_definitions/font_selection_steps.rb:13
Expected pending 'bug #56081' to fail. No Error was raised. No longer
pending? (Cucumber::Pending)
./features/step_definitions/font_selection_steps.rb:14:in `/^the active
content font must be the same as font prior to the preview$/'
features/font_selection.feature:26:in `Then the active content font must
be the same as font prior to the preview'
And the selected content font must be system
# features/step_definitions/font_selection_steps.rb:19
1 scenario (1 pending)
9 steps (1 skipped, 1 pending, 7 passed)
0m59.764s
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243 --- Comment #6 from Željko Filipin zfili...@wikimedia.org --- I think the problem is that we are using Cucumber::Pending instead of RSpec::Pending, and looks like Cucumber does not behave like RSpec. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 56243] Cucumber step should fail if pending rspec expectation no longer fails
https://bugzilla.wikimedia.org/show_bug.cgi?id=56243
--- Comment #7 from Željko Filipin zfili...@wikimedia.org ---
I am not sure how to call RSpec#pending from Cucumber, I have tried this:
RSpec::Core::Pending.pending('bug #56081')
but no luck:
uninitialized constant RSpec::Core (NameError)
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
