Unfortunately, there’s no way to use :skip metadata at the group level to
skip before(:all) hooks. That’s because of how metadata in RSpec is
modeled, and how RSpec implements :skip. :skip metadata is handled at the
level of individual examples, and metadata is inherited from a group to its
enclosing examples. So, for example, this works:
RSpec.describe MyClass, :skip do
it("is skipped"){ }
it("is also skipped") { }end
Both examples will be skipped here.
But consider that you can also do this:
RSpec.describe MyClass, :skip do
it("is skipped"){ }
it("is also skipped") { }
it("is not skipped", skip: false) { }end
Here we have on example that is overriding the skip: true metadata
inherited from the group. As a result, a before(:all) hook can’t simply
look at the group metadata to decide whether or not to skip or not. In
general, before(:all) hooks have lots of gotchas like this because they
don’t really fit well into the per-example semantics of so many parts of
RSpec. If you can refactor your tests to no longer need such a hook
(possibly using a aggregate_failures
<https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures>),
that is worth considering. Besides that, the other option you can do is to
manually call the skip method from your before(:all) hook:
before(:all) do
skip "reason to skip" if should_skip?
# rest of your hook logicend
HTH,
Myron
On Tue, Dec 19, 2017 at 3:47 PM, Jan P. <[email protected]> wrote:
> Jon, I tried
>
> describe "xcpretty reporter options generation", requires_xcodebuild: true do
>
>
> and
>
> before(:all), requires_xcodebuild: true do
>
>
> and
>
>
> describe Scan::XCPrettyReporterOptionsGenerator, requires_xcodebuild: true do
>
>
> but all didn't work and the tests were logged as failure because of the
> `before` being executed.
>
> Am I doing this wrong somehow?
>
>
> Code (last iteration) is here:
> https://github.com/fastlane/fastlane/blob/janpio-mark_
> skipped_tests_as_pending_with_reason/scan/spec/xcpretty_
> reporter_options_generator_spec.rb
> https://github.com/fastlane/fastlane/blob/janpio-mark_
> skipped_tests_as_pending_with_reason/spec_helper.rb#L68-L83
> Matching Circle CI run:
> https://circleci.com/gh/fastlane/fastlane/12586
>
> -Jan
>
>
>
> Am Dienstag, 19. Dezember 2017 23:05:53 UTC+1 schrieb Jon Rowe:
>>
>> You should be able to specify `requires_xcodebuild: true` on the
>> describe as we’ll to prevent the before all being triggered
>>
>> Jon Rowe
>> ---------------------------
>> [email protected]
>> jonrowe.co.uk
>>
>> On Wednesday, 20 December 2017 at 05:56, Jan P. wrote:
>>
>> Excuse the terrible code formatting, seems Google Groups wanted to be
>> helpful.
>>
>> Another try:
>>
>> describe Scan do
>> describe Scan::XCPrettyReporterOptionsGenerator do
>> before(:all) do
>> // code that fails when executed on non-macOS
>> end
>>
>> describe "xcpretty reporter options generation" do
>> it "generates options for the junit tempfile report required by scan",
>> requires_xcodebuild: true do
>> ...
>>
>> -J
>>
>>
>> Am Dienstag, 19. Dezember 2017 19:55:45 UTC+1 schrieb Jan P.:
>>
>> Yes, that works mostly like expected:
>>
>>
>> 4704 examples, 21 failures, 154 pending
>>
>>
>> and
>>
>> ...
>> [18:39:20]: ▸ Pending: (Failures listed here are expected and do not
>> affect your suite's status)
>> [18:39:20]: ▸ 1) Fastlane Fastlane::EnvironmentPrinter contains main
>> information about the stack
>> [18:39:20]: ▸ # Requires Xcode to be installed which is not possible on
>> this platform
>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:28
>> [18:39:20]: ▸ 2) Fastlane Fastlane::EnvironmentPrinter
>> FastlaneCore::Helper.xcode_version cannot be obtained contains stack
>> information other than Xcode Version
>> [18:39:20]: ▸ # Requires Xcode to be installed which is not possible on
>> this platform
>> [18:39:20]: ▸ # ./fastlane/spec/env_spec.rb:47
>> ...
>>
>>
>> Awesome!
>>
>>
>> The 21 failures are new though.
>>
>> I have a _spec.rb file with 21 examples that has a *before(:all)* that
>> seems to have been filtered with my old solution, but with *skip *is now
>> executed and fails:
>>
>> describe Scan do
>> describe Scan::XCPrettyReporterOptionsGenerator do
>> before(:all) do
>> .. code that fails when executed on non-macOS ...
>> end
>>
>> describe "xcpretty reporter options generation" do
>> it "generates options for the junit tempfile report required by scan",
>> requires_xcodebuild: true do
>> ...
>>
>> Any idea what I can do about this?
>> -J
>>
>>
>>
>> Am Dienstag, 19. Dezember 2017 19:26:17 UTC+1 schrieb Jan P.:
>>
>> Thanks for the quick answer.
>>
>> I missed "skipping examples" because I was so happy to have found
>> exclusion filters. Sounds like pretty much what I am looking for - even
>> better with the explicit reason I can set for skipping. Will try and report
>> back.
>>
>> Best,
>> Jan
>>
>>
>>
>> Am Dienstag, 19. Dezember 2017 17:37:56 UTC+1 schrieb Myron Marston:
>>
>> RSpec does not provide a way to get the number of examples that were
>> excluded by its inclusion or exclusion filters, but there’s a different
>> mechanism that will do what you want. Instead of filtering the examples
>> (which excludes them from consideration entirely), you can skip them, which
>> prevents the body of the example from running, sets the example’s status to
>> :pending, will print the example in yellow in the formatter output, and
>> will count the example in the summary total printed at the end (e.g. “500
>> examples, 0 failures, 20 pending”). Normally, :skip metadata will cause
>> an example to be skipped
>> <https://www.google.com/url?q=https%3A%2F%2Frelishapp.com%2Frspec%2Frspec-core%2Fv%2F3-7%2Fdocs%2Fpending-and-skipped-examples%2Fskip-examples%23skipping-using-metadata&sa=D&sntz=1&usg=AFQjCNH2bA7au32CPocvg1H0M1Vzmcf5IQ>,
>> but you’ve overwritten it to cause :skip to cause examples to be
>> filtered out.
>>
>> Here’s my suggestion for how to wire this up.
>>
>> First, tag any examples that depend upon xcode with :uses_xcode (rather
>> than :skip), e.g.:
>>
>> it "uses a feature of xcode", :xcode do
>> # ...end
>>
>> it "does not use xcode at all" do
>> # ...end
>>
>> Then use define_derived_metadata to automatically tag these examples
>> with :skip if you are not running on OS X:
>>
>> # spec_helper.rbrequire 'rbconfig'
>> RSpec.configure do |config|
>> unless RbConfig::CONFIG['host_os'] =~ /darwin/
>> config.define_derived_metadata(:xcode) do |meta|
>> meta[:skip] = "Can only be run on OS X"
>> end
>> endend
>>
>> The “Can only be run on OS X” bit will be printed in the output as the
>> reason the examples are pending.
>>
>> HTH,
>> Myron
>>
>>
>>
>> On Tue, Dec 19, 2017 at 3:06 AM, Jan P. <[email protected]> wrote:
>>
>> RSpec has this nice method to exclude individual tests/examples or whole
>> groups by using filter_run_excluding in the config, then tagging the
>> examples:
>>
>> https://relishapp.com/rspec/rspec-core/v/3-7/docs/filtering/
>> exclusion-filters
>>
>> RSpec.configure do |c|
>> c.filter_run_excluding :skip => trueend
>> RSpec.describe "something" do
>> it "does one thing" do
>> end
>>
>> it "does another thing", :skip => true do
>> endend
>>
>> "does one thing" will be checked,
>> "does another thing" will not.
>>
>>
>> We are using this, for example, to skip some tests depending on the
>> platform the test is run on by wrapping the c.filter_run_excluding :skip
>> => true in an if block:
>>
>> If Mac,
>> no exclusions, if Ubuntu,
>> exclude tests that do something with Xcode.
>>
>>
>> Right now the numbers of passing examples/test is just lower if the
>> exclusion filter is used, but it would be nice to see the actual number of
>> tests that are skipped.
>>
>> Is there a way to get the number of tests skipped by this method during a
>> test run?
>>
>> Thanks,
>>
>> Jan
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "rspec" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To post to this group, send email to [email protected].
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/rspec/3108ef8e-303d-425b-9b00-ab83dfec7633%40googlegroups.com
>> <https://groups.google.com/d/msgid/rspec/3108ef8e-303d-425b-9b00-ab83dfec7633%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "rspec" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To post to this group, send email to [email protected].
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/rspec/bcfd5676-92bd-4051-a3dd-5cb942784698%40googlegroups.com
>> <https://groups.google.com/d/msgid/rspec/bcfd5676-92bd-4051-a3dd-5cb942784698%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "rspec" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/rspec/715c3e8b-c8e0-4c98-a0ee-9ebf1d9c4ae4%40googlegroups.com
> <https://groups.google.com/d/msgid/rspec/715c3e8b-c8e0-4c98-a0ee-9ebf1d9c4ae4%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/rspec/CADUxQmtSE9_P-YvEmXp%2BgHFSPxHKzNXjRO_gq%2BGjiN2Lqqw1vw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
