Hello Myron, thanks yet again for the detailed answer :) Well, I will time it. So a rule of a thumb would be anything below 500ms can go inside the test iself?
Also, you mentioned your following 'one behaviour per example'. If I'm correct, 'it` is a behaviour per example. So I'm assuming that in integration tests/acceptance tests you always have on it block per context with multiple assertions in it? I see what you mean. Specific on the example needs :needs_schema is much more elegant then using instance variable that is harder to decipher. Schema creation however is not enough, as I need the name of the scehma. Will the above let me use "UserSchemCache.schema" to get the schema name? Thanks again! On Tuesday, August 1, 2017 at 6:43:03 PM UTC+3, Myron Marston wrote: > > I don’t have sufficient information to tell you what I’d do in this > situation, but here are some general principles to help you think through > the tradeoffs here. > > - Whether or not I’d create a user schema per example really depends > on how long that operation takes. If it’s a pretty quick operation (e.g. > milliseconds) I’d probably just make one per example; since these are > end-to-end tests they’re expected to be slow and doing something like a > before(:context) hook caries a high maintenance cost since it has so > many caveats compared to a typical before(:example) hook. > - I personally don’t follow the “one expectation per example” rule at > all. It was a useful corrective to poorly written tests that contained > tons > of expectations and were hard to debug, but bears a high cost (much higher > than I’m willing to pay) due to all the time wasted repeating setup. In > fast, isolated unit tests, I do follow a principle I’d call “one behavior > per example” (where a single behavior can often be specified with a single > expectation, but not always). I care a lot about keeping a fast, snappy > test suite so in slower integration or acceptance tests I write much > courser-grained tests that may encapsulate a whole workflow. > - I tend to use :aggregate_failures for all my integration and > acceptance tests. > - If a user schema can safely be re-used in many examples, and > creation of the schema was slow enough to warrant sharing it among > multiple > examples, I’d probably use a different approach than a before(:context) > hook. Instead, I’d probably do something like this: > > # spec/spec_helper.rbmodule UserSchemaCache > def self.schema > @schema ||= SecureRandom.uuid.tap do |schema_name| > SchemaService.new.create_schema(schema_name) > end > endend > RSpec.configure do |config| > config.when_first_matching_example_defined(:needs_schema) do > UserSchemaCache.schema > endend > > # some_spec.rb > RSpec.describe 'Authentication', :needs_schema do > # ...end > > That allows you to use the same user schema among all your end-to-end > examples, incurring the cost of creating it only once. The > when_first_matching_example_defined hook is only invoked if there’s an > example matching :needs_schema, so there’s no cost when running your > other specs. > > HTH, > Myron > > > On Tue, Aug 1, 2017 at 5:34 PM, Jon Gordon <[email protected] <javascript:> > > wrote: > >> Hello again Myron :) >> >> I'm guessing that's why I couldn't find many before(:all) examples >> online. I wasn't aware of aggregate_failures, that's pretty cool feature! I >> always tried to follow the 'single assertion per test' principle. However, >> the example above is small portion of a bigger Spec. In the same Spec file >> I will also try to delete an entity (which require me to create an entity >> first), get a list of entities (create at least 2 entities) and expect >> exception to get raised when I'm trying to create an entity that I already >> created. All of those require a working schema. I can't group all up them >> under a single aggregate_failures test. So sadly, I will have to use >> instance variable, or to wrap a set block like the example you shared above >> (unless you have another idea) >> >> But I'm wondering, let's say I could control every point in the process >> and design this whole spec from scratch, having the option to do whatever I >> want on each of the remote micro-services. What would be the proper way of >> addressing it? Doesn't feel like the tests above are something specials, >> more like a standard REST service testing. Here's the option I can think >> about: >> >> 1. Create the schema over and over each test with 'before' block like so : >> >> let(schema_name) { SecureRandom.uuid.to } >> >> before >> user_schema.create_schema(schema_name) >> end >> >> it 'my test' do >> puts schema_name >> end >> >> This will be slower as schema will getting created every-test, and I only >> really need a single one as the schema is just a dependency and no the main >> focus of the test. But - perhaps clearer to read and avoid sharing context >> with before(:all). >> >> 2. Pre-populate the remote test database with default user schema with >> default name. I can either create it directly on the database when I spin >> the environment or from spec_helper . I can even store the default values >> in YML file, create an object from it (config) and use it on test. >> >> payload = JSON.parse(File.read('spec/acceptence/fixtures/feature.json')) >> payload['schema_name'] = config(:default_schema_name) >> >> This however makes the test less clear because not all information is >> being exposed when you read just the spec file. I would like to do it the >> 'proper' way - as I can probably talk with the other teams in the future - >> makes those test better align with the standards. >> Thanks for the help again! >> >> >> >> On Tuesday, August 1, 2017 at 12:48:45 PM UTC+3, Myron Marston wrote: >> >>> before(:all) hooks require special care and we usually recommend you >>> avoid them. Most RSpec tooling assumes a per-example lifecycle for >>> resources (such as DB transactions, test doubles, and let memoization), >>> and before(:all) hooks operate outside that lifecycle. If you’re not >>> careful to manage the resources explicitly you’re likely to experience >>> problems from using :all hooks. For this reason we don’t provide >>> syntactic sugar for them (such as a let construct), but it’s pretty >>> trivial to create your own construct if you want: >>> >>> module BeforeAllConstructs >>> def set(name, &block) >>> attr_reader name >>> before(:context) { instance_variable_set("@#{name}", block.call } >>> endend >>> RSpec.configure do |config| >>> config.extend BeforeAllConstructsend >>> >>> With this in place, you could use set in place of let to have a >>> construct like let designed for :all/:context hooks. >>> >>> That said, in your case, I wouldn’t recommend you take that route. It >>> doesn’t seem like having two separate examples here provides you any clear >>> benefit, and the fact you are considering using a before(:all) hook >>> indicates you want to do some computation once, and then make multiple >>> assertions about that. Instead, I’d recommend you combine the two examples, >>> but then use :aggregate_failures so that you get a list of all failures >>> (and not just the first one). Here’s how you could do that: >>> >>> RSpec.describe 'Authentication' do >>> subject { AuthenticationService.new } >>> let(:user_schema) { SchemaService.new } >>> >>> context 'When creating a user that does not exists ' do >>> it 'creates a new user', :aggregate_failures do >>> schema_name = SecureRandom.uuid.to_s >>> user_schema.create_schema(schema_name) >>> payload = >>> JSON.parse(File.read('spec/acceptence/fixtures/feature.json')) >>> payload['schema_name'] = schema_name >>> >>> response = subject.create_user(user: 'my_user', payload: payload) >>> >>> expect(response.code).to eq 200 >>> >>> response = subject.get_user_by_category(category: payload['category']) >>> remote_entity = JSON.parse(response.body) >>> >>> expect(payload.to_json).to eq( >>> remote_entity['list'][unique_value] >>> ) >>> end >>> endend >>> >>> HTH, >>> Myron >>> >>> >>> On Tue, Aug 1, 2017 at 2:49 PM, Jon Gordon <[email protected]> wrote: >>> >>>> Refactor how? the system consists of 4 micro-services. In a single >>>> sanity scenario, they are all being called (some sort of a smoke test to >>>> verify all can communicate with each other when spun up). The database is >>>> being written on the last micro-service in line, it's code that being >>>> handled by another group. I can ask them to give me an API to mock the db, >>>> but that still requires me during setup to access the mock, name the >>>> entity >>>> and then use that name in the example block. As I don't communicate here >>>> with objects like I do with unit-testings (instead I'm using REST API) I >>>> see no way how I can achieve that? >>>> >>>> Thanks! >>>> >>>> On Tuesday, August 1, 2017 at 9:25:40 AM UTC+3, Jon Rowe wrote: >>>>> >>>>> > As it not allowed by RSpec to use a let value inside a >>>>> before(:all) block. >>>>> >>>>> This is for good reason as it’s a bad idea to share test stare, you >>>>> could assign a constant but it would be better to refactor your code base >>>>> not to depend on an external db for each test like this. >>>>> >>>>> Jon Rowe >>>>> --------------------------- >>>>> [email protected] >>>>> jonrowe.co.uk >>>>> >>>>> On Tuesday, 1 August 2017 at 01:39, Jon Gordon wrote: >>>>> >>>>> Hi again :-) >>>>> >>>>> So I tried to write couple of RSpec test since we last talked, I'll >>>>> mention again I'm writing couple end to end tests to verify all >>>>> micro-services are up and running. Please consider the following example. >>>>> It's an authentication service, that can create users. Before user is >>>>> being >>>>> created, a schema for the user-type needs to be created on a different >>>>> micro-service: >>>>> >>>>> RSpec.describe 'Authentication' do >>>>> subject { AuthenticationService.new } >>>>> let(:user_schema) { SchemaService.new } >>>>> >>>>> context 'When creating a user that does not exists ' do >>>>> it 'response with status code 200 (success)' do >>>>> schema_name = SecureRandom.uuid.to_s >>>>> user_schema.create_schema(schema_name) >>>>> payload = JSON.parse(File.read( >>>>> 'spec/acceptence/fixtures/feature.json')) >>>>> payload['schema_name'] = schema_name >>>>> >>>>> response = subject.create_user(user: 'my_user', payload: payload >>>>> ) >>>>> >>>>> expect(response.code).to eq 200 >>>>> end >>>>> >>>>> it 'creates a new user' do >>>>> schema_name = SecureRandom.uuid.to_s >>>>> user_schema.create_schema(schema_name) >>>>> payload = JSON.parse(File.read( >>>>> 'spec/acceptence/fixtures/feature.json')) >>>>> payload['schema_name'] = schema_name >>>>> >>>>> subject.create_user(user: 'my_user', payload: payload) >>>>> >>>>> response = subject.get_user_by_category(category: payload[ >>>>> 'category']) >>>>> remote_entity = JSON.parse(response.body) >>>>> >>>>> expect(payload.to_json).to eq( >>>>> remote_entity['list'][unique_value] >>>>> ) >>>>> end >>>>> end >>>>> end >>>>> >>>>> To keep it dry, I should be moving the whole schema creation into a >>>>> before block: >>>>> >>>>> before >>>>> schema_name = SecureRandom.uuid.to >>>>> user_schema.create_schema(schema_name) >>>>> end >>>>> >>>>> Before makes sense to me over let here, because it's an action. >>>>> However, because the schema is more of a 'pre-condition', I can create it >>>>> just once, and avoid multiple schema in my database. Therefore, >>>>> before(:all) seems like a better option. >>>>> >>>>> before(:all) >>>>> schema_name = SecureRandom.uuid.to_s >>>>> user_schema.create_schema(schema_name) >>>>> end >>>>> >>>>> Now that problem is that when I create user in my examples, I NEED to >>>>> schema name, so I need to share context between the before and it block. >>>>> It >>>>> makes sense for me to do it like so: >>>>> >>>>> let(:schema_name) { SecureRandom.uuid.to_s } >>>>> >>>>> before(:all) >>>>> user_schema.create_schema(schema_name) >>>>> end >>>>> >>>>> Then I can create easily create user in my example like so: >>>>> >>>>> payload = JSON.parse(File.read('spec/acceptence/fixtures/feature.json' >>>>> )) >>>>> payload['schema_name'] = schema_name >>>>> >>>>> subject.create_user(user: 'my_user', payload: payload) >>>>> >>>>> Alas, this will not work. As it not allowed by RSpec to use a let >>>>> value inside a before(:all) block. So I need to hold a string that can be >>>>> used in both the it and before block. It can solved it by defining a >>>>> Constant or using Instance variable but both methods feels reek to me. I >>>>> mentioned I don't have access to the database (as those are remote >>>>> machines >>>>> and they don't expose the ip for that database) so I can't truncate the >>>>> db >>>>> information and avoid the before block here. >>>>> >>>>> Thanks! >>>>> >>>>> On Thursday, July 20, 2017 at 10:57:05 AM UTC+3, Jon Gordon wrote: >>>>> >>>>> Not in Unit-tests of-course, but It seems like the only option in real >>>>> end-to-end testing. The system is quite complex, as it's basically a set >>>>> of >>>>> couple micro-services. Each has it's own unique database (Can be >>>>> Postgress, >>>>> Casanda, Oracle...). In a single End to End test, all databases are >>>>> populated with information. Clearing tables between each test can take >>>>> time, and is quite complex. The CI process does re-start the containers >>>>> at >>>>> the very start of the test-run (so it's like restarting them to a fresh >>>>> state), but not during tests. >>>>> >>>>> if I'll take the list of music instruments in the Faker gem for >>>>> example, It only has around 10 options. So even if I use the unique flag >>>>> - >>>>> It will run out of options after 10 test-cases. I guess use 'msuic >>>>> instruments' in one spec file, and 'cat-names' on the other to avoid it, >>>>> but that means I need to 'remember' what pool of string I already used in >>>>> previous tests, and that's feel even worse for me. >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>>> but I thought that there no better way around it. Because >>>>> >>>>> On Thursday, July 20, 2017 at 3:29:25 AM UTC+3, Myron Marston wrote: >>>>> >>>>> In general, if you need absolutely unique strings, `SecureRandom.uuid` >>>>> is a good way to get one. UUIDs are universally unique, after all :). >>>>> >>>>> That said, the fact that you are running out of unique random strings >>>>> from faker is concerning. All tests should work off of a "clean slate" >>>>> in >>>>> your database, either by wrapping each test in a rolled-back transaction, >>>>> or by truncating your DB tables. Are you "leaking" DB records between >>>>> tests? >>>>> >>>>> Myron >>>>> >>>>> On Wed, Jul 19, 2017 at 12:42 PM, Jon Gordon <[email protected]> wrote: >>>>> >>>>> Hi Myron, >>>>> >>>>> I will definitely check the book - looks like it's perfect for RSpec >>>>> beginner. Also, thanks for sharing the example online - that's alone can >>>>> give me a good starting base :) >>>>> >>>>> I will ask another question while posting - for unit-testing I'm >>>>> using Faker gem to fake common strings. However, in end to end tests, we >>>>> work against a database - so even if I'm using the 'unique' method, I'm >>>>> running out of unique strings after a while. I'm using 'securerandom' to >>>>> generate random number, but wondering if there's a better approach for >>>>> that. >>>>> >>>>> Thanks! >>>>> >>>>> On Wednesday, July 19, 2017 at 6:33:49 PM UTC+3, Myron Marston wrote: >>>>> >>>>> My upcoming book, Effective Testing with RSpec 3 >>>>> <https://www.google.com/url?q=https%3A%2F%2Fpragprog.com%2Fbook%2Frspec3%2Feffective-testing-with-rspec-3&sa=D&sntz=1&usg=AFQjCNHGLaAn9OUSvszwbNhLSkP9Ypy-7A>, >>>>> >>>>> has an example of building a JSON API using end-to-end acceptance tests, >>>>> isolated unit tests, and integration tests. It might fit what you're >>>>> looking for better since you mentioned you're looking for examples of >>>>> end-to-end testing of REST services. >>>>> >>>>> The code for the book is all online <https://github.com/rspec-3-book>, >>>>> as well. >>>>> >>>>> All that said, Xavier's screen cast is very good, and I definitely >>>>> recommend it, particularly if you do better with videos than printed >>>>> materials. >>>>> >>>>> Myron >>>>> >>>>> On Wed, Jul 19, 2017 at 4:30 AM, Jon Gordon <[email protected]> wrote: >>>>> >>>>> Thanks Xavier :) >>>>> I will be checking this course! >>>>> >>>>> Is there perhaps an open-source project with end-to-end spec tests you >>>>> can recommend (REST tests are preferred, not Capybara)? something to get >>>>> a >>>>> reference from? >>>>> Thank you. >>>>> >>>>> On Wednesday, July 19, 2017 at 2:24:46 AM UTC+3, Xavier Shay wrote: >>>>> >>>>> Obligatory plug for >>>>> https://www.pluralsight.com/courses/rspec-ruby-application-testing which >>>>> touches on some of the themes you're asking about :) >>>>> >>>>> >>>>> On Tue, Jul 18, 2017, at 04:06 PM, Jon Rowe wrote: >>>>> >>>>> Hi Jon >>>>> >>>>> A couple of tips, firstly you can stub out your external dependencies >>>>> for an end to end test, it just depends on the level of integration you >>>>> want, it’s equally fine to do what you propose. For injecting your >>>>> endpoint >>>>> (IP, hostname or otherwise) you have a couple of ways of doing it, the >>>>> simplest is to use environment variables e.g. `ENV[‘API_ENDPOINT’]`, or >>>>> you >>>>> can build yourself a config system like you mention. The reason why you >>>>> don’t see big projects using external configuration files is it is >>>>> usually >>>>> done at the app level rather than in rspec. >>>>> >>>>> If you chose to go down the config file route, xml, yml or otherwise, >>>>> you’d be better off loading it in a spec_helper or other such support >>>>> file, >>>>> and assigning it somewhere. >>>>> >>>>> Personally I would go with json fixture files for static json, or a >>>>> generator method if it needs to be dynamic. >>>>> >>>>> Cheers. >>>>> Jon >>>>> >>>>> Jon Rowe >>>>> --------------------------- >>>>> [email protected] >>>>> jonrowe.co.uk >>>>> >>>>> On Wednesday, 19 July 2017 at 01:52, Jon Gordon wrote: >>>>> >>>>> >>>>> Hi everyone, >>>>> >>>>> I'm quite new to RSpec, and I have used it mainly for unit-testing. >>>>> Lately, a need for a small number of end-to-end tests became relevant. >>>>> When >>>>> writing test-cases, I'm trying to stub all dependencies, but because >>>>> that's >>>>> not an option when doing integration tests, I need some help to >>>>> understand >>>>> what's the proper way to do things. Here's couple of questions: >>>>> >>>>> 1. The test requires an IP for remote machine (which is not local and >>>>> sadly can not be). Obviously, I shouldn't supply the IP inside the spec >>>>> file. The simple way is reading an external YML file with the IP (that >>>>> will >>>>> get created automatically during the CI process with the right IP for >>>>> example) and populate the IP directly from it. But, I was checking couple >>>>> of big project that uses rspec, and I never seen an external >>>>> configuration >>>>> file, so I'm thinking perhaps there is a better way of doing it >>>>> >>>>> 2. If indeed YML file is the right answer, I'm not sure if reading >>>>> from the YML file every spec file (that uses this service) is the right >>>>> thing to do? Shouldn't I be using hooks instead for that? >>>>> >>>>> 3. The test-object is a REST service, and some of the requests require >>>>> big json object. I have two options: >>>>> a. I can create the json object in the spec file itself (which >>>>> makes all information visible to you from the spec file itself, but >>>>> clutters the spec) >>>>> b. Creating an external default fixture (which is basically a json >>>>> file), read from it during the spec, and re-write the values that are >>>>> relevant for the specific tests. >>>>> >>>>> Thank you! >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/61ac9ade-1045-4211-80d3-441ef01ae7cb%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/rspec/61ac9ade-1045-4211-80d3-441ef01ae7cb%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/3FF6FCF2018A482CBDC70C02BAFFB643%40jonrowe.co.uk >>>>> >>>>> <https://groups.google.com/d/msgid/rspec/3FF6FCF2018A482CBDC70C02BAFFB643%40jonrowe.co.uk?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/28f3f239-1515-437b-b011-82b2dd163502%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/rspec/28f3f239-1515-437b-b011-82b2dd163502%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/c297c4c9-5225-47d9-a6e2-80f461bd1226%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/rspec/c297c4c9-5225-47d9-a6e2-80f461bd1226%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "rspec" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/rspec/48d88387-8e71-49a5-b25a-850a79fe4181%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/rspec/48d88387-8e71-49a5-b25a-850a79fe4181%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "rspec" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/rspec/3ae74ab4-f4fd-4f17-b87f-4256656b57d4%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/rspec/3ae74ab4-f4fd-4f17-b87f-4256656b57d4%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "rspec" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] <javascript:> >> . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/rspec/adfd947f-875e-47cf-91e5-a95264f7aedc%40googlegroups.com >> >> <https://groups.google.com/d/msgid/rspec/adfd947f-875e-47cf-91e5-a95264f7aedc%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "rspec" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/0f5083dc-5a77-44ee-af4f-44a34d8a4e56%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
