That's good feedback, thanks Shane! On Thu, Sep 20, 2018 at 6:23 AM Shane Ardell <shane.m.ard...@gmail.com> wrote:
> While the Cypress team suggests taking advantage of stubs where you can, > especially during development, we would definitely be able to test real > endpoints [1]. It can be used exactly like how Protractor is used, but with > the many benefits and features it provides [2]. Cypress also offers tools > for unit testing [3], which I think may be causing confusion as to what > exactly the library does. Cypress' main focus is e2e tests, but because of > its architecture, it can be used for all types of tests. > > I agree with everything you mentioned, Mike. I think our approach now is > fine, but in the future I do think it's worth considering the Cypress > team's suggestions for when and when not to stub, but there are no hard and > fast rules [4][5]. > > I currently have a branch available on my fork where I've migrated over > some e2e tests from Protractor to Cypress. With the exception of a little > code cleanup, these tests perform the same steps as they do with > Protractor. I have yet to include instructions in the README or include an > npm script, but if anyone wants to see it in action they can do the > following: > > - download this branch: > https://github.com/sardell/metron/tree/METRON-1648, > - run `npm ci` from meron-alerts, > - start the e2e test server, > - run `./node_modules/.bin/cypress open` > - start a single test by clicking on a file name in the Cypress user > interface, or run them all by clicking the play button. > > I'll try to send some sort of benchmarks when I get a chance to show the > speed difference between the two libraries. > > [1] https://docs.cypress.io/api/commands/request.html > [2] https://www.cypress.io/features/ > [3] https://docs.cypress.io/guides/guides/stubs-spies-and-clocks.html > [4] > > https://docs.cypress.io/guides/guides/network-requests.html#Testing-Strategies > . > [5] > > https://docs.cypress.io/guides/getting-started/testing-your-app.html#Stubbing-the-Server > > On Thu, Sep 20, 2018 at 12:09 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > Shane, > > > > Can you elaborate on the testing model you're proposing? I looked through > > the overview and some of the documentation. As far as I can tell, this > > would effectively be and e2e test for the UI *only*, so we would be > missing > > testing the actual integration points with the REST API or any other > > potential endpoints. > > > > 1. Are you proposing we migrate all existing e2e tests, including > those > > that currently hit Elasticsearch? > > 2. Would shifting to Cypress mean that all e2e tests would be isolated > > to only what is rendered via the browser? i.e. our e2e suite is no > > longer > > testing integration to a backend? > > > > My assumption with the term e2e testing is that you are testing an entire > > vertical slice with no substantive mock/stub/fake/spy/dummy [1] in the > way > > except for maybe some strategic cross-cutting concerns. It sounds like > > Cypress does NOT mean full e2e. My initial reaction to this is that > there's > > a place for both forms of testing. If Cypress would help UI developers > work > > on incremental changes, similar to how unit tests via JUnit help Java > > developers iterate on features, then I think that's great. I'm all for > > that! But unit tests are only one form of testing - we also do > integration > > testing, which can flex multiple classes/components together, as well as > > more broad stack integration/functional testing that verifies everything > > works when integrated together. Generally speaking, total # of unit > tests > > > # of integration tests > # functional/acceptance tests. I think we should > > carve out and define a testing approach for each. Can you elaborate a bit > > on your vision for how to manage the test interactions, or lack thereof, > > with the REST API as an integration endpoint? [2] > > > > At the time the write-up James shared was written, it appears that > Cypress > > was not yet open source. Now, it's MIT license - > > https://github.com/cypress-io/cypress/blob/develop/LICENSE.md. > > > > Mike > > > > 1. > > > > > https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs > > 2. https://martinfowler.com/articles/practical-test-pyramid.html#UiTests > > > > > > On Wed, Sep 19, 2018 at 8:47 AM James Sirota <jsir...@apache.org> wrote: > > > > > This article comparing the two is not favorable for Cypress. Are any > of > > > these concerns relevant to us? If not, then I think Cypress is fine > > > > > > > > > > > > https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7 > > > > > > > > >
