Re: [DISCUSS] Migrate from Protractor to Cypress
I'm still for the migration to Cypress, Tibor. Thanks for the PR and for circling back on this thread. On Tue, Oct 30, 2018 at 6:39 AM Tibor Meller wrote: > Hi All, > > To continue this thread and keep you updated I would like to share that I'm > opened a PR with Cypress UI tests for PCAP. > I was also able to integrate these tests into Travis CI. > https://github.com/apache/metron/pull/1226 > You can find more information in the PR description. > > I also opened a new Jira ticket to track a possible migration to Cypress > from Protractor. > https://issues.apache.org/jira/browse/METRON-1848 > In this ticket, I collected the existing UI tests written in protractor. > Also added links to this thread and the previous related tickets to make it > easier to follow this discussion. > > Please let me know what do you think about a migration to Cypress as a next > step! > > Thanks, > Tibor > > On Thu, Sep 27, 2018 at 9:23 AM Tibor Meller > wrote: > > > Great Guys! Thanks for the feedback. I'll move forward as discussed. > > > > Thx > > > > On Wed, Sep 26, 2018 at 11:44 PM Michael Miklavcic < > > michael.miklav...@gmail.com> wrote: > > > >> I'm good with it. We can see some tests in action (and hopefully running > >> in > >> Travis! :-D) and then migrate and deprecate Protractor accordingly if we > >> still agree that's the way to go. When you submit the first PR, please > >> link > >> to this DISCUSS via permalink from the mailing list archives. Thanks > guys. > >> > >> Cheers, > >> Mike > >> > >> On Wed, Sep 26, 2018 at 7:17 AM Shane Ardell > >> wrote: > >> > >> > I think Tibor's idea of using PCAP tests as an introduction to Cypress > >> for > >> > Metron is a great idea. As he pointed out, PCAP tests can take > >> advantage of > >> > Cypress' capability to mock responses, and we can set it up to run in > >> > Travis. Once the community is able to see the benefits from an actual > >> set > >> > of Cypress tests inside the project and running in Travis, I think any > >> > questions about migrating the rest of the existing tests from > >> Protractor to > >> > Cypress will be settled. However, if for some reason we run into > issues > >> > implementing or running the tests, we will have invested a fraction of > >> time > >> > vs. migrating all the tests right away. > >> > > >> > On Wed, Sep 26, 2018 at 2:12 PM Tibor Meller > >> > wrote: > >> > > >> > > Hi Team, > >> > > > >> > > Many of us agreed on that Cypress could be a more capable tool for > us > >> to > >> > > write high-level UI tests, whether those be e2e, integration or > >> automated > >> > > regression tests. If there is no open question left about cypress we > >> > could > >> > > to bring it a test drive. My suggestion is to implement the PCAP UI > >> tests > >> > > with Cypress. Some services and PCAP semple data yet not available > >> from > >> > our > >> > > CI environment so protractor is hardly applicable here. This would > be > >> a > >> > > great opportunity for cypress to shine. With Cypress, we are able to > >> mock > >> > > out those responses and make it run in Travis. > >> > > Anytime we make PCAP data available in Travis we could be able to > plug > >> > out > >> > > those mocks and run the same test as integration or e2e tests if we > >> like. > >> > > > >> > > Because it is relatively easy to migrate across cypress and > >> protractor I > >> > > see no major risks here if we decide to stick with Protractor for > some > >> > > reason. > >> > > > >> > > What do you think? > >> > > > >> > > Thanks for your feedback, > >> > > Tibor > >> > > > >> > > On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell < > >> shane.m.ard...@gmail.com> > >> > > wrote: > >> > > > >> > > > Hello everyone, > >> > > > > >> > > > Currently, we use Protractor to run our UI "end-to-end" tests. > >> However, > >> > > > there are a handful of major advantages we can gain from switching > >> to > >> > > > Cypress: https://www.cypress.io/features/. > >> > > > > >> > > >- As with most Selenium-based e2e testing frameworks, > Protractor > >> > > suffers > >> > > >from test flakiness. This is because Selenium runs outside of > the > >> > > > browser > >> > > >and executes remote commands across the network. To work around > >> this > >> > > at > >> > > > the > >> > > >moment, we are using protractor-flake to re-run failed tests, > but > >> > this > >> > > > is > >> > > >more of a crutch than a fix. Cypress executes in the same run > >> loop > >> > as > >> > > > the > >> > > >application it's testing, and as a result does not suffer from > >> the > >> > > same > >> > > >flakiness. > >> > > >- As a result of its architecture, Cypress runs much faster > than > >> > > >Protractor. This is especially critical if e2e tests are added > to > >> > the > >> > > CI > >> > > >build in the future. > >> > > >- Protractor is incredibly hard to debug. In contrast, Cypress > >> comes > >> > > >with a plethora of debugging features, some of which
Re: [DISCUSS] Migrate from Protractor to Cypress
Hi All, To continue this thread and keep you updated I would like to share that I'm opened a PR with Cypress UI tests for PCAP. I was also able to integrate these tests into Travis CI. https://github.com/apache/metron/pull/1226 You can find more information in the PR description. I also opened a new Jira ticket to track a possible migration to Cypress from Protractor. https://issues.apache.org/jira/browse/METRON-1848 In this ticket, I collected the existing UI tests written in protractor. Also added links to this thread and the previous related tickets to make it easier to follow this discussion. Please let me know what do you think about a migration to Cypress as a next step! Thanks, Tibor On Thu, Sep 27, 2018 at 9:23 AM Tibor Meller wrote: > Great Guys! Thanks for the feedback. I'll move forward as discussed. > > Thx > > On Wed, Sep 26, 2018 at 11:44 PM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > >> I'm good with it. We can see some tests in action (and hopefully running >> in >> Travis! :-D) and then migrate and deprecate Protractor accordingly if we >> still agree that's the way to go. When you submit the first PR, please >> link >> to this DISCUSS via permalink from the mailing list archives. Thanks guys. >> >> Cheers, >> Mike >> >> On Wed, Sep 26, 2018 at 7:17 AM Shane Ardell >> wrote: >> >> > I think Tibor's idea of using PCAP tests as an introduction to Cypress >> for >> > Metron is a great idea. As he pointed out, PCAP tests can take >> advantage of >> > Cypress' capability to mock responses, and we can set it up to run in >> > Travis. Once the community is able to see the benefits from an actual >> set >> > of Cypress tests inside the project and running in Travis, I think any >> > questions about migrating the rest of the existing tests from >> Protractor to >> > Cypress will be settled. However, if for some reason we run into issues >> > implementing or running the tests, we will have invested a fraction of >> time >> > vs. migrating all the tests right away. >> > >> > On Wed, Sep 26, 2018 at 2:12 PM Tibor Meller >> > wrote: >> > >> > > Hi Team, >> > > >> > > Many of us agreed on that Cypress could be a more capable tool for us >> to >> > > write high-level UI tests, whether those be e2e, integration or >> automated >> > > regression tests. If there is no open question left about cypress we >> > could >> > > to bring it a test drive. My suggestion is to implement the PCAP UI >> tests >> > > with Cypress. Some services and PCAP semple data yet not available >> from >> > our >> > > CI environment so protractor is hardly applicable here. This would be >> a >> > > great opportunity for cypress to shine. With Cypress, we are able to >> mock >> > > out those responses and make it run in Travis. >> > > Anytime we make PCAP data available in Travis we could be able to plug >> > out >> > > those mocks and run the same test as integration or e2e tests if we >> like. >> > > >> > > Because it is relatively easy to migrate across cypress and >> protractor I >> > > see no major risks here if we decide to stick with Protractor for some >> > > reason. >> > > >> > > What do you think? >> > > >> > > Thanks for your feedback, >> > > Tibor >> > > >> > > On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell < >> shane.m.ard...@gmail.com> >> > > wrote: >> > > >> > > > Hello everyone, >> > > > >> > > > Currently, we use Protractor to run our UI "end-to-end" tests. >> However, >> > > > there are a handful of major advantages we can gain from switching >> to >> > > > Cypress: https://www.cypress.io/features/. >> > > > >> > > >- As with most Selenium-based e2e testing frameworks, Protractor >> > > suffers >> > > >from test flakiness. This is because Selenium runs outside of the >> > > > browser >> > > >and executes remote commands across the network. To work around >> this >> > > at >> > > > the >> > > >moment, we are using protractor-flake to re-run failed tests, but >> > this >> > > > is >> > > >more of a crutch than a fix. Cypress executes in the same run >> loop >> > as >> > > > the >> > > >application it's testing, and as a result does not suffer from >> the >> > > same >> > > >flakiness. >> > > >- As a result of its architecture, Cypress runs much faster than >> > > >Protractor. This is especially critical if e2e tests are added to >> > the >> > > CI >> > > >build in the future. >> > > >- Protractor is incredibly hard to debug. In contrast, Cypress >> comes >> > > >with a plethora of debugging features, some of which you can see >> in >> > > > action >> > > >here: https://vimeo.com/242961930#t=264s >> > > > >> > > > Does anyone else have thoughts or opinions on switching to Cypress >> or >> > > > staying with Protractor? >> > > > >> > > > Cheers, >> > > > Shane >> > > > >> > > >> > >> >
Re: [DISCUSS] Migrate from Protractor to Cypress
Great Guys! Thanks for the feedback. I'll move forward as discussed. Thx On Wed, Sep 26, 2018 at 11:44 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I'm good with it. We can see some tests in action (and hopefully running in > Travis! :-D) and then migrate and deprecate Protractor accordingly if we > still agree that's the way to go. When you submit the first PR, please link > to this DISCUSS via permalink from the mailing list archives. Thanks guys. > > Cheers, > Mike > > On Wed, Sep 26, 2018 at 7:17 AM Shane Ardell > wrote: > > > I think Tibor's idea of using PCAP tests as an introduction to Cypress > for > > Metron is a great idea. As he pointed out, PCAP tests can take advantage > of > > Cypress' capability to mock responses, and we can set it up to run in > > Travis. Once the community is able to see the benefits from an actual set > > of Cypress tests inside the project and running in Travis, I think any > > questions about migrating the rest of the existing tests from Protractor > to > > Cypress will be settled. However, if for some reason we run into issues > > implementing or running the tests, we will have invested a fraction of > time > > vs. migrating all the tests right away. > > > > On Wed, Sep 26, 2018 at 2:12 PM Tibor Meller > > wrote: > > > > > Hi Team, > > > > > > Many of us agreed on that Cypress could be a more capable tool for us > to > > > write high-level UI tests, whether those be e2e, integration or > automated > > > regression tests. If there is no open question left about cypress we > > could > > > to bring it a test drive. My suggestion is to implement the PCAP UI > tests > > > with Cypress. Some services and PCAP semple data yet not available from > > our > > > CI environment so protractor is hardly applicable here. This would be a > > > great opportunity for cypress to shine. With Cypress, we are able to > mock > > > out those responses and make it run in Travis. > > > Anytime we make PCAP data available in Travis we could be able to plug > > out > > > those mocks and run the same test as integration or e2e tests if we > like. > > > > > > Because it is relatively easy to migrate across cypress and protractor > I > > > see no major risks here if we decide to stick with Protractor for some > > > reason. > > > > > > What do you think? > > > > > > Thanks for your feedback, > > > Tibor > > > > > > On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell > > > > wrote: > > > > > > > Hello everyone, > > > > > > > > Currently, we use Protractor to run our UI "end-to-end" tests. > However, > > > > there are a handful of major advantages we can gain from switching to > > > > Cypress: https://www.cypress.io/features/. > > > > > > > >- As with most Selenium-based e2e testing frameworks, Protractor > > > suffers > > > >from test flakiness. This is because Selenium runs outside of the > > > > browser > > > >and executes remote commands across the network. To work around > this > > > at > > > > the > > > >moment, we are using protractor-flake to re-run failed tests, but > > this > > > > is > > > >more of a crutch than a fix. Cypress executes in the same run loop > > as > > > > the > > > >application it's testing, and as a result does not suffer from the > > > same > > > >flakiness. > > > >- As a result of its architecture, Cypress runs much faster than > > > >Protractor. This is especially critical if e2e tests are added to > > the > > > CI > > > >build in the future. > > > >- Protractor is incredibly hard to debug. In contrast, Cypress > comes > > > >with a plethora of debugging features, some of which you can see > in > > > > action > > > >here: https://vimeo.com/242961930#t=264s > > > > > > > > Does anyone else have thoughts or opinions on switching to Cypress or > > > > staying with Protractor? > > > > > > > > Cheers, > > > > Shane > > > > > > > > > >
Re: [DISCUSS] Migrate from Protractor to Cypress
I'm good with it. We can see some tests in action (and hopefully running in Travis! :-D) and then migrate and deprecate Protractor accordingly if we still agree that's the way to go. When you submit the first PR, please link to this DISCUSS via permalink from the mailing list archives. Thanks guys. Cheers, Mike On Wed, Sep 26, 2018 at 7:17 AM Shane Ardell wrote: > I think Tibor's idea of using PCAP tests as an introduction to Cypress for > Metron is a great idea. As he pointed out, PCAP tests can take advantage of > Cypress' capability to mock responses, and we can set it up to run in > Travis. Once the community is able to see the benefits from an actual set > of Cypress tests inside the project and running in Travis, I think any > questions about migrating the rest of the existing tests from Protractor to > Cypress will be settled. However, if for some reason we run into issues > implementing or running the tests, we will have invested a fraction of time > vs. migrating all the tests right away. > > On Wed, Sep 26, 2018 at 2:12 PM Tibor Meller > wrote: > > > Hi Team, > > > > Many of us agreed on that Cypress could be a more capable tool for us to > > write high-level UI tests, whether those be e2e, integration or automated > > regression tests. If there is no open question left about cypress we > could > > to bring it a test drive. My suggestion is to implement the PCAP UI tests > > with Cypress. Some services and PCAP semple data yet not available from > our > > CI environment so protractor is hardly applicable here. This would be a > > great opportunity for cypress to shine. With Cypress, we are able to mock > > out those responses and make it run in Travis. > > Anytime we make PCAP data available in Travis we could be able to plug > out > > those mocks and run the same test as integration or e2e tests if we like. > > > > Because it is relatively easy to migrate across cypress and protractor I > > see no major risks here if we decide to stick with Protractor for some > > reason. > > > > What do you think? > > > > Thanks for your feedback, > > Tibor > > > > On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell > > wrote: > > > > > Hello everyone, > > > > > > Currently, we use Protractor to run our UI "end-to-end" tests. However, > > > there are a handful of major advantages we can gain from switching to > > > Cypress: https://www.cypress.io/features/. > > > > > >- As with most Selenium-based e2e testing frameworks, Protractor > > suffers > > >from test flakiness. This is because Selenium runs outside of the > > > browser > > >and executes remote commands across the network. To work around this > > at > > > the > > >moment, we are using protractor-flake to re-run failed tests, but > this > > > is > > >more of a crutch than a fix. Cypress executes in the same run loop > as > > > the > > >application it's testing, and as a result does not suffer from the > > same > > >flakiness. > > >- As a result of its architecture, Cypress runs much faster than > > >Protractor. This is especially critical if e2e tests are added to > the > > CI > > >build in the future. > > >- Protractor is incredibly hard to debug. In contrast, Cypress comes > > >with a plethora of debugging features, some of which you can see in > > > action > > >here: https://vimeo.com/242961930#t=264s > > > > > > Does anyone else have thoughts or opinions on switching to Cypress or > > > staying with Protractor? > > > > > > Cheers, > > > Shane > > > > > >
Re: [DISCUSS] Migrate from Protractor to Cypress
I think Tibor's idea of using PCAP tests as an introduction to Cypress for Metron is a great idea. As he pointed out, PCAP tests can take advantage of Cypress' capability to mock responses, and we can set it up to run in Travis. Once the community is able to see the benefits from an actual set of Cypress tests inside the project and running in Travis, I think any questions about migrating the rest of the existing tests from Protractor to Cypress will be settled. However, if for some reason we run into issues implementing or running the tests, we will have invested a fraction of time vs. migrating all the tests right away. On Wed, Sep 26, 2018 at 2:12 PM Tibor Meller wrote: > Hi Team, > > Many of us agreed on that Cypress could be a more capable tool for us to > write high-level UI tests, whether those be e2e, integration or automated > regression tests. If there is no open question left about cypress we could > to bring it a test drive. My suggestion is to implement the PCAP UI tests > with Cypress. Some services and PCAP semple data yet not available from our > CI environment so protractor is hardly applicable here. This would be a > great opportunity for cypress to shine. With Cypress, we are able to mock > out those responses and make it run in Travis. > Anytime we make PCAP data available in Travis we could be able to plug out > those mocks and run the same test as integration or e2e tests if we like. > > Because it is relatively easy to migrate across cypress and protractor I > see no major risks here if we decide to stick with Protractor for some > reason. > > What do you think? > > Thanks for your feedback, > Tibor > > On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell > wrote: > > > Hello everyone, > > > > Currently, we use Protractor to run our UI "end-to-end" tests. However, > > there are a handful of major advantages we can gain from switching to > > Cypress: https://www.cypress.io/features/. > > > >- As with most Selenium-based e2e testing frameworks, Protractor > suffers > >from test flakiness. This is because Selenium runs outside of the > > browser > >and executes remote commands across the network. To work around this > at > > the > >moment, we are using protractor-flake to re-run failed tests, but this > > is > >more of a crutch than a fix. Cypress executes in the same run loop as > > the > >application it's testing, and as a result does not suffer from the > same > >flakiness. > >- As a result of its architecture, Cypress runs much faster than > >Protractor. This is especially critical if e2e tests are added to the > CI > >build in the future. > >- Protractor is incredibly hard to debug. In contrast, Cypress comes > >with a plethora of debugging features, some of which you can see in > > action > >here: https://vimeo.com/242961930#t=264s > > > > Does anyone else have thoughts or opinions on switching to Cypress or > > staying with Protractor? > > > > Cheers, > > Shane > > >
Re: [DISCUSS] Migrate from Protractor to Cypress
Hi Team, Many of us agreed on that Cypress could be a more capable tool for us to write high-level UI tests, whether those be e2e, integration or automated regression tests. If there is no open question left about cypress we could to bring it a test drive. My suggestion is to implement the PCAP UI tests with Cypress. Some services and PCAP semple data yet not available from our CI environment so protractor is hardly applicable here. This would be a great opportunity for cypress to shine. With Cypress, we are able to mock out those responses and make it run in Travis. Anytime we make PCAP data available in Travis we could be able to plug out those mocks and run the same test as integration or e2e tests if we like. Because it is relatively easy to migrate across cypress and protractor I see no major risks here if we decide to stick with Protractor for some reason. What do you think? Thanks for your feedback, Tibor On Wed, Sep 19, 2018 at 1:49 PM Shane Ardell wrote: > Hello everyone, > > Currently, we use Protractor to run our UI "end-to-end" tests. However, > there are a handful of major advantages we can gain from switching to > Cypress: https://www.cypress.io/features/. > >- As with most Selenium-based e2e testing frameworks, Protractor suffers >from test flakiness. This is because Selenium runs outside of the > browser >and executes remote commands across the network. To work around this at > the >moment, we are using protractor-flake to re-run failed tests, but this > is >more of a crutch than a fix. Cypress executes in the same run loop as > the >application it's testing, and as a result does not suffer from the same >flakiness. >- As a result of its architecture, Cypress runs much faster than >Protractor. This is especially critical if e2e tests are added to the CI >build in the future. >- Protractor is incredibly hard to debug. In contrast, Cypress comes >with a plethora of debugging features, some of which you can see in > action >here: https://vimeo.com/242961930#t=264s > > Does anyone else have thoughts or opinions on switching to Cypress or > staying with Protractor? > > Cheers, > Shane >
Re: [DISCUSS] Migrate from Protractor to Cypress
That's good feedback, thanks Shane! On Thu, Sep 20, 2018 at 6:23 AM Shane Ardell wrote: > While the Cypress team suggests taking advantage of stubs where you can, > especially during development, we would definitely be able to test real > endpoints [1]. It can be used exactly like how Protractor is used, but with > the many benefits and features it provides [2]. Cypress also offers tools > for unit testing [3], which I think may be causing confusion as to what > exactly the library does. Cypress' main focus is e2e tests, but because of > its architecture, it can be used for all types of tests. > > I agree with everything you mentioned, Mike. I think our approach now is > fine, but in the future I do think it's worth considering the Cypress > team's suggestions for when and when not to stub, but there are no hard and > fast rules [4][5]. > > I currently have a branch available on my fork where I've migrated over > some e2e tests from Protractor to Cypress. With the exception of a little > code cleanup, these tests perform the same steps as they do with > Protractor. I have yet to include instructions in the README or include an > npm script, but if anyone wants to see it in action they can do the > following: > >- download this branch: >https://github.com/sardell/metron/tree/METRON-1648, >- run `npm ci` from meron-alerts, >- start the e2e test server, >- run `./node_modules/.bin/cypress open` >- start a single test by clicking on a file name in the Cypress user >interface, or run them all by clicking the play button. > > I'll try to send some sort of benchmarks when I get a chance to show the > speed difference between the two libraries. > > [1] https://docs.cypress.io/api/commands/request.html > [2] https://www.cypress.io/features/ > [3] https://docs.cypress.io/guides/guides/stubs-spies-and-clocks.html > [4] > > https://docs.cypress.io/guides/guides/network-requests.html#Testing-Strategies > . > [5] > > https://docs.cypress.io/guides/getting-started/testing-your-app.html#Stubbing-the-Server > > On Thu, Sep 20, 2018 at 12:09 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > Shane, > > > > Can you elaborate on the testing model you're proposing? I looked through > > the overview and some of the documentation. As far as I can tell, this > > would effectively be and e2e test for the UI *only*, so we would be > missing > > testing the actual integration points with the REST API or any other > > potential endpoints. > > > >1. Are you proposing we migrate all existing e2e tests, including > those > >that currently hit Elasticsearch? > >2. Would shifting to Cypress mean that all e2e tests would be isolated > >to only what is rendered via the browser? i.e. our e2e suite is no > > longer > >testing integration to a backend? > > > > My assumption with the term e2e testing is that you are testing an entire > > vertical slice with no substantive mock/stub/fake/spy/dummy [1] in the > way > > except for maybe some strategic cross-cutting concerns. It sounds like > > Cypress does NOT mean full e2e. My initial reaction to this is that > there's > > a place for both forms of testing. If Cypress would help UI developers > work > > on incremental changes, similar to how unit tests via JUnit help Java > > developers iterate on features, then I think that's great. I'm all for > > that! But unit tests are only one form of testing - we also do > integration > > testing, which can flex multiple classes/components together, as well as > > more broad stack integration/functional testing that verifies everything > > works when integrated together. Generally speaking, total # of unit > tests > > > # of integration tests > # functional/acceptance tests. I think we should > > carve out and define a testing approach for each. Can you elaborate a bit > > on your vision for how to manage the test interactions, or lack thereof, > > with the REST API as an integration endpoint? [2] > > > > At the time the write-up James shared was written, it appears that > Cypress > > was not yet open source. Now, it's MIT license - > > https://github.com/cypress-io/cypress/blob/develop/LICENSE.md. > > > > Mike > > > > 1. > > > > > https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs > > 2. https://martinfowler.com/articles/practical-test-pyramid.html#UiTests > > > > > > On Wed, Sep 19, 2018 at 8:47 AM James Sirota wrote: > > > > > This article comparing the two is not favorable for Cypress. Are any > of > > > these concerns relevant to us? If not, then I think Cypress is fine > > > > > > > > > > > > https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7 > > > > > > > > >
Re: [DISCUSS] Migrate from Protractor to Cypress
While the Cypress team suggests taking advantage of stubs where you can, especially during development, we would definitely be able to test real endpoints [1]. It can be used exactly like how Protractor is used, but with the many benefits and features it provides [2]. Cypress also offers tools for unit testing [3], which I think may be causing confusion as to what exactly the library does. Cypress' main focus is e2e tests, but because of its architecture, it can be used for all types of tests. I agree with everything you mentioned, Mike. I think our approach now is fine, but in the future I do think it's worth considering the Cypress team's suggestions for when and when not to stub, but there are no hard and fast rules [4][5]. I currently have a branch available on my fork where I've migrated over some e2e tests from Protractor to Cypress. With the exception of a little code cleanup, these tests perform the same steps as they do with Protractor. I have yet to include instructions in the README or include an npm script, but if anyone wants to see it in action they can do the following: - download this branch: https://github.com/sardell/metron/tree/METRON-1648, - run `npm ci` from meron-alerts, - start the e2e test server, - run `./node_modules/.bin/cypress open` - start a single test by clicking on a file name in the Cypress user interface, or run them all by clicking the play button. I'll try to send some sort of benchmarks when I get a chance to show the speed difference between the two libraries. [1] https://docs.cypress.io/api/commands/request.html [2] https://www.cypress.io/features/ [3] https://docs.cypress.io/guides/guides/stubs-spies-and-clocks.html [4] https://docs.cypress.io/guides/guides/network-requests.html#Testing-Strategies . [5] https://docs.cypress.io/guides/getting-started/testing-your-app.html#Stubbing-the-Server On Thu, Sep 20, 2018 at 12:09 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Shane, > > Can you elaborate on the testing model you're proposing? I looked through > the overview and some of the documentation. As far as I can tell, this > would effectively be and e2e test for the UI *only*, so we would be missing > testing the actual integration points with the REST API or any other > potential endpoints. > >1. Are you proposing we migrate all existing e2e tests, including those >that currently hit Elasticsearch? >2. Would shifting to Cypress mean that all e2e tests would be isolated >to only what is rendered via the browser? i.e. our e2e suite is no > longer >testing integration to a backend? > > My assumption with the term e2e testing is that you are testing an entire > vertical slice with no substantive mock/stub/fake/spy/dummy [1] in the way > except for maybe some strategic cross-cutting concerns. It sounds like > Cypress does NOT mean full e2e. My initial reaction to this is that there's > a place for both forms of testing. If Cypress would help UI developers work > on incremental changes, similar to how unit tests via JUnit help Java > developers iterate on features, then I think that's great. I'm all for > that! But unit tests are only one form of testing - we also do integration > testing, which can flex multiple classes/components together, as well as > more broad stack integration/functional testing that verifies everything > works when integrated together. Generally speaking, total # of unit tests > > # of integration tests > # functional/acceptance tests. I think we should > carve out and define a testing approach for each. Can you elaborate a bit > on your vision for how to manage the test interactions, or lack thereof, > with the REST API as an integration endpoint? [2] > > At the time the write-up James shared was written, it appears that Cypress > was not yet open source. Now, it's MIT license - > https://github.com/cypress-io/cypress/blob/develop/LICENSE.md. > > Mike > > 1. > > https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs > 2. https://martinfowler.com/articles/practical-test-pyramid.html#UiTests > > > On Wed, Sep 19, 2018 at 8:47 AM James Sirota wrote: > > > This article comparing the two is not favorable for Cypress. Are any of > > these concerns relevant to us? If not, then I think Cypress is fine > > > > > > > https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7 > > > > >
Re: [DISCUSS] Migrate from Protractor to Cypress
Shane, Can you elaborate on the testing model you're proposing? I looked through the overview and some of the documentation. As far as I can tell, this would effectively be and e2e test for the UI *only*, so we would be missing testing the actual integration points with the REST API or any other potential endpoints. 1. Are you proposing we migrate all existing e2e tests, including those that currently hit Elasticsearch? 2. Would shifting to Cypress mean that all e2e tests would be isolated to only what is rendered via the browser? i.e. our e2e suite is no longer testing integration to a backend? My assumption with the term e2e testing is that you are testing an entire vertical slice with no substantive mock/stub/fake/spy/dummy [1] in the way except for maybe some strategic cross-cutting concerns. It sounds like Cypress does NOT mean full e2e. My initial reaction to this is that there's a place for both forms of testing. If Cypress would help UI developers work on incremental changes, similar to how unit tests via JUnit help Java developers iterate on features, then I think that's great. I'm all for that! But unit tests are only one form of testing - we also do integration testing, which can flex multiple classes/components together, as well as more broad stack integration/functional testing that verifies everything works when integrated together. Generally speaking, total # of unit tests > # of integration tests > # functional/acceptance tests. I think we should carve out and define a testing approach for each. Can you elaborate a bit on your vision for how to manage the test interactions, or lack thereof, with the REST API as an integration endpoint? [2] At the time the write-up James shared was written, it appears that Cypress was not yet open source. Now, it's MIT license - https://github.com/cypress-io/cypress/blob/develop/LICENSE.md. Mike 1. https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs 2. https://martinfowler.com/articles/practical-test-pyramid.html#UiTests On Wed, Sep 19, 2018 at 8:47 AM James Sirota wrote: > This article comparing the two is not favorable for Cypress. Are any of > these concerns relevant to us? If not, then I think Cypress is fine > > > https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7 > >
Re: [DISCUSS] Migrate from Protractor to Cypress
This article comparing the two is not favorable for Cypress. Are any of these concerns relevant to us? If not, then I think Cypress is fine https://hackernoon.com/cypress-io-vs-protractor-e2e-testing-battle-d124ece91dc7
