limited commented on issue #3056: Add experimental end-to-end test framework URL: https://github.com/apache/trafficcontrol/pull/3056#issuecomment-443776902 @rob05c Here are a few really rough notes, didn't have time to write full sentences and explanations around them Components of an end to end test bed Goals - Well documented and extensible - must be very easy for newcomers to 1) run existing tests and 2) add a test. - Speed - There may eventually be hundreds/thousands of tests in here. Three days for a full run is too long- most tests should run on the order of seconds including setup and teardown. Setup - Minimize external dependencies - Separate test execution from resources - Flexibility is still important, should be able to run against existing testbed (bare metal) or create VMs if needed without impacting the test’s topologies and services (really useful for CI or getting new users started with the framework) - For a CDN a complete framework might be able to create and manipulate servers, create caches, DS, even origin servers and content - Failure injection is critical, even into external systems like DNS and the network. Test Definition Prerequisities - Can individual test require certain CDN topology and content be present. Can framework create resources if not present (pytest does this pretty well with its idea of fixtures) Execution - *Lots* of common actions between, i.e making HTTP or DNS requests. Should be very simple to make a request but extensible if needed. i.e. default headers are necessary, but easy to add new headers. Similarly it should be really easy to make sure an HTTP request succeeded. If needed, more detailed validation should be possible if the test-writer needs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
