robdiciuccio opened a new pull request #13189: URL: https://github.com/apache/superset/pull/13189
### SUMMARY Github Actions that provide the ability to create an ephemeral environment running a PR's latest merge commit via a command executed via PR comment. The Docker build is run in a virtual environment created via [AWS ECS](https://aws.amazon.com/ecs/), using the [Fargate](https://aws.amazon.com/fargate/) container runtime. This functionality is sponsored by @preset-io. - Adds an additional Docker build target (**ci**) which includes additional scripts to initialize and run the container instance - Builds the Docker **ci** target on `pull_request` events (utilizing the cache from previous `docker build` runs) and saves the build as a [workflow artifact](https://docs.github.com/en/actions/guides/storing-workflow-data-as-artifacts). - Adds a **Push ephmereral env image** (`workflow_run`) workflow to tag and push the Docker **ci** image to [AWS ECR](https://aws.amazon.com/ecr/). - Adds an **Ephemeral env workflow** (`issue_comment`) workflow to evaluate PR comments for the `/testenv up` or `/testenv down` commands and deploy ECS services accordingly. - Adds new secrets to the repo: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. These credentials correspond to an AWS IAM user with appropriate ECS privileges. An Apache INFRA ticket is pending to add these secrets to the repo.  ### Running - When a PR is opened or updated, a Docker **ci** build of the merge commit will be saved and uploaded to ECR. - Posting a comment on the PR containing `/testenv up` (and nothing else) will trigger the ephemeral environment creation or update. _Note: this functionality is currently restricted to members of the **apache** Github organization._ - When the environment is created, a comment will be posted to the PR with the relevant details: <img width="926" alt="Screen Shot 2021-02-17 at 12 34 15 PM" src="https://user-images.githubusercontent.com/296227/108267592-4080b100-7120-11eb-8de6-35f6e452af49.png";> - Note that the environment will take several minutes to become available, as all DB migrations and example data loading is performed upon startup. - If the environment fails to startup, for example, if the Docker image is not yet built and pushed to ECR, a comment will also be posted to the PR: <img width="925" alt="Screen Shot 2021-02-17 at 1 04 21 PM" src="https://user-images.githubusercontent.com/296227/108267928-b08f3700-7120-11eb-8c9b-99ed915ce095.png";> ### Security The enclosed workflows follow [Github's security guidelines](https://securitylab.github.com/research/github-actions-preventing-pwn-requests) for builds. Docker images are built via the read-only `pull_request` event, preventing malicious code from accessing tokens with repo write access. `workflow_run` and `issue_comment` privileged workflows are limited to uploading and interacting with the AWS API, not building or running any of the untrusted code. ### TEST PLAN - Follow the procedure in [Running](#Running), above. _Note that the `workflow_run` and `issue_comment` workflows will not run until they are merged to `master`._ - These workflows have been tested pretty extensively on [my fork of Superset](https://github.com/robdiciuccio/incubator-superset/pulls). ### TODO (in future PRs) - Add handler for `/testenv down` command to manually shutdown environments - Automate shutting down server resources after ~48h - Shutdown and cleanup on PR close ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Changes UI - [ ] Requires DB Migration. - [ ] Confirm DB Migration upgrade and downgrade tested. - [ ] Introduces new feature or API - [ ] Removes existing feature or API ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/296227/108267592-4080b100-7120-11eb-8de6-35f6e452af49.png"](https://user-images.githubusercontent.com/296227/108267592-4080b100-7120-11eb-8de6-35f6e452af49.png"){kind=link}
![https://user-images.githubusercontent.com/296227/108267928-b08f3700-7120-11eb-8c9b-99ed915ce095.png"](https://user-images.githubusercontent.com/296227/108267928-b08f3700-7120-11eb-8c9b-99ed915ce095.png"){kind=link}