potiuk commented on code in PR #65: URL: https://github.com/apache/airflow-ci-infra/pull/65#discussion_r4353978165
########## terraform/runs-on/main.tf: ########## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +data "aws_partition" "current" {} +data "aws_caller_identity" "current" {} + +# The license key is never in Terraform state: RunsOn accepts LicenseKey as the ARN of an SSM +# SecureString parameter that MUST be named /<stack name>/license-key, and only the stack's +# service role decrypts it. The parameter is created by hand (see README.rst); this lookup +# without decryption only proves it exists before the stack is planned, storing the ciphertext. +data "aws_ssm_parameter" "license_key" { + name = "/${var.stack_name}/license-key" + with_decryption = false +} + +resource "aws_cloudformation_stack" "runs_on" { + name = var.stack_name + template_url = "https://runs-on.s3.eu-west-1.amazonaws.com/cloudformation/template-${var.runs_on_version}.yaml"; + capabilities = ["CAPABILITY_IAM", "CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND"] + + # Everything not listed keeps the template default; the values match the stack validated on + # 2026-09-25 (see the AIP-118 appendix) apart from the organization and the CIDR. + parameters = { + GithubOrganization = var.github_organization + LicenseKey = data.aws_ssm_parameter.license_key.arn + EmailAddress = var.alert_email + Environment = "production" + VpcCidrBlock = var.vpc_cidr + Private = "false" # runners get public IPs: no NAT cost, Docker Hub pulls per runner IP + SSHAllowed = "false" + SSMAllowed = "true" Review Comment: This keeps `/setup` reachable on the public API Gateway for the life of the stack, and every apply turns it back on. Could we make it a variable defaulting to `false`, set to `true` only for the GitHub App registration (and for re-registering if that's ever needed)? ########## terraform/runs-on/README.rst: ########## @@ -0,0 +1,165 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +RunsOn — self-hosted CI runners on AWS +====================================== + +`RunsOn <https://runs-on.com/>`_ is the approach selected by +`AIP-118 <https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974652>`_ for Airflow's +self-hosted GitHub Actions runners. It packages runner provisioning into a CloudFormation stack in +Airflow's AWS account: a small control plane (ECS Fargate + API Gateway) receives GitHub's +``workflow_job`` webhooks and launches one ephemeral EC2 instance per job, Spot first, sized from +the job's ``runs-on:`` label. RunsOn's `free open-source license <https://runs-on.com/billing/free/>`_ +covers the product; EC2/EBS/S3 usage is billed to the account. + +This module creates the stack for the **apache** organization from RunsOn's published, +version-pinned template, with the configuration validated on 2026-09-25 (full ``Tests (AMD)`` +graph green apart from two non-hermetic tests — see the AIP-118 appendix). + +Files +----- + +``main.tf`` + The ``aws_cloudformation_stack`` resource, its parameters and the license-key lookup. +``variables.tf`` + All variables have defaults: ``stack_name`` (``runs-on``), ``github_organization`` + (``apache``), ``alert_email`` (``[email protected]``), ``runs_on_version`` (``v3.3.2``), + ``vpc_cidr`` (``10.2.0.0/16``) and ``region`` (``us-east-2``). +``state.tf`` + S3 backend ``airflow-ci-tfstate`` / ``tf-state/runs-on.tfstate`` (its own key next to the + EKS states). +``runs-on.yml.example`` + Runner presets for Airflow's three size classes, to be committed as ``.github/runs-on.yml`` + in the organization's ``.github-private`` repository (or a repository's default branch). + +Step 1 — create the license-key parameter (manual, once) +-------------------------------------------------------- + +RunsOn accepts ``LicenseKey`` either as plain text or as the ARN of an SSM Parameter Store +parameter that must be named ``/<stack name>/license-key`` (`reference +<https://runs-on.com/docs/maintenance/configuration/flex/cloudformation/>`_). The ARN form keeps +the key out of Terraform variables, plan output and state: ``main.tf`` reads the parameter +**without decryption** (only to prove it exists) and passes its ARN; the stack's service role is +the only principal that decrypts it. Secrets Manager is not supported by the template. + +.. code-block:: bash + + aws ssm put-parameter --region us-east-2 \ + --name /runs-on/license-key \ + --type SecureString \ + --description "RunsOn license key for the runs-on stack (GitHub organization apache)" \ + --value 'RUNS-ON-LICENSE-KEY' + +Use the default SSM KMS key (``alias/aws/ssm``): the template grants the stack's role access to +that key; a customer key would need an extra key policy. Never put the key in ``terraform.tfvars``. +Rotating the key is ``aws ssm put-parameter --overwrite`` followed by a stack update. + +The free open-source license is issued per GitHub organization; the key must be the one issued +for ``apache``. + +Step 2 — plan and apply +----------------------- + +For the first, local run, keep the shared S3 state out of the way with a gitignored override:: + + cd terraform/runs-on + printf 'terraform {\n backend "local" {}\n}\n' > backend_override.tf + terraform init + terraform plan -out=runs-on.tfplan + terraform apply runs-on.tfplan # ~10 minutes + terraform output + +The plan must show exactly one resource to create (``aws_cloudformation_stack.runs_on``) and read +one data source (the SSM parameter). Delete ``backend_override.tf`` and run ``terraform init +-migrate-state`` to move the state to S3 once the future GitHub Actions workflow takes over; Review Comment: Is the `apache` stack already applied with the local backend? If so, the committed S3 key is empty until someone runs `-migrate-state`. Anyone who runs `terraform init`/`plan` without the override in the meantime gets a plan to create the stack, and apply then fails with `AlreadyExistsException`. Meanwhile the only copy of the state is on one machine. Could we migrate it to S3 before merging, or at least say clearly here that the migration is pending? ########## terraform/runs-on/README.rst: ########## @@ -0,0 +1,165 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +RunsOn — self-hosted CI runners on AWS +====================================== + +`RunsOn <https://runs-on.com/>`_ is the approach selected by +`AIP-118 <https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974652>`_ for Airflow's +self-hosted GitHub Actions runners. It packages runner provisioning into a CloudFormation stack in +Airflow's AWS account: a small control plane (ECS Fargate + API Gateway) receives GitHub's +``workflow_job`` webhooks and launches one ephemeral EC2 instance per job, Spot first, sized from +the job's ``runs-on:`` label. RunsOn's `free open-source license <https://runs-on.com/billing/free/>`_ +covers the product; EC2/EBS/S3 usage is billed to the account. + +This module creates the stack for the **apache** organization from RunsOn's published, +version-pinned template, with the configuration validated on 2026-09-25 (full ``Tests (AMD)`` +graph green apart from two non-hermetic tests — see the AIP-118 appendix). + +Files +----- + +``main.tf`` + The ``aws_cloudformation_stack`` resource, its parameters and the license-key lookup. +``variables.tf`` + All variables have defaults: ``stack_name`` (``runs-on``), ``github_organization`` + (``apache``), ``alert_email`` (``[email protected]``), ``runs_on_version`` (``v3.3.2``), + ``vpc_cidr`` (``10.2.0.0/16``) and ``region`` (``us-east-2``). +``state.tf`` + S3 backend ``airflow-ci-tfstate`` / ``tf-state/runs-on.tfstate`` (its own key next to the + EKS states). +``runs-on.yml.example`` + Runner presets for Airflow's three size classes, to be committed as ``.github/runs-on.yml`` + in the organization's ``.github-private`` repository (or a repository's default branch). + +Step 1 — create the license-key parameter (manual, once) +-------------------------------------------------------- + +RunsOn accepts ``LicenseKey`` either as plain text or as the ARN of an SSM Parameter Store +parameter that must be named ``/<stack name>/license-key`` (`reference +<https://runs-on.com/docs/maintenance/configuration/flex/cloudformation/>`_). The ARN form keeps +the key out of Terraform variables, plan output and state: ``main.tf`` reads the parameter +**without decryption** (only to prove it exists) and passes its ARN; the stack's service role is +the only principal that decrypts it. Secrets Manager is not supported by the template. + +.. code-block:: bash + + aws ssm put-parameter --region us-east-2 \ + --name /runs-on/license-key \ + --type SecureString \ + --description "RunsOn license key for the runs-on stack (GitHub organization apache)" \ + --value 'RUNS-ON-LICENSE-KEY' + +Use the default SSM KMS key (``alias/aws/ssm``): the template grants the stack's role access to +that key; a customer key would need an extra key policy. Never put the key in ``terraform.tfvars``. +Rotating the key is ``aws ssm put-parameter --overwrite`` followed by a stack update. + Review Comment: After `put-parameter --overwrite`, `LicenseKey` is still the same ARN, so `terraform plan` shows no changes and Terraform never triggers a stack update. How is the "stack update" meant to happen? A restart of the ECS service, or a documented manual step? ########## terraform/runs-on/outputs.tf: ########## @@ -0,0 +1,34 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +output "entry_point" { + description = "RunsOn web entry point: open it once after creation to register the GitHub App." + value = aws_cloudformation_stack.runs_on.outputs["RunsOnEntryPoint"] +} + +output "cache_bucket" { + description = "S3 bucket the runners can use under the cache/ prefix (stash payloads)." + value = aws_cloudformation_stack.runs_on.outputs["RunsOnBucketCache"] +} + +output "app_version" { Review Comment: Nit: these two outputs have no `description`, unlike the other two. ########## terraform/runs-on/variables.tf: ########## @@ -0,0 +1,55 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +variable "region" { + description = "AWS region for the RunsOn stack and its runners." + type = string + default = "us-east-2" +} + +variable "stack_name" { + description = <<-EOT + CloudFormation stack name. RunsOn derives the license-key SSM parameter name from it + (/<stack_name>/license-key), so changing it means re-creating the parameter. + EOT + type = string + default = "runs-on" +} + +variable "runs_on_version" { + description = "RunsOn release to deploy; selects the version-pinned CloudFormation template." + type = string + default = "v3.3.2" +} + +variable "github_organization" { + description = "GitHub organization the runners serve (case-sensitive)." + type = string + default = "apache" +} + +variable "alert_email" { + description = "Recipient of RunsOn cost reports and alerts; AWS sends an SNS confirmation email on creation." + type = string + default = "[email protected]" Review Comment: Defaulting alerts, budget notices and cost reports to a personal address ties them to one person's inbox and SNS confirmation. Could this be a list address (for example a private@ list, or an ci-infra alias)? ########## terraform/runs-on/README.rst: ########## @@ -0,0 +1,165 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +RunsOn — self-hosted CI runners on AWS +====================================== + +`RunsOn <https://runs-on.com/>`_ is the approach selected by +`AIP-118 <https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974652>`_ for Airflow's +self-hosted GitHub Actions runners. It packages runner provisioning into a CloudFormation stack in +Airflow's AWS account: a small control plane (ECS Fargate + API Gateway) receives GitHub's +``workflow_job`` webhooks and launches one ephemeral EC2 instance per job, Spot first, sized from +the job's ``runs-on:`` label. RunsOn's `free open-source license <https://runs-on.com/billing/free/>`_ +covers the product; EC2/EBS/S3 usage is billed to the account. + +This module creates the stack for the **apache** organization from RunsOn's published, +version-pinned template, with the configuration validated on 2026-09-25 (full ``Tests (AMD)`` +graph green apart from two non-hermetic tests — see the AIP-118 appendix). + +Files +----- + +``main.tf`` + The ``aws_cloudformation_stack`` resource, its parameters and the license-key lookup. +``variables.tf`` + All variables have defaults: ``stack_name`` (``runs-on``), ``github_organization`` + (``apache``), ``alert_email`` (``[email protected]``), ``runs_on_version`` (``v3.3.2``), + ``vpc_cidr`` (``10.2.0.0/16``) and ``region`` (``us-east-2``). +``state.tf`` + S3 backend ``airflow-ci-tfstate`` / ``tf-state/runs-on.tfstate`` (its own key next to the + EKS states). +``runs-on.yml.example`` + Runner presets for Airflow's three size classes, to be committed as ``.github/runs-on.yml`` + in the organization's ``.github-private`` repository (or a repository's default branch). + +Step 1 — create the license-key parameter (manual, once) +-------------------------------------------------------- + +RunsOn accepts ``LicenseKey`` either as plain text or as the ARN of an SSM Parameter Store +parameter that must be named ``/<stack name>/license-key`` (`reference +<https://runs-on.com/docs/maintenance/configuration/flex/cloudformation/>`_). The ARN form keeps +the key out of Terraform variables, plan output and state: ``main.tf`` reads the parameter +**without decryption** (only to prove it exists) and passes its ARN; the stack's service role is +the only principal that decrypts it. Secrets Manager is not supported by the template. + +.. code-block:: bash + + aws ssm put-parameter --region us-east-2 \ + --name /runs-on/license-key \ + --type SecureString \ + --description "RunsOn license key for the runs-on stack (GitHub organization apache)" \ + --value 'RUNS-ON-LICENSE-KEY' + +Use the default SSM KMS key (``alias/aws/ssm``): the template grants the stack's role access to +that key; a customer key would need an extra key policy. Never put the key in ``terraform.tfvars``. +Rotating the key is ``aws ssm put-parameter --overwrite`` followed by a stack update. + +The free open-source license is issued per GitHub organization; the key must be the one issued +for ``apache``. + +Step 2 — plan and apply +----------------------- + +For the first, local run, keep the shared S3 state out of the way with a gitignored override:: + + cd terraform/runs-on + printf 'terraform {\n backend "local" {}\n}\n' > backend_override.tf + terraform init + terraform plan -out=runs-on.tfplan + terraform apply runs-on.tfplan # ~10 minutes + terraform output + +The plan must show exactly one resource to create (``aws_cloudformation_stack.runs_on``) and read +one data source (the SSM parameter). Delete ``backend_override.tf`` and run ``terraform init +-migrate-state`` to move the state to S3 once the future GitHub Actions workflow takes over; +the workflow only needs an OIDC role, nothing secret. + +Step 3 — register the GitHub App and the presets +------------------------------------------------ + +1. Open the ``entry_point`` output → **Register app**. This creates a private GitHub App in the + organization and installs it; for ``apache`` this is done together with ASF Infra, who also + grant the app access to ``apache/airflow`` and to ``.github-private`` if the presets live there. +2. Commit ``runs-on.yml.example`` as ``.github/runs-on.yml`` in ``apache/.github-private`` + (stack parameter ``RunnerConfigAutoExtendsFrom``, default ``.github-private``). RunsOn reads + ``.github/runs-on.yml`` in a public repository **only from its default branch**, so presets + cannot be trialled from a feature branch; the validation therefore spelled the same values + out in the labels. +3. Point jobs at the runners: + + .. code-block:: yaml + + runs-on: runs-on=${{ github.run_id }}/runner=airflow-amd-medium + + ``runs-on=<run id>`` is mandatory (it routes the job and namespaces the runner). Labels can + also carry everything inline, which is what the validation did: + ``runs-on=<id>/family=m7a+m7i+m6a+m6i/image=ubuntu22-full-x64/volume=150gb/cpu=8/ram=32``. + ``runs-on/action`` (per-job cost line in the job summary) is already on the ASF GitHub Actions + allowlist. + +Upgrades and changes +-------------------- + +Bump ``runs_on_version`` (or any parameter) and ``terraform plan`` / ``apply``: the resource +always resubmits the template URL, which is what RunsOn's `upgrade guide +<https://runs-on.com/docs/maintenance/upgrades/>`_ requires (a reused processed template can +silently skip parameter changes). Minor and patch releases have no downtime; major releases may. +Rollback is the previous ``runs_on_version``. Template ``v3.3.2`` has SHA-256 +``3017805d77c1d717fdd229a1e1242db12c985944699d3d8dca01f6f28b35c16d``. + +``prevent_destroy`` is set on the stack: a ``terraform destroy`` fails until the lifecycle block is +removed on purpose. Deleting the stack also needs its cache bucket emptied first if CloudFormation +stops on it. + +What the stack contains +----------------------- + +VPC ``10.2.0.0/16`` with public and private subnets (``Private=false`` → runners get public IPs and +no NAT, so each runner pulls from Docker Hub from its own address), an ECS Fargate service for the +control plane, an API Gateway entry point (``/``, ``/setup``, ``/readyz`` only), an S3 cache bucket +(``cache_bucket`` output; the runners' instance role can read and write the ``cache/`` prefix, +lifecycle ``S3CacheExpirationInDays``), the EC2 instance role and launch templates, DynamoDB +tables for locks and job tracking, an SNS alert topic, an AWS Budget (``AppBudgetDailyUsd``, +alert only — not a spending cap), the CloudWatch dashboard ``RunsOnDashboard-*`` (queue health, +scheduling trend, queue-duration P50/P90, Spot circuit breaker), log groups +``/aws/ecs/<stack>/runs-on-worker`` and ``<stack>-EC2InstanceLogGroup-*``, and a daily cost report +by email. + +Observability +------------- + +- CloudWatch dashboard and log groups above; ``roc`` CLI (`docs + <https://runs-on.com/docs/observability/cli/>`_): ``roc stack doctor``, ``roc stack logs``, + ``roc logs <job url> --watch``, ``roc connect <job url>`` over SSM (``SSMAllowed=true``). +- Optional OTLP export (``OtelExporterEndpoint`` / ``OtelExporterHeaders`` + ``extras=otel`` on the + label) to SigNoz or any OTLP backend; RunsOn publishes a SigNoz dashboard. Not deployed by + the stack. + +Things learned in the validation +-------------------------------- + +- Runner instances use IMDSv2 with a hop limit of 2, so Docker containers started by a job can + obtain the instance role's credentials. Tests that assume "no AWS credentials" must set Review Comment: Beyond test hermeticity, this means any job container can get the instance role's credentials, and that role can write `cache/`, including `cache/airflow-stash/`. If PRs from forks ever run on these runners, a PR could overwrite stash payloads that a later main-branch run restores. Do we plan to keep untrusted PRs off these runners, or to separate cache prefixes by trust level? Worth stating here either way. ########## terraform/runs-on/version.tf: ########## @@ -0,0 +1,27 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 6.0" + } + } + # 1.10+ for S3-native state locking (use_lockfile) + required_version = ">= 1.10.0" Review Comment: S3 native locking (`use_lockfile`) was experimental in 1.10 and became GA in 1.11. Since this state will be shared with a CI workflow, `>= 1.11.0` seems safer. ########## terraform/runs-on/README.rst: ########## @@ -0,0 +1,165 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +RunsOn — self-hosted CI runners on AWS +====================================== + +`RunsOn <https://runs-on.com/>`_ is the approach selected by +`AIP-118 <https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974652>`_ for Airflow's +self-hosted GitHub Actions runners. It packages runner provisioning into a CloudFormation stack in +Airflow's AWS account: a small control plane (ECS Fargate + API Gateway) receives GitHub's +``workflow_job`` webhooks and launches one ephemeral EC2 instance per job, Spot first, sized from +the job's ``runs-on:`` label. RunsOn's `free open-source license <https://runs-on.com/billing/free/>`_ +covers the product; EC2/EBS/S3 usage is billed to the account. + +This module creates the stack for the **apache** organization from RunsOn's published, +version-pinned template, with the configuration validated on 2026-09-25 (full ``Tests (AMD)`` +graph green apart from two non-hermetic tests — see the AIP-118 appendix). + +Files +----- + +``main.tf`` + The ``aws_cloudformation_stack`` resource, its parameters and the license-key lookup. +``variables.tf`` + All variables have defaults: ``stack_name`` (``runs-on``), ``github_organization`` + (``apache``), ``alert_email`` (``[email protected]``), ``runs_on_version`` (``v3.3.2``), + ``vpc_cidr`` (``10.2.0.0/16``) and ``region`` (``us-east-2``). +``state.tf`` + S3 backend ``airflow-ci-tfstate`` / ``tf-state/runs-on.tfstate`` (its own key next to the + EKS states). +``runs-on.yml.example`` + Runner presets for Airflow's three size classes, to be committed as ``.github/runs-on.yml`` + in the organization's ``.github-private`` repository (or a repository's default branch). + +Step 1 — create the license-key parameter (manual, once) +-------------------------------------------------------- + +RunsOn accepts ``LicenseKey`` either as plain text or as the ARN of an SSM Parameter Store +parameter that must be named ``/<stack name>/license-key`` (`reference +<https://runs-on.com/docs/maintenance/configuration/flex/cloudformation/>`_). The ARN form keeps +the key out of Terraform variables, plan output and state: ``main.tf`` reads the parameter +**without decryption** (only to prove it exists) and passes its ARN; the stack's service role is +the only principal that decrypts it. Secrets Manager is not supported by the template. + +.. code-block:: bash + + aws ssm put-parameter --region us-east-2 \ + --name /runs-on/license-key \ Review Comment: `var.region` looks configurable, but this command and the backend in `state.tf` both hard-code `us-east-2`. With any other region, the SSM lookup fails with `ParameterNotFound`. Either drop the variable or mention that the region has to be changed in these places too. ########## terraform/runs-on/main.tf: ########## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + Review Comment: These two data sources aren't used anywhere. The README (L87) also says the plan reads only one data source, the SSM parameter, so whoever follows that check will see three. I suggest removing them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
