ashb commented on code in PR #59: URL: https://github.com/apache/airflow-ci-infra/pull/59#discussion_r1273494873
########## docker/update-status: ########## Review Comment: Is there anyway we can use the files/iamges/builds directly from https://github.com/actions/actions-runner-controller/blob/dc58f6ba13403f1476a4de4463d27c3d3419bc3e/runner/update-status without having to copy it in to our tree? (I'm thinking drift over time and long term maintenance) ########## docker/Dockerfile: ########## Review Comment: Is this file based off of something else, or is it created by you from scratch? If the former a comment linking to the inspiration would be good. ########## terraform/eks/eks.tf: ########## @@ -0,0 +1,138 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +data "aws_availability_zones" "available" {} + +locals { + cluster_name = "airflow" +} + +module "eks" { + source = "terraform-aws-modules/eks/aws" + version = "19.15.3" + + cluster_name = local.cluster_name + cluster_version = "1.27" + + vpc_id = module.vpc.vpc_id + subnet_ids = module.vpc.private_subnets + + cluster_endpoint_public_access = true + + eks_managed_node_group_defaults = { + ami_type = "AL2_ARM_64" + use_custom_launch_template = false + disk_size = 50 + desired_size = 0 + } + + eks_managed_node_groups = { + + default_nodes = { + name = "default" + + instance_types = ["t4g.medium"] Review Comment: Our current runners are running on any of the following instance types -- which ever is the cheapest spot price r6a.2xlarge r5.2xlarge r5ad.2xlarge r5n.2xlarge r5b.2xlarge r5a.2xlarge r6i.2xlarge And we mount things as tmpfs using ~50% of the RAM (I thought): ``` - [tmpfs, /var/lib/docker, tmpfs, "defaults,noatime,size=85%"] - [tmpfs, /home/runner/actions-runner/_work, tmpfs, "defaults,noatime"] ``` ########## docker/Dockerfile: ########## Review Comment: Thinking more about this, rather than having the whole dockerfile here, could we instead do: ``` FROM summerwind/actions-runner/v2.306.0-ubuntu-22.04-e0a7e14 RUN ... # install our custom `runner` binary ```? ########## terraform/eks/vpc.tf: ########## @@ -0,0 +1,44 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# TODO: use the existing VPC? +module "vpc" { + source = "terraform-aws-modules/vpc/aws" + version = "5.0.0" + + name = "airflow-vpc" + + cidr = "10.1.0.0/16" + azs = slice(data.aws_availability_zones.available.names, 0, 3) + + private_subnets = ["10.1.1.0/24", "10.1.2.0/24", "10.1.3.0/24"] + public_subnets = ["10.1.4.0/24", "10.1.5.0/24", "10.1.6.0/24"] + + enable_nat_gateway = true Review Comment: NAT gateway is suprisingly costly -- we should run on public only subnets instead so that we don't have to put all the image pull trafic through a NAT gateway. ########## terraform/eks/provider.tf: ########## @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +provider "aws" { + profile = "airflow" + region = "us-east-2" Review Comment: Shouldn't this use the region variable? ########## docker/Dockerfile: ########## @@ -0,0 +1,159 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +FROM ubuntu:20.04 Review Comment: 22.04 is the current latest LTS -- any reason not to use that? ########## docker/Dockerfile: ########## @@ -0,0 +1,159 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +FROM ubuntu:20.04 + +ARG TARGETPLATFORM=linux/arm64 +ARG RUNNER_VERSION=2.304.0-airflow8 +ARG RUNNER_CONTAINER_HOOKS_VERSION=0.3.2 +ARG RUNNER_OWNER=ashb +# Docker and Docker Compose arguments +ARG CHANNEL=stable +ARG DOCKER_VERSION=20.10.23 +ARG DOCKER_COMPOSE_VERSION=v2.16.0 +ARG DUMB_INIT_VERSION=1.2.5 + +# Use 1001 and 121 for compatibility with GitHub-hosted runners +ARG RUNNER_UID=1000 +ARG DOCKER_GID=1001 + +ENV DEBIAN_FRONTEND=noninteractive +RUN apt-get update -y \ + && apt-get install -y software-properties-common \ + && add-apt-repository -y ppa:git-core/ppa \ + && apt-get update -y \ + && apt-get install -y --no-install-recommends \ + build-essential \ + curl \ + ca-certificates \ + dnsutils \ + ftp \ + git \ + git-lfs \ + iproute2 \ + iputils-ping \ + jq \ + libunwind8 \ + locales \ + netcat \ + openssh-client \ + parallel \ + python3-pip \ + rsync \ + shellcheck \ + sudo \ + telnet \ + time \ + tzdata \ + unzip \ + upx \ + wget \ + zip \ + zstd \ + && ln -sf /usr/bin/python3 /usr/bin/python \ + && ln -sf /usr/bin/pip3 /usr/bin/pip \ + && rm -rf /var/lib/apt/lists/* + +RUN adduser --disabled-password --gecos "" --uid $RUNNER_UID runner \ + && groupadd docker --gid $DOCKER_GID \ + && usermod -aG sudo runner \ + && usermod -aG docker runner \ + && echo "%sudo ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers \ + && echo "Defaults env_keep += \"DEBIAN_FRONTEND\"" >> /etc/sudoers + +ENV HOME=/home/runner + +RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \ Review Comment: Nit: Can we get this from `uname` instead of a parameter so that it always reflects the arch of the image we are building with no scope for being wrong? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
