This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new b55e4c3e2a Increase size of ARM build instance (#24036)
b55e4c3e2a is described below
commit b55e4c3e2ac5bf3c7ea9f467447fa1c661da936d
Author: Jarek Potiuk <[email protected]>
AuthorDate: Tue May 31 10:15:03 2022 +0200
Increase size of ARM build instance (#24036)
Our ARM cache builds started to hang recently at yarn prod step.
The most likely reason are limited resources we had for the ARM
instance to run the docker build - it was rather small instance
with 2GB RAM and it is likely not nearly enought to cope with
recent changes related to Grid View where we likely need much
more memory during the yarn build step.
This change increases the instance memory to 8 GB (c6g.xlarge).
Also this instance type gives 70% cost saving and has very low
probability of being evicted (it's not in high demand in Ohio
Region of AWS.
Also the AMI used is refreshed with latest software (docker)
---
.github/workflows/ci.yml | 6 +++---
scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh | 4 ++--
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 4b82120db2..c5438eebb2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1653,7 +1653,7 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
permissions:
packages: write
timeout-minutes: 120
- name: "Push images as cache to GitHub Registry"
+ name: "Push Image Cache"
runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
needs:
- build-info
@@ -1687,7 +1687,7 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- name: "Start ARM instance"
run: ./scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
if: matrix.platform == 'linux/arm64'
- - name: "Build & Push CI image ${{ matrix.python-version }}:latest"
+ - name: "Push CI cache ${{ matrix.python-version }} ${{ matrix.platform
}}"
run: >
breeze build-image
--prepare-buildx-cache
@@ -1717,7 +1717,7 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
VERSION_SUFFIX_FOR_PYPI: "dev0"
- name: "Move dist packages to docker-context files"
run: mv -v ./dist/*.whl ./docker-context-files
- - name: Build & Push PROD image ${{ env.PYTHON_MAJOR_MINOR_VERSION
}}:latest
+ - name: "Push PROD cache ${{ matrix.python-version }} ${{
matrix.platform }}"
run: >
breeze build-prod-image
--airflow-is-in-context
diff --git a/scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
b/scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
index c65ecd7453..a3fbf6b5ba 100755
--- a/scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
+++ b/scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
@@ -22,8 +22,8 @@ SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"
# This is an AMI that is based on Basic Amazon Linux AMI with installed and
configured docker service
WORKING_DIR="/tmp/armdocker"
INSTANCE_INFO="${WORKING_DIR}/instance_info.json"
-ARM_AMI="ami-002fa24639ab2520a"
-INSTANCE_TYPE="c6gd.medium"
+ARM_AMI="ami-06b8158ea372d3259"
+INSTANCE_TYPE="c6g.xlarge"
MARKET_OPTIONS="MarketType=spot,SpotOptions={MaxPrice=0.1,SpotInstanceType=one-time}"
REGION="us-east-2"
EC2_USER="ec2-user"
