zentol commented on a change in pull request #16332:
URL: https://github.com/apache/flink/pull/16332#discussion_r662803072
##########
File path: tools/azure-pipelines/jobs-template.yml
##########
@@ -160,6 +176,13 @@ jobs:
targetPath: $(DEBUG_FILES_OUTPUT_DIR)
artifact: logs-${{parameters.stage_name}}-$(DEBUG_FILES_NAME)
+ - script: |
+ mkdir -p $(TESTCONTAINER_CACHE_FOLDER)
+ docker save $(docker image ls --format "{{.Repository}}:{{.Tag}}") -o
$(TESTCONTAINER_CACHE_FOLDER)/cache.tar || true
Review comment:
I'm wondering if we'd should filter some of these out. Even the [core
stage](https://dev.azure.com/Autophagy/Flink-CI/_build/results?buildId=80&view=logs&j=9dc1b5dc-bcfa-5f83-eaa7-0cb181ddc267&t=95a75a7f-839b-5434-cae3-c0698e9852d8),
which doesn't use test containers at all still caches 4-5gigs. (consuming 8
minutes in the process)
##########
File path: tools/azure-pipelines/build-apache-repo.yml
##########
@@ -47,6 +47,9 @@ variables:
MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
CACHE_KEY: maven | $(Agent.OS) | **/pom.xml, !**/target/**
CACHE_FALLBACK_KEY: maven | $(Agent.OS)
+ TESTCONTAINER_CACHE_KEY: testcontainer-cache | $(Agent.OS) | **/pom.xml,
!**/target/**
Review comment:
```suggestion
TESTCONTAINER_CACHE_KEY: testcontainer-cache | $(Agent.OS)
```
The used images are usually unrelated to the poms, so I'm not sure whether
using the poms as a key makes sense.
##########
File path: tools/azure-pipelines/jobs-template.yml
##########
@@ -129,6 +129,22 @@ jobs:
condition: not(eq('${{parameters.test_pool_definition.name}}', 'Default'))
displayName: Cache Maven local repo
+ - task: Cache@2
+ inputs:
+ key: '"$(module)" | $(TESTCONTAINER_CACHE_KEY)'
+ restoreKeys: '"$(module)" | $(TESTCONTAINER_CACHE_FALLBACK_KEY)'
+ path: $(TESTCONTAINER_CACHE_FOLDER)
+ cacheHitVar: TESTCONTAINER_CACHE_HIT
+ continueOnError: true
+ condition: not(eq('${{parameters.test_pool_definition.name}}', 'Default'))
+ displayName: Cache Testcontainer docker images
+
+ - script: |
+ docker load -i $(TESTCONTAINER_CACHE_FOLDER)/cache.tar || true
+ displayName: Restore Testcontainer docker images
+ condition: and(not(canceled()), eq(variables.TESTCONTAINER_CACHE_HIT,
'true'))
+ continueOnError: true
Review comment:
Based on [this
build](https://dev.azure.com/Autophagy/Flink-CI/_build/results?buildId=80&view=logs&j=9dc1b5dc-bcfa-5f83-eaa7-0cb181ddc267&t=b1670b3c-3f19-5f34-3e91-67ac7d80384f)
this doesn't seem to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
