This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 231d3760fe5 [SPARK-39693][INFRA] Do Not Execute tpcds-1g-gen for
Benchmarks Other Than TPCDSQueryBenchmark
231d3760fe5 is described below
commit 231d3760fe587973e3c1699912015907d6b26766
Author: Kazuyuki Tanimura <[email protected]>
AuthorDate: Fri Jul 8 09:26:35 2022 +0900
[SPARK-39693][INFRA] Do Not Execute tpcds-1g-gen for Benchmarks Other Than
TPCDSQueryBenchmark
### What changes were proposed in this pull request?
Currently `tpcds-1g-gen` runs for any benchmarks even that do not require
TPC-DS data on Github Actions.
This PR proposes to skip running `tpcds-1g-gen` if the benchmark class does
not contain `TPCDSQueryBenchmark` or `*` based on the discussion on #37020
### Why are the changes needed?
This PR should save time to launch benchmarks on Github Actions
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Tested on Github Actions.
Closes #37120 from kazuyukitanimura/SPARK-39693.
Authored-by: Kazuyuki Tanimura <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.github/workflows/benchmark.yml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml
index 3170c7c6bb0..4a5fd661c78 100644
--- a/.github/workflows/benchmark.yml
+++ b/.github/workflows/benchmark.yml
@@ -59,6 +59,7 @@ jobs:
# Any TPC-DS related updates on this job need to be applied to tpcds-1g job
of build_and_test.yml as well
tpcds-1g-gen:
name: "Generate an input dataset for TPCDSQueryBenchmark with SF=1"
+ if: contains(github.event.inputs.class, 'TPCDSQueryBenchmark') ||
contains(github.event.inputs.class, '*')
runs-on: ubuntu-20.04
env:
SPARK_LOCAL_IP: localhost
@@ -113,6 +114,7 @@ jobs:
benchmark:
name: "Run benchmarks: ${{ github.event.inputs.class }} (JDK ${{
github.event.inputs.jdk }}, Scala ${{ github.event.inputs.scala }}, ${{
matrix.split }} out of ${{ github.event.inputs.num-splits }} splits)"
+ if: always()
needs: [matrix-gen, tpcds-1g-gen]
# Ubuntu 20.04 is the latest LTS. The next LTS is 22.04.
runs-on: ubuntu-20.04
@@ -158,6 +160,7 @@ jobs:
with:
java-version: ${{ github.event.inputs.jdk }}
- name: Cache TPC-DS generated data
+ if: contains(github.event.inputs.class, 'TPCDSQueryBenchmark') ||
contains(github.event.inputs.class, '*')
id: cache-tpcds-sf-1
uses: actions/cache@v2
with:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
