This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 4bc6e19dde0 [SPARK-39576][INFRA] Support GitHub Actions generate benchmark results using Scala 2.13 4bc6e19dde0 is described below commit 4bc6e19dde0eae5d100b7bfdfcf22e719fd59cb5 Author: yangjie01 <yangji...@baidu.com> AuthorDate: Fri Jun 24 09:09:34 2022 -0700 [SPARK-39576][INFRA] Support GitHub Actions generate benchmark results using Scala 2.13 ### What changes were proposed in this pull request? This pr aims let `benchmark` GitHub Actions support the specified Scala version, then it can produce benchmark results using Scala 2.13. ### Why are the changes needed? Help us check the microbenchmark results using Scala 2.13 and ensure they are not slower than using Scala 2.12 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass Github Actions - Closes #36975 from LuciferYang/213-bench. Authored-by: yangjie01 <yangji...@baidu.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- .github/workflows/benchmark.yml | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 91e168210fb..a322fe065b5 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -30,6 +30,10 @@ on: description: 'JDK version: 8, 11 or 17' required: true default: '8' + scala: + description: 'Scala version: 2.12 or 2.13' + required: true + default: '2.12' failfast: description: 'Failfast: true or false' required: true @@ -53,7 +57,7 @@ jobs: run: echo "::set-output name=matrix::["`seq -s, 1 $SPARK_BENCHMARK_NUM_SPLITS`"]" benchmark: - name: "Run benchmarks: ${{ github.event.inputs.class }} (JDK ${{ github.event.inputs.jdk }}, ${{ matrix.split }} out of ${{ github.event.inputs.num-splits }} splits)" + name: "Run benchmarks: ${{ github.event.inputs.class }} (JDK ${{ github.event.inputs.jdk }}, Scala ${{ github.event.inputs.scala }}, ${{ matrix.split }} out of ${{ github.event.inputs.num-splits }} splits)" needs: matrix-gen # Ubuntu 20.04 is the latest LTS. The next LTS is 22.04. runs-on: ubuntu-20.04 @@ -99,7 +103,8 @@ jobs: java-version: ${{ github.event.inputs.jdk }} - name: Run benchmarks run: | - ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pspark-ganglia-lgpl test:package + dev/change-scala-version.sh ${{ github.event.inputs.scala }} + ./build/sbt -Pscala-${{ github.event.inputs.scala }} -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pspark-ganglia-lgpl test:package # Make less noisy cp conf/log4j2.properties.template conf/log4j2.properties sed -i 's/rootLogger.level = info/rootLogger.level = warn/g' conf/log4j2.properties @@ -109,13 +114,15 @@ jobs: --jars "`find . -name '*-SNAPSHOT-tests.jar' -o -name '*avro*-SNAPSHOT.jar' | paste -sd ',' -`" \ "`find . -name 'spark-core*-SNAPSHOT-tests.jar'`" \ "${{ github.event.inputs.class }}" + # Revert to default Scala version to clean up unnecessary git diff + dev/change-scala-version.sh 2.12 # To keep the directory structure and file permissions, tar them # See also https://github.com/actions/upload-artifact#maintaining-file-permissions-and-case-sensitive-files echo "Preparing the benchmark results:" - tar -cvf benchmark-results-${{ github.event.inputs.jdk }}.tar `git diff --name-only` `git ls-files --others --exclude-standard` + tar -cvf benchmark-results-${{ github.event.inputs.jdk }}-${{ github.event.inputs.scala }}.tar `git diff --name-only` `git ls-files --others --exclude-standard` - name: Upload benchmark results uses: actions/upload-artifact@v2 with: - name: benchmark-results-${{ github.event.inputs.jdk }}-${{ matrix.split }} - path: benchmark-results-${{ github.event.inputs.jdk }}.tar + name: benchmark-results-${{ github.event.inputs.jdk }}-${{ github.event.inputs.scala }}-${{ matrix.split }} + path: benchmark-results-${{ github.event.inputs.jdk }}-${{ github.event.inputs.scala }}.tar --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org