HyukjinKwon commented on a change in pull request #29460:
URL: https://github.com/apache/spark/pull/29460#discussion_r471930843
##########
File path: .github/workflows/master.yml
##########
@@ -9,148 +9,242 @@ on:
- branch-3.0
jobs:
+ # Build: build Spark and run the tests for specified modules.
build:
-
+ name: "Build modules: ${{ matrix.modules }} ${{ matrix.comment }} (JDK ${{
matrix.java }}, ${{ matrix.hadoop }}, ${{ matrix.hive }})"
runs-on: ubuntu-latest
strategy:
+ fail-fast: false
matrix:
- java: [ '1.8', '11' ]
- hadoop: [ 'hadoop-2.7', 'hadoop-3.2' ]
- hive: [ 'hive-1.2', 'hive-2.3' ]
- exclude:
- - java: '11'
- hive: 'hive-1.2'
- - hadoop: 'hadoop-3.2'
- hive: 'hive-1.2'
- name: Build Spark - JDK${{ matrix.java }}/${{ matrix.hadoop }}/${{
matrix.hive }}
-
+ java:
+ - 1.8
+ hadoop:
+ - hadoop3.2
+ hive:
+ - hive2.3
+ # TODO(SPARK-32246): We don't test 'streaming-kinesis-asl' for now.
+ # Kinesis tests depends on external Amazon kinesis service.
+ # Note that the modules below are from sparktestsupport/modules.py.
+ modules:
+ - >-
+ core, unsafe, kvstore, avro,
+ network-common, network-shuffle, repl, launcher,
+ examples, sketch, graphx
+ - >-
+ catalyst, hive-thriftserver
+ - >-
+ streaming, sql-kafka-0-10, streaming-kafka-0-10,
+ mllib-local, mllib,
+ yarn, mesos, kubernetes, hadoop-cloud, spark-ganglia-lgpl
+ - >-
+ pyspark-sql, pyspark-mllib
Review comment:
`pyspark-resource` modules does not exist in Spark 3.0. It was added in
SPARK-29641.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
