This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 1b48de5606fb [SPARK-47051][INFRA] Create a new test pipeline for
`yarn` and `connect`
1b48de5606fb is described below
commit 1b48de5606fbdb26b4459dee0aa94be6560ef14a
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Feb 14 21:12:26 2024 -0800
[SPARK-47051][INFRA] Create a new test pipeline for `yarn` and `connect`
### What changes were proposed in this pull request?
This PR aims to spin off `yarn` and `connect` as a new test pipeline for
the following:
- To stabilize more by off-loading
- To re-trigger easily in case of failures.
- To isolate `yarn` module change and avoid triggering other module's tests
like Kafka module.
- To isolate `connect` module change and avoid triggering other module's
tests like Kafka module.
### Why are the changes needed?
These two modules are known to be flaky in various GitHub Action CI
pipelines.
- https://github.com/apache/spark/actions/runs/7905202256/job/21577289425
(`YarnClusterSuite`)
- https://github.com/apache/spark/actions/runs/7905202256/job/21585092863
(`SparkSessionE2ESuite`)
- https://github.com/apache/spark/actions/runs/7828944523/job/21359886644
(`SparkSessionE2ESuite`)
- https://github.com/apache/spark/actions/runs/7795415730/job/21258341216
(`SparkSessionE2ESuite`)
- https://github.com/apache/spark/actions/runs/7858754074/job/21444107806
(`SparkSessionE2ESuite`)
- https://github.com/apache/spark/actions/runs/7879934827/job/21501133320
(`SparkConnectServiceSuite`)
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs in this PR.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45107 from dongjoon-hyun/SPARK-47051.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 1d98727a4231..43903d139d1f 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -147,8 +147,9 @@ jobs:
mllib-local, mllib, graphx
- >-
streaming, sql-kafka-0-10, streaming-kafka-0-10,
streaming-kinesis-asl,
- yarn, kubernetes, hadoop-cloud, spark-ganglia-lgpl,
- connect, protobuf
+ kubernetes, hadoop-cloud, spark-ganglia-lgpl, protobuf
+ - >-
+ yarn, connect
# Here, we split Hive and SQL tests into some of slow ones and the
rest of them.
included-tags: [""]
excluded-tags: [""]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
