This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 72b55ccf832 [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number
of partitions in TPC-DS build to avoid out-of-memory
72b55ccf832 is described below
commit 72b55ccf8327c00e173ab6130fdb428ad0d5aacc
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Tue Jul 26 15:03:17 2022 +0900
[SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number of partitions in
TPC-DS build to avoid out-of-memory
### What changes were proposed in this pull request?
This PR increases the number of partitions further more (see also
https://github.com/apache/spark/pull/37270)
### Why are the changes needed?
To make the build pass.
At least, two builds
(https://github.com/apache/spark/runs/7500542538?check_suite_focus=true and
https://github.com/apache/spark/runs/7511748355?check_suite_focus=true) passed
after https://github.com/apache/spark/pull/37273. I assume that the number of
partitions helps, and this PR increases some more.
### Does this PR introduce _any_ user-facing change?
No, test and dev-only.
### How was this patch tested?
It's tested in
https://github.com/LuciferYang/spark/runs/7497163716?check_suite_focus=true
Closes #37286 from HyukjinKwon/SPARK-39856-follwup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
index f3eaa898e59..9affe827bc1 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
@@ -62,7 +62,7 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase
with SQLQueryTestHelp
// To make output results deterministic
override protected def sparkConf: SparkConf = super.sparkConf
- .set(SQLConf.SHUFFLE_PARTITIONS.key, 16.toString)
+ .set(SQLConf.SHUFFLE_PARTITIONS.key, 32.toString)
protected override def createSparkSession: TestSparkSession = {
new TestSparkSession(new SparkContext("local[1]",
this.getClass.getSimpleName, sparkConf))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
