This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 0d3768293bf [SPARK-42110][SQL][TESTS] Reduce the number of repetition
in ParquetDeltaEncodingSuite.`random data test`
0d3768293bf is described below
commit 0d3768293bf86135fbace3997acc5af43df0175f
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jan 18 14:51:29 2023 -0800
[SPARK-42110][SQL][TESTS] Reduce the number of repetition in
ParquetDeltaEncodingSuite.`random data test`
### What changes were proposed in this pull request?
`random data test` is consuming about 4 minutes in GitHub Action and worse
in some other environment.
### Why are the changes needed?
- https://github.com/apache/spark/actions/runs/3948081724/jobs/6757667891
```
ParquetDeltaEncodingInt
- random data test (1 minute, 51 seconds)
...
ParquetDeltaEncodingLong
...
- random data test (1 minute, 54 seconds)
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass the CIs.
Closes #39648 from dongjoon-hyun/SPARK-42110.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit d4f757baca64b4b66dd8f4e0b09bf085cce34af5)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
index 844fa543145..ac82d66fbd4 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
@@ -215,7 +215,7 @@ abstract class ParquetDeltaEncodingSuite[T] extends
ParquetCompatibilityTest
test("random data test") {
val maxSize = 1000
val data = allocDataArray(maxSize)
- for (round <- 0 until 100000) {
+ for (round <- 0 until 10) {
val size = random.nextInt(maxSize)
for (i <- 0 until size) {
data(i) = getNextRandom
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
