This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d4f757baca6 [SPARK-42110][SQL][TESTS] Reduce the number of repetition
in ParquetDeltaEncodingSuite.`random data test`
d4f757baca6 is described below
commit d4f757baca64b4b66dd8f4e0b09bf085cce34af5
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jan 18 14:51:29 2023 -0800
[SPARK-42110][SQL][TESTS] Reduce the number of repetition in
ParquetDeltaEncodingSuite.`random data test`
### What changes were proposed in this pull request?
`random data test` is consuming about 4 minutes in GitHub Action and worse
in some other environment.
### Why are the changes needed?
- https://github.com/apache/spark/actions/runs/3948081724/jobs/6757667891
```
ParquetDeltaEncodingInt
- random data test (1 minute, 51 seconds)
...
ParquetDeltaEncodingLong
...
- random data test (1 minute, 54 seconds)
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass the CIs.
Closes #39648 from dongjoon-hyun/SPARK-42110.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
index c56e867b8bc..1c7b7b14da3 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaEncodingSuite.scala
@@ -215,7 +215,7 @@ abstract class ParquetDeltaEncodingSuite[T] extends
ParquetCompatibilityTest
test("random data test") {
val maxSize = 1000
val data = allocDataArray(maxSize)
- for (round <- 0 until 100000) {
+ for (round <- 0 until 10) {
val size = random.nextInt(maxSize)
for (i <- 0 until size) {
data(i) = getNextRandom
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
