wangyum commented on issue #25877: [NO-NOT-MERGE][test-maven] Verify timeout for maven SparkPullRequestBuilder URL: https://github.com/apache/spark/pull/25877#issuecomment-533868013 Yes. We can reduce time by reducing shuffle partitions. `spark.sql.shuffle.partitions=200`(default): ``` [info] - subquery/in-subquery/in-set-operations.sql (1 minute, 32 seconds) ``` `spark.sql.shuffle.partitions=5`: ``` subquery/in-subquery/in-set-operations.sql (30 seconds, 390 milliseconds) ``` `spark.sql.shuffle.partitions=200`(default): ``` ... [info] - csv-functions.sql (1 second, 47 milliseconds) [info] - udaf.sql (705 milliseconds) [info] - null-handling.sql (8 seconds, 302 milliseconds) [info] - comparator.sql (407 milliseconds) [info] - string-functions.sql (3 seconds, 17 milliseconds) [info] - decimalArithmeticOperations.sql (6 seconds, 100 milliseconds) [info] - subquery/subquery-in-from.sql (1 second, 51 milliseconds) [info] - subquery/negative-cases/subq-input-typecheck.sql (477 milliseconds) [info] - subquery/negative-cases/invalid-correlation.sql (546 milliseconds) [info] - subquery/exists-subquery/exists-basic.sql (2 seconds, 916 milliseconds) [info] - subquery/exists-subquery/exists-orderby-limit.sql (7 seconds, 565 milliseconds) [info] - subquery/exists-subquery/exists-cte.sql (9 seconds, 958 milliseconds) [info] - subquery/exists-subquery/exists-joins-and-set-ops.sql (2 minutes, 42 seconds) [info] - subquery/exists-subquery/exists-aggregate.sql (11 seconds, 83 milliseconds) [info] - subquery/exists-subquery/exists-having.sql (10 seconds, 290 milliseconds) [info] - subquery/exists-subquery/exists-within-and-or.sql (1 second, 969 milliseconds) [info] - subquery/in-subquery/in-group-by.sql (52 seconds, 560 milliseconds) [info] - subquery/in-subquery/in-order-by.sql (40 seconds, 77 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-single-column.sql (2 seconds, 285 milliseconds) [info] - subquery/in-subquery/not-in-group-by.sql (12 seconds, 476 milliseconds) [info] - subquery/in-subquery/simple-in.sql (4 seconds, 461 milliseconds) [info] - subquery/in-subquery/in-basic.sql (1 second, 410 milliseconds) [info] - subquery/in-subquery/in-limit.sql (12 seconds, 211 milliseconds) [info] - subquery/in-subquery/in-set-operations.sql (1 minute, 32 seconds) [info] - subquery/in-subquery/in-with-cte.sql (30 seconds, 837 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-single-column-literal.sql (649 milliseconds) [info] - subquery/in-subquery/in-multiple-columns.sql (3 seconds, 981 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql (638 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-multi-column.sql (1 second, 779 milliseconds) [info] - subquery/in-subquery/in-having.sql (22 seconds, 496 milliseconds) ... ``` `spark.sql.shuffle.partitions=5`: ``` ... [info] - csv-functions.sql (794 milliseconds) [info] - udaf.sql (652 milliseconds) [info] - null-handling.sql (6 seconds, 103 milliseconds) [info] - comparator.sql (312 milliseconds) [info] - string-functions.sql (2 seconds, 429 milliseconds) [info] - decimalArithmeticOperations.sql (5 seconds, 422 milliseconds) [info] - subquery/subquery-in-from.sql (831 milliseconds) [info] - subquery/negative-cases/subq-input-typecheck.sql (384 milliseconds) [info] - subquery/negative-cases/invalid-correlation.sql (454 milliseconds) [info] - subquery/exists-subquery/exists-basic.sql (2 seconds, 525 milliseconds) [info] - subquery/exists-subquery/exists-orderby-limit.sql (4 seconds, 329 milliseconds) [info] - subquery/exists-subquery/exists-cte.sql (4 seconds, 902 milliseconds) [info] - subquery/exists-subquery/exists-joins-and-set-ops.sql (35 seconds, 851 milliseconds) [info] - subquery/exists-subquery/exists-aggregate.sql (4 seconds, 658 milliseconds) [info] - subquery/exists-subquery/exists-having.sql (3 seconds, 719 milliseconds) [info] - subquery/exists-subquery/exists-within-and-or.sql (1 second, 589 milliseconds) [info] - subquery/in-subquery/in-group-by.sql (16 seconds, 478 milliseconds) [info] - subquery/in-subquery/in-order-by.sql (12 seconds, 296 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-single-column.sql (1 second, 853 milliseconds) [info] - subquery/in-subquery/not-in-group-by.sql (4 seconds, 819 milliseconds) [info] - subquery/in-subquery/simple-in.sql (3 seconds, 513 milliseconds) [info] - subquery/in-subquery/in-basic.sql (1 second, 308 milliseconds) [info] - subquery/in-subquery/in-limit.sql (3 seconds, 181 milliseconds) [info] - subquery/in-subquery/in-set-operations.sql (30 seconds, 390 milliseconds) [info] - subquery/in-subquery/in-with-cte.sql (12 seconds, 766 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-single-column-literal.sql (550 milliseconds) [info] - subquery/in-subquery/in-multiple-columns.sql (3 seconds, 350 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql (575 milliseconds) [info] - subquery/in-subquery/not-in-unit-tests-multi-column.sql (1 second, 487 milliseconds) [info] - subquery/in-subquery/in-having.sql (7 seconds, 41 milliseconds) ... ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
