HyukjinKwon opened a new pull request, #45268: URL: https://github.com/apache/spark/pull/45268
### What changes were proposed in this pull request? This PR is a followup of https://github.com/apache/spark/pull/43494, which reduces the test cases by shuffling `taskNum`, and taking 5. ### Why are the changes needed? It runts too many test cases, https://github.com/apache/spark/actions/runs/8054862135/job/22000403549 which consumes the limited resources in CI ``` - SPARK-45527 default rp with task.gpu.amount=1.0 can restrict 1 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.5 can restrict 2 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.3333333333333333 can restrict 3 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.25 can restrict 4 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.2 can restrict 5 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1666666666666666 can restrict 6 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1428571428571428 can restrict 7 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.125 can restrict 8 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1111111111111111 can restrict 9 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1 can restrict 10 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0909090909090909 can restrict 11 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0833333333333333 can restrict 12 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0769230769230769 can restrict 13 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0714285714285714 can restrict 14 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0666666666666666 can restrict 15 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0625 can restrict 16 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0588235294117647 can restrict 17 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0555555555555555 can restrict 18 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0526315789473684 can restrict 19 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.05 can restrict 20 barrier tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=1.0 can restrict 1 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.5 can restrict 2 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.3333333333333333 can restrict 3 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.25 can restrict 4 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.2 can restrict 5 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1666666666666666 can restrict 6 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1428571428571428 can restrict 7 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.125 can restrict 8 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1111111111111111 can restrict 9 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.1 can restrict 10 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0909090909090909 can restrict 11 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0833333333333333 can restrict 12 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0769230769230769 can restrict 13 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0714285714285714 can restrict 14 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0666666666666666 can restrict 15 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0625 can restrict 16 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0588235294117647 can restrict 17 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0555555555555555 can restrict 18 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.0526315789473684 can restrict 19 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=0.05 can restrict 20 tasks run in the same executor - SPARK-45527 default rp with task.gpu.amount=1.0 can restrict 1 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.5 can restrict 2 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.3333333333333333 can restrict 3 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.25 can restrict 4 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.2 can restrict 5 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.1666666666666666 can restrict 6 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.1428571428571428 can restrict 7 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.125 can restrict 8 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.1111111111111111 can restrict 9 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.1 can restrict 10 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0909090909090909 can restrict 11 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0833333333333333 can restrict 12 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0769230769230769 can restrict 13 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0714285714285714 can restrict 14 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0666666666666666 can restrict 15 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0625 can restrict 16 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0588235294117647 can restrict 17 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0555555555555555 can restrict 18 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.0526315789473684 can restrict 19 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.05 can restrict 20 barrier tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=1.0 can restrict 1 tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.5 can restrict 2 tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.3333333333333333 can restrict 3 tasks run on the different executor - SPARK-45527 default rp with task.gpu.amount=0.25 can restrict 4 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0666666666666666 can restrict 15 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0625 can restrict 16 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0588235294117647 can restrict 17 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0555555555555555 can restrict 18 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0526315789473684 can restrict 19 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.05 can restrict 20 tasks run in the same executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=1.0 can restrict 1 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.5 can restrict 2 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.3333333333333333 can restrict 3 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.25 can restrict 4 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.2 can restrict 5 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1666666666666666 can restrict 6 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1428571428571428 can restrict 7 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.125 can restrict 8 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1111111111111111 can restrict 9 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1 can restrict 10 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0909090909090909 can restrict 11 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0833333333333333 can restrict 12 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0769230769230769 can restrict 13 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0714285714285714 can restrict 14 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0666666666666666 can restrict 15 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0625 can restrict 16 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0588235294117647 can restrict 17 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0555555555555555 can restrict 18 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0526315789473684 can restrict 19 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.05 can restrict 20 barrier tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=1.0 can restrict 1 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.5 can restrict 2 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.3333333333333333 can restrict 3 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.25 can restrict 4 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.2 can restrict 5 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1666666666666666 can restrict 6 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1428571428571428 can restrict 7 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.125 can restrict 8 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1111111111111111 can restrict 9 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.1 can restrict 10 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0909090909090909 can restrict 11 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0833333333333333 can restrict 12 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0769230769230769 can restrict 13 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0714285714285714 can restrict 14 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0666666666666666 can restrict 15 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0625 can restrict 16 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0588235294117647 can restrict 17 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0555555555555555 can restrict 18 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.0526315789473684 can restrict 19 tasks run on the different executor - SPARK-45527 TaskResourceProfile with task.gpu.amount=0.05 can restrict 20 tasks run on the different executor Warning: [766.327s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attribute ``` ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Manually ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
