[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 not sure what you are asking, if you are saying its flaky then file a jira and please cc me and Sanket. The link above doesn't load for me, please give me permissions if its needed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22628 Is it supposed to the flaky ChunkFetchIntegrationSuite? http://spark-tests.appspot.com/test-details?suite_name=org.apache.spark.network.ChunkFetchIntegrationSuite_name=fetchFileChunk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 pulled into master, thanks @redsanket --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/22628 Thanks @tgravescs for explaining the issue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 the default of 0 gets you the netty default which is always 2 * the # of cores. It ignores if you set the io.serverThreads to set the # of shuffle threads. With a default of 100, it actually applies the setting for io.serverThreads or if you don't have it set then uses the 2 * # of cores. So having a default of 100 is better here. io.serverThreads = 10 and spark.shuffle.server.chunkFetchHandlerThreadsPercent=100 then the # of threads here for chunked fetches is 10 io.serverThreads=10 and spark.shuffle.server.chunkFetchHandlerThreadsPercent=0 then the # of threads here for chunked fetches is 2*#cores Its better to have a default of 100 to keep the current behavior. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22628 I'm not clear from the description what the issue is. I get that the number of threads is undesirable but can you illustrate with a clearer example? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96951/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22628 **[Test build #96951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96951/testReport)** for PR 22628 at commit [`a95c608`](https://github.com/apache/spark/commit/a95c6080d1eb2dd23e33be5655f48d92bd028e10). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22628 **[Test build #96951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96951/testReport)** for PR 22628 at commit [`a95c608`](https://github.com/apache/spark/commit/a95c6080d1eb2dd23e33be5655f48d92bd028e10). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96943/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22628 **[Test build #96943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96943/testReport)** for PR 22628 at commit [`0258d19`](https://github.com/apache/spark/commit/0258d197e27e6a053023d1b49955343659e240ac). * This patch **fails Java style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22628 **[Test build #96943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96943/testReport)** for PR 22628 at commit [`0258d19`](https://github.com/apache/spark/commit/0258d197e27e6a053023d1b49955343659e240ac). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22628 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22628 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22628: [SPARK-25641] Change the spark.shuffle.server.chunkFetch...
Github user redsanket commented on the issue: https://github.com/apache/spark/pull/22628 @tgravescs @abellina plz take a look thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
