[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4186/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90736/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21353 **[Test build #90736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90736/testReport)** for PR 21353 at commit [`42e2a63`](https://github.com/apache/spark/commit/42e2a63f68ba12264f001ed53ff33c27c1287de8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class ContinuousShuffleDependency[K: ClassTag, V: ClassTag, C: ClassTag](` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21353: [SPARK-24304][SS] Scheduler changes for continuous proce...
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21353 As I've mentioned elsewhere, stages are currently submitted sequentially. That is, for a stage X, all the stage dependencies of X are completed before the tasks within X start. This change proposes to violate that invariant, and it's not obvious that this is a safe approach. The questions we need to answer are: * How can we attempt to validate that this is indeed safe to change, and will not break the scheduler or things dependent on it in subtle ways? * What benefits do we derive from adding the additional risk of a scheduler change, rather than handling continuous shuffles entirely at the RDD layer? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
