[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 The new PR is here: https://github.com/apache/spark/pull/19848 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 I am closing this PR in favor of another one with a simpler approach that I will open shortly. Thank you everyone for the comments, here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 @mridulm thank you very much for the comment. I was thinking about something similar, but more like sending both stageID and a monotonically increasing jobId to the commit method and then let try the canCommit use the stageId while we use the jobID for hadoop commits. I am testing this right now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19388 Internally, spark can continue to use stageId for its commit protocol - which is independent of what we expose as jobId to mapred/mapreduce committer. As an implementation detail, we were making using of stageId in executor and rddId at driver. Spark does not really care about what is presented to hadoop committer, and hadoop committer does not really care about what spark is using internally; as long as both are used consistently. Will not that help make things simpler ? Or am I missing something here ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 Thank you very much, @mridulm for the comment. Yeah, I agree that the provided solution is complex. The complexity came form the fact that in this solution both executors and the driver use a jobId that is based on the stageId. I tried a simpler solution in October that uses rddId as a unified Id between drivers and executors, but that solution failed since committer was expecting a stageId as the jobId. Let me discuss this with more details. In the current code there is a call to commitTask (the taskContext here is created by using stageID as the jobId): https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala#L134 That call will ended up reaching here: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala#L172 This will follows to the following point: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala#L73 Which will call canCommit and canCommit will check whether the jobId is an actual stageID: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala#L89 If we want to use a simpler solution for creating the jobId based on a monotonically increasing value then we need to make some changes in other parts of the code to remove the need for using a stageId. I need to check whether this will actually create a simpler fix (especially with regards to SPARK-4879). I will update here as soon as I finished investigating this. Thank you again. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19388 I agree with @vanzin - this looks very complicated for a enforcing a fairly simple constraint. It would be easier to depend on a AtomicInteger in driver for the id - and propagate that to executors via Configuration (so that driver and executors use the same id). Will that not be simpler @rezasafi ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84033/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84033/testReport)** for PR 19388 at commit [`66192fd`](https://github.com/apache/spark/commit/66192fd61dae1489216fffcea0f104800aefbd31). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84033/testReport)** for PR 19388 at commit [`66192fd`](https://github.com/apache/spark/commit/66192fd61dae1489216fffcea0f104800aefbd31). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84006/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84006/testReport)** for PR 19388 at commit [`66192fd`](https://github.com/apache/spark/commit/66192fd61dae1489216fffcea0f104800aefbd31). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 Thank you very much, @dongjoon-hyun. I forgot to pushed the fix for streaming failure. @vanzin My understanding based on reading the code before this change is that the logic behind it avoids multiple committing of a single stage. So I think based on that logic it should use stageId somehow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84006/testReport)** for PR 19388 at commit [`66192fd`](https://github.com/apache/spark/commit/66192fd61dae1489216fffcea0f104800aefbd31). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84002/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84002/testReport)** for PR 19388 at commit [`500c73c`](https://github.com/apache/spark/commit/500c73cc96290efe0194e371ab84e0cda863347d). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StageInfo extends Serializable ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #84002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84002/testReport)** for PR 19388 at commit [`500c73c`](https://github.com/apache/spark/commit/500c73cc96290efe0194e371ab84e0cda863347d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19388 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83976/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #83976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83976/testReport)** for PR 19388 at commit [`500c73c`](https://github.com/apache/spark/commit/500c73cc96290efe0194e371ab84e0cda863347d). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class StageInfo extends Serializable ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19388 High level question: do you need to create the commit id based on the rdd + stage ids, or could it be something else (like a monotonically increasing value, or a uuid)? That would probably simplify the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #83976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83976/testReport)** for PR 19388 at commit [`500c73c`](https://github.com/apache/spark/commit/500c73cc96290efe0194e371ab84e0cda863347d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 also @mridulm I think you worked on the committer protocol before, both in spark and MR, so would be great to get your thoughts. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19388 I haven't seen any issues with speculation, most people aren't on 2.2 yet though either. Unfortunately won't have time to look at this today in detail, will try to early next week. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 @rezasafi and I talked about this for a while offline. Its a lot trickier than this seems at first. Here's whats going on: 1. `SparkHadoopWriter` is pretending `rddId` is `stageId`, but clearly its not. In the simple test cases they're the same, but they can be arbitrarily different. That is not new -- it was in the old RDD commit protocol (https://github.com/apache/spark/blob/branch-2.0/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala#L1084). However, SPARK-18191 changed it so that on the executors, it was using the real `stageId`, which resulted in an inconsistency. 2. We really want the `stageId`. Reza had initially made a change to just use the `rddId` consistently, since that is what was being used before. However, this leads to an inconsistency, because the `OutputCommitCoordinator` actually expects the `stageId`. Previously, this seemed to work because `PairRDDFunctions.saveAsNewAPIHadoopDataset` didn't go through the `OutputCommitCoordinator` (!!! can this be right?) and `PairRDDFunctions.saveAsHadoopDataset`, which did go through the `OutputCommitCoordinator`, always used the actual `stageId` (it didn't do any setup before the `stageId` was available). e.g here's the code in 2.0: https://github.com/apache/spark/blob/branch-2.0/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala#L1161 The RDD alone is *not* a good unique identifier -- the same RDD can get saved to hadoop multiple times, and those could even be run in parallel. We need to make sure those tasks don't get mixed up with each other. 3. Its actually quite hard to get the right `stageId`. Its not exposed at all to the user before the job starts. In fact, we *can't* expose the `stageId` before your call to `sc.runJob()`, because there could other threads racing to submit jobs, and the `stageId` isn't known until its inside the `DAGScheduler`'s eventLoop here: https://github.com/apache/spark/blob/4f8dc6b01ea787243a38678ea8199fbb0814cffc/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala#L866 One way to do this would be to add another version of `sc.runJob()` which let you pass in yet another function, which is run inside the `DAGScheduler` after it has the `stageId`. I don't even think we should expose that function publicly, we just need it inside inside PairRDDFunctions. Another possibility would be to forget about the actual stageId, and instead just get some unique id from the `OutputCommitCoordinator`, which is then passed along inside the closures. But then you'd have to figure out how to replicate `OutputCommitCoordinator.stageStart() / stageEnd()`, which probably boils down to the same thing -- another version of `runJob`, this one which pushes the given ID down into the scheduler. Any other ideas? @tgravescs @markhamstra you may be interested in this as well. I know you've told me you've had success w/ speculation lately, but seems like it would be pretty broken given the `rddId` vs. `stageId` (or maybe it doesn't matter b/c in most cases the Hadoop output committer ignores the `jobContext` passed in on `setupJob`?) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/19388 Sorry for the delay. It seems that to be able to commit the same rdd in different stages we need to use stageId. So the jobId and other configurations in the write method of SparkHadoopWriter should be based on the stageId of the rdd and not the rddId. I have a hacky solution for this, but I am working on a better one and will update this PR ASAP. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82331/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82331/testReport)** for PR 19388 at commit [`4c0985e`](https://github.com/apache/spark/commit/4c0985ef9584d893e73a09100e93881ac6687dca). * This patch **fails from timeout after a configured wait of \`250m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82325/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82325/testReport)** for PR 19388 at commit [`4c0985e`](https://github.com/apache/spark/commit/4c0985ef9584d893e73a09100e93881ac6687dca). * This patch **fails from timeout after a configured wait of \`250m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82331/testReport)** for PR 19388 at commit [`4c0985e`](https://github.com/apache/spark/commit/4c0985ef9584d893e73a09100e93881ac6687dca). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19388 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19388 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82325/testReport)** for PR 19388 at commit [`4c0985e`](https://github.com/apache/spark/commit/4c0985ef9584d893e73a09100e93881ac6687dca). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82304 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82304/testReport)** for PR 19388 at commit [`2450333`](https://github.com/apache/spark/commit/2450333821f59e489e20fdcaef7e2d6f41467793). * This patch **fails from timeout after a configured wait of \`250m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82304/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82304/testReport)** for PR 19388 at commit [`2450333`](https://github.com/apache/spark/commit/2450333821f59e489e20fdcaef7e2d6f41467793). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82300/testReport)** for PR 19388 at commit [`2df1831`](https://github.com/apache/spark/commit/2df18318ccd62a15458f1f4588b1126f507386a7). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82300/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82300/testReport)** for PR 19388 at commit [`2df1831`](https://github.com/apache/spark/commit/2df18318ccd62a15458f1f4588b1126f507386a7). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19388 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
