[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-68034277 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68034288 [Test build #24777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24777/consoleFull) for PR 3784 at commit [`0e51101`](https://githu

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-68034273 [Test build #24766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24766/consoleFull) for PR 3645 at commit [`45d0356`](https://gith

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68034060 [Test build #24776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24776/consoleFull) for PR 3784 at commit [`3cf7937`](https://githu

[GitHub] spark pull request: [SPARK-4946] [CORE] Using AkkaUtils.askWithRep...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3785#issuecomment-68034001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4946] [CORE] Using AkkaUtils.askWithRep...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3785#issuecomment-68033992 [Test build #24765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24765/consoleFull) for PR 3785 at commit [`9ca6541`](https://gith

[GitHub] spark pull request: [WIP][SPARK-4937][SQL] Adding optimization to ...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68033865 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [WIP][SPARK-4937][SQL] Adding optimization to ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68033864 [Test build #24768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24768/consoleFull) for PR 3778 at commit [`f1a487f`](https://gith

[GitHub] spark pull request: [SPARK-4465] runAsSparkUser doesn't affect Tas...

2014-12-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3741#issuecomment-68033611 @jongyoul just to give more context on the Mesos side, a user was chosen to launch the executor command with by the following: - If mesos slave flag switch_user is dis

[GitHub] spark pull request: [SPARK-4409][MLlib] Additional Linear Algebra ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3319#issuecomment-68033509 [Test build #24775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24775/consoleFull) for PR 3319 at commit [`b0354f6`](https://githu

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-68033344 @WangTaoTheTonic what I mean is more like the following. Currently we set `amMemory` to the value of `--driverMemory` in `parseArgs`. We should do this only if this is

[GitHub] spark pull request: [SPARK-4409][MLlib] Additional Linear Algebra ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3319#issuecomment-68033235 [Test build #24774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24774/consoleFull) for PR 3319 at commit [`04c4829`](https://githu

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-68033141 Looks great. Thanks for adding the warning about `inprogress` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4951][Core] Fix the issue that a busy e...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3783#issuecomment-68032741 [Test build #24772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24772/consoleFull) for PR 3783 at commit [`105ba3a`](https://githu

[GitHub] spark pull request: [WIP][SPARK-4937][SQL] Adding optimization to ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68032736 [Test build #24773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24773/consoleFull) for PR 3778 at commit [`8c0316f`](https://githu

[GitHub] spark pull request: [SPARK-4953][Doc] Fix the description of build...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3787#issuecomment-68032494 [Test build #24771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24771/consoleFull) for PR 3787 at commit [`9ab0c24`](https://githu

[GitHub] spark pull request: [SPARK-4953] Fix the description of building S...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3787#issuecomment-68032259 [Test build #24769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24769/consoleFull) for PR 3787 at commit [`264e4e0`](https://githu

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2765#issuecomment-68032263 [Test build #24770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24770/consoleFull) for PR 2765 at commit [`ce86bcc`](https://githu

[GitHub] spark pull request: [SPARK-4953] Fix the description of building S...

2014-12-23 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/3787 [SPARK-4953] Fix the description of building Spark with YARN At the section "Specifying the Hadoop Version" In building-spark.md, there is description about building with YARN with Hadoop 0.23.

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3710#issuecomment-68032031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3710#issuecomment-68032028 [Test build #24761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24761/consoleFull) for PR 3710 at commit [`f947226`](https://gith

[GitHub] spark pull request: [WIP][SPARK-4937][SQL] Adding optimization to ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68031794 [Test build #24768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24768/consoleFull) for PR 3778 at commit [`f1a487f`](https://githu

[GitHub] spark pull request: [SPARK-4723] [CORE] To abort the stages which ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3786#issuecomment-68031585 [Test build #24767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24767/consoleFull) for PR 3786 at commit [`003774a`](https://githu

[GitHub] spark pull request: [SPARK-4723] [CORE] To abort the stages which ...

2014-12-23 Thread YanTangZhai
GitHub user YanTangZhai opened a pull request: https://github.com/apache/spark/pull/3786 [SPARK-4723] [CORE] To abort the stages which have attempted some times For some reason, some stages may attempt many times. A threshold could be added and the stages which have attempted more t

[GitHub] spark pull request: [SPARK-4890] Upgrade Boto to 2.34.0; automatic...

2014-12-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3737#issuecomment-68031420 @JoshRosen There are a couple things that we may want to change about this new behavior in the future. The first is that `--help` now requires a download to work, which

[GitHub] spark pull request: [SPARK-4951][Core] Fix the issue that a busy e...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3783#issuecomment-68030911 [Test build #24760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24760/consoleFull) for PR 3783 at commit [`d5c615d`](https://gith

[GitHub] spark pull request: [SPARK-4951][Core] Fix the issue that a busy e...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3783#issuecomment-68030915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/3707#discussion_r22247187 --- Diff: build/mvn --- @@ -0,0 +1,130 @@ +#!/usr/bin/env bash + +# Determine the current working directory +_DIR="$( cd "$( dirname "${BASH_S

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68030428 [Test build #24764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24764/consoleFull) for PR 3784 at commit [`5d54349`](https://gith

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68030433 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-68030283 /cc @rxin @aarondav, since this is NIO connection manager related. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-68030262 [Test build #24766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24766/consoleFull) for PR 3645 at commit [`45d0356`](https://githu

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-68030188 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68030094 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4501][Core] - Create build/mvn to autom...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3707#issuecomment-68030097 Thanks for updating! It looks like this is still merge-conflicted for some reason, but we can probably resolve this ourselves on merge. This looks good to me.

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68030092 [Test build #24763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24763/consoleFull) for PR 3784 at commit [`2abbf8e`](https://gith

[GitHub] spark pull request: [SPARK-4946] [CORE] Using AkkaUtils.askWithRep...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3785#issuecomment-68030034 [Test build #24765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24765/consoleFull) for PR 3785 at commit [`9ca6541`](https://githu

[GitHub] spark pull request: [SPARK-4946] [CORE] Using AkkaUtils.askWithRep...

2014-12-23 Thread YanTangZhai
GitHub user YanTangZhai opened a pull request: https://github.com/apache/spark/pull/3785 [SPARK-4946] [CORE] Using AkkaUtils.askWithReply in MapOutputTracker.askTracker to reduce the chance of the communicating problem Using AkkaUtils.askWithReply in MapOutputTracker.askTracker to r

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68029804 [Test build #24764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24764/consoleFull) for PR 3784 at commit [`5d54349`](https://githu

[GitHub] spark pull request: [SPARK-3629][Doc] improve spark on yarn doc

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2813#issuecomment-68029649 I _do_ think that it would be helpful to re-arrange this page so the high-level "here's how to run on YARN" comes before the big configuration table (I've spoken with a

[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...

2014-12-23 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-68029618 @avulanov It's very encouraging benchmark result you saw in real world cluster setup. Since I'm on vacation recently, I don't actually deploy the new code and benchmark in

[GitHub] spark pull request: [SPARK-4937][SQL] Normalizes conjunctions and ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68029598 [Test build #24763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24763/consoleFull) for PR 3784 at commit [`2abbf8e`](https://githu

[GitHub] spark pull request: [WIP][SPARK-4937][SQL] Adding optimization to ...

2014-12-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68029380 @scwf These optimizations are useful, particularly the one that eliminates common predicates. Thanks for bringing them up! However, the implementation in this PR is rea

[GitHub] spark pull request: [SPARK-4465] runAsSparkUser doesn't affect Tas...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3741#issuecomment-68029290 @jongyoul I think @tnachen is suggesting that Akka's use of threadpools might mean that threads which have properly inherited the settings from `runAsSparkUser` might s

[GitHub] spark pull request: [SPARK-4949] [SPARK-4949] shutdownCallback in ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3781#issuecomment-68029208 [Test build #24759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24759/consoleFull) for PR 3781 at commit [`5942765`](https://gith

[GitHub] spark pull request: [SPARK-4949] [SPARK-4949] shutdownCallback in ...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3781#issuecomment-68029211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2014-12-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3725#issuecomment-68029024 Ah, gotcha; this sounds like it fixes a fairly serious bug, so I'll try to cherry-pick this into our backport branches. I'll take a pass through this tomorrow when I'm

[GitHub] spark pull request: [SPARK-4937] Normalizes conjunctions and disju...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68028990 [Test build #24762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24762/consoleFull) for PR 3784 at commit [`cf95639`](https://githu

[GitHub] spark pull request: [SPARK-4937] Normalizes conjunctions and disju...

2014-12-23 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/3784 [SPARK-4937] Normalizes conjunctions and disjunctions to eliminate common predicates This PR is a simplified version of several filter optimization rules introduced in #3778. Newly introduced opt

[GitHub] spark pull request: [SPARK-4937] Normalizes conjunctions and disju...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68028993 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4937] Normalizes conjunctions and disju...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3784#issuecomment-68028992 [Test build #24762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24762/consoleFull) for PR 3784 at commit [`cf95639`](https://gith

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22246575 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListenerBus.scala --- @@ -17,20 +17,25 @@ package org.apache.spar

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68028615 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68028612 [Test build #24758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24758/consoleFull) for PR 3780 at commit [`52adf46`](https://gith

[GitHub] spark pull request: [SPARK-4937][SQL] Adding optimization to simpl...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3778#issuecomment-68028489 change to WIP, todos: 1 fix test case 2 optimize and simplify the code --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3710#issuecomment-68028341 [Test build #24761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24761/consoleFull) for PR 3710 at commit [`f947226`](https://githu

[GitHub] spark pull request: [SPARK-4951][Core] Fix the issue that a busy e...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3783#issuecomment-68028107 [Test build #24760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24760/consoleFull) for PR 3783 at commit [`d5c615d`](https://githu

[GitHub] spark pull request: [SPARK-4951][Core] Fix the issue that a busy e...

2014-12-23 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3783 [SPARK-4951][Core] Fix the issue that a busy executor may be killed Three changes to fix this issue: 1. Handle the case that receiving `SparkListenerTaskStart` before `SparkListenerBlockMan

[GitHub] spark pull request: [SPARK-2823][GraphX]fix GraphX EdgeRDD zipPart...

2014-12-23 Thread luluorta
Github user luluorta commented on the pull request: https://github.com/apache/spark/pull/1763#issuecomment-68027993 Thanks, @Earne Actually we already had a method to customize the partition number of EdgeRDD by using `Graph.partitionBy` [Graph.scala#L136](https://github.com

[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...

2014-12-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3466#discussion_r22246102 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingSource.scala --- @@ -28,46 +28,58 @@ private[streaming] class StreamingSource(ssc: S

[GitHub] spark pull request: [SPARK-4950] Delete obsolete mapReduceTripelet...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3782#issuecomment-68026322 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3061#issuecomment-68026214 Actually let's close this one. We have already implemented the same functionality in Netty. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4950] Delete obsolete mapReduceTripelet...

2014-12-23 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/3782#issuecomment-68026190 Any reason not to replace the api along with SPARK-3936? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-4950Delete obsolete mapReduceTripelets u...

2014-12-23 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/3782 SPARK-4950Delete obsolete mapReduceTripelets used in Pregel You can merge this pull request into a Git repository by running: $ git pull https://github.com/maropu/spark DeleteObsoleteAPIInPrege

[GitHub] spark pull request: [SPARK-4465] runAsSparkUser doesn't affect Tas...

2014-12-23 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3741#issuecomment-68025888 @tnachen I'm sorry but I don't understand what you say to me. Could you tell me the meaning of "akka execution path"? I think user info is preserved by passing SPARK_USE

[GitHub] spark pull request: [SPARK-4935][SQL] When hive.cli.print.header c...

2014-12-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-68025596 Updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-4949] [SPARK-4949] shutdownCallback in ...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3781#issuecomment-68025557 [Test build #24759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24759/consoleFull) for PR 3781 at commit [`5942765`](https://githu

[GitHub] spark pull request: [SPARK-4949] [SPARK-4949] shutdownCallback in ...

2014-12-23 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/3781 [SPARK-4949] [SPARK-4949] shutdownCallback in SparkDeploySchedulerBackend should be enclosed by synchronized block. A variable `shutdownCallback` in SparkDeploySchedulerBackend can be accessed from

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2014-12-23 Thread stephenh
Github user stephenh commented on the pull request: https://github.com/apache/spark/pull/3725#issuecomment-68025292 Hey Josh, thanks! (262 open PRs...wow... :-)) Per the branches, I checked branch-1.0, branch-1.1, and branch-1.2, and all of them have the spark.files.userClassP

[GitHub] spark pull request: [SPARK-4465] runAsSparkUser doesn't affect Tas...

2014-12-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3741#issuecomment-68025077 Btw this is also assuming in the run block, when spark executor task is launched the user info is still preserved through out the akka execution path. --- If your proje

[GitHub] spark pull request: [SPARK-4790][STREAMING] Fix ReceivedBlockTrack...

2014-12-23 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/3726#issuecomment-68025072 Eventually is pretty much trying a test in a loop and that encourages flakey tests - I am in general not a very big fan of using eventually. The correct appr

[GitHub] spark pull request: [SPARK-4935][SQL] When hive.cli.print.header c...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-68024974 [Test build #24754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24754/consoleFull) for PR 3761 at commit [`46dc344`](https://gith

[GitHub] spark pull request: [SPARK-4935][SQL] When hive.cli.print.header c...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3761#issuecomment-68024977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68024602 [Test build #24758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24758/consoleFull) for PR 3780 at commit [`52adf46`](https://githu

[GitHub] spark pull request: [SPARK-3398] [SPARK-4325] [EC2] Use EC2 status...

2014-12-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3195#issuecomment-68024562 FYI: The commit message at 317e114 now incorrectly states that the linear backoff was removed. Is there any way to fix that? --- If your project is set up for it, you c

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3061#issuecomment-68024263 [Test build #24757 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24757/consoleFull) for PR 3061 at commit [`5a73b04`](https://githu

[GitHub] spark pull request: [SPARK-4570][SQL]add BroadcastLeftSemiJoinHash

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3442#issuecomment-68024274 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3061#issuecomment-68024339 [Test build #24757 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24757/consoleFull) for PR 3061 at commit [`5a73b04`](https://gith

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3061#issuecomment-68024341 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/3061#discussion_r22245488 --- Diff: core/src/main/scala/org/apache/spark/network/nio/NioBlockTransferService.scala --- @@ -121,8 +132,34 @@ final class NioBlockTransferService(con

[GitHub] spark pull request: [SPARK-4570][SQL]add BroadcastLeftSemiJoinHash

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3442#issuecomment-68024272 [Test build #24753 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24753/consoleFull) for PR 3442 at commit [`a4a43c9`](https://gith

[GitHub] spark pull request: [SPARK-4195][Core]retry to fetch blocks's resu...

2014-12-23 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/3061#discussion_r22245434 --- Diff: core/src/main/scala/org/apache/spark/network/nio/NioBlockTransferService.scala --- @@ -39,6 +41,10 @@ final class NioBlockTransferService(conf:

[GitHub] spark pull request: [YARN] Delete confusion configurations

2014-12-23 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/3776#issuecomment-68024052 ok, I have got it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [YARN] Delete confusion configurations

2014-12-23 Thread XuTingjun
Github user XuTingjun closed the pull request at: https://github.com/apache/spark/pull/3776 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-4465] runAsSparkUser doesn't affect Tas...

2014-12-23 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3741#issuecomment-68023902 @JoshRosen I didn't know that exact behaviour of `runAsSparkUser` because I understood that `runAsSparkUser` influenced all method of instances created in that block. In

[GitHub] spark pull request: [SPARK-4314][Streaming] Exception throws when ...

2014-12-23 Thread maji2014
Github user maji2014 closed the pull request at: https://github.com/apache/spark/pull/3203 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request: [SPARK-4314][Streaming] Exception throws when ...

2014-12-23 Thread maji2014
Github user maji2014 commented on the pull request: https://github.com/apache/spark/pull/3203#issuecomment-68023838 refer to jira, got it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-4006] Block Manager - Double Register C...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2854#issuecomment-68023532 [Test build #24756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24756/consoleFull) for PR 2854 at commit [`95ae4db`](https://gith

[GitHub] spark pull request: [SPARK-4006] Block Manager - Double Register C...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2854#issuecomment-68023536 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3466#issuecomment-68023274 [Test build #24751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24751/consoleFull) for PR 3466 at commit [`c097ddc`](https://gith

[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3466#issuecomment-68023279 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22245214 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -75,6 +78,11 @@ private[spark] class LiveListenerBus extends SparkListener

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22245199 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -75,6 +78,11 @@ private[spark] class LiveListenerBus extends SparkListe

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22245204 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListenerBus.scala --- @@ -17,20 +17,25 @@ package org.apache.spar

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22245148 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -75,6 +78,11 @@ private[spark] class LiveListenerBus extends SparkListener

[GitHub] spark pull request: [SPARK-4859][Core][Streaming] Improve LiveList...

2014-12-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3710#discussion_r22245060 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListenerBus.scala --- @@ -17,20 +17,25 @@ package org.apache.spark.s

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68022467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68022466 [Test build #24755 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24755/consoleFull) for PR 3780 at commit [`b776284`](https://gith

[GitHub] spark pull request: [SPARK-4537][Streaming] Expand StreamingSource...

2014-12-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3466#discussion_r22245026 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingSource.scala --- @@ -28,46 +28,58 @@ private[streaming] class StreamingSource(ssc: S

[GitHub] spark pull request: [SPARK-4945] [SQL] Add overwrite option suppor...

2014-12-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3780#issuecomment-68022423 [Test build #24755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24755/consoleFull) for PR 3780 at commit [`b776284`](https://githu

  1   2   3   4   >