[GitHub] spark pull request: [SPARK-11259] [ML] Params.validateParams() sho...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9224#issuecomment-150239505 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11259] [ML] Params.validateParams() sho...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9224#issuecomment-150239477 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11259] [ML] Params.validateParams() sho...

2015-10-22 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/9224 [SPARK-11259] [ML] Params.validateParams() should be called automatically See JIRA: https://issues.apache.org/jira/browse/SPARK-11259 You can merge this pull request into a Git repository by runn

[GitHub] spark pull request: [SPARK-11212][Core][Streaming]Make preferred l...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9181#issuecomment-150236704 **[Test build #44148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44148/consoleFull)** for PR 9181 at commit [`35f7936`](https://gith

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150236398 **[Test build #44149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44149/consoleFull)** for PR 9214 at commit [`1c12ca5`](https://gith

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150235777 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150235752 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11212][Core][Streaming]Make preferred l...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9181#issuecomment-150233670 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11212][Core][Streaming]Make preferred l...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9181#issuecomment-150233593 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-150229225 **[Test build #44147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44147/consoleFull)** for PR 9051 at commit [`6d55ef8`](https://gith

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-150228205 **[Test build #1940 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1940/consoleFull)** for PR 9051 at commit [`6d55ef8`](https://git

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-150228006 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-150227975 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-10-22 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-150226880 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-10-22 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/8669#issuecomment-150225081 If `SPARK_HOME` is the standard way, we should update the scripts to honor this, it's not a good idea to make this issue as won't fix, because lots of deployment and p

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-10-22 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/8669#issuecomment-150223128 Hi @srowen , I admitted currently solution is quite ugly, but like what you mentioned, it is a chicken-and-egg problem, hard to remove the duplication. Another solutio

[GitHub] spark pull request: [SPARK-11215] [ML] Add multiple columns suppor...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9183#issuecomment-150222594 [Test build #44146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44146/consoleFull) for PR 9183 at commit [`a64f71d`](https://gith

[GitHub] spark pull request: [SPARK-11215] [ML] Add multiple columns suppor...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9183#issuecomment-150220436 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-11215] [ML] Add multiple columns suppor...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9183#issuecomment-150220280 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-11242][SQL] In conf/spark-env.sh.templa...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9201#issuecomment-150211387 **[Test build #1939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1939/consoleFull)** for PR 9201 at commit [`5a11872`](https://gi

[GitHub] spark pull request: [SPARK-10857] [SQL] Block SQL injection vulner...

2015-10-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9202#discussion_r42743817 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/SqlIdentifierUtil.java --- @@ -0,0 +1,322 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-10857] [SQL] Block SQL injection vulner...

2015-10-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9202#discussion_r42743770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -86,19 +94,58 @@ abstract class JdbcDialect { * name is a reserve

[GitHub] spark pull request: [SPARK-10857] [SQL] Block SQL injection vulner...

2015-10-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9202#discussion_r42743556 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/SqlIdentifierUtil.java --- @@ -0,0 +1,322 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8669#issuecomment-150210063 Gotcha, is the way forward really to not over-write `SPARK_HOME` if set? Because most of the work here seems to be about discovering the home otherwise. The symlinks them

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150209795 **[Test build #1938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1938/consoleFull)** for PR 9220 at commit [`396c66a`](https://gi

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-10-22 Thread patrungel
Github user patrungel commented on the pull request: https://github.com/apache/spark/pull/8669#issuecomment-150209426 The change aims to make spark binaries launch properly when symlinked, not just to avoid setting `SPARK_HOME`. As of current master, ```SPARK_HOME``` value is overw

[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...

2015-10-22 Thread kmadhugit
Github user kmadhugit closed the pull request at: https://github.com/apache/spark/pull/7020 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-10694][STREAMING]Prevent Data Loss in S...

2015-10-22 Thread dibbhatt
Github user dibbhatt commented on the pull request: https://github.com/apache/spark/pull/8817#issuecomment-150203251 Hi @andrewor14 , if you have some time, can you look into this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9223#issuecomment-150202942 **[Test build #44145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44145/consoleFull)** for PR 9223 at commit [`05b3f42`](https://gith

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9223#issuecomment-150202338 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9223#issuecomment-150202356 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread zjffdu
GitHub user zjffdu opened a pull request: https://github.com/apache/spark/pull/9223 [SPARK-11102] [SQL] Uninformative exception when specifing non-exist … …input for JSON data source You can merge this pull request into a Git repository by running: $ git pull https://

[GitHub] spark pull request: [SPARK-11244][SPARKR] sparkR.stop() should rem...

2015-10-22 Thread saurfang
Github user saurfang commented on the pull request: https://github.com/apache/spark/pull/9205#issuecomment-150202122 I still advocate making developer API documentation public. However I think one workaround is to make them roxygen doc again with `#'` but add `#' @rdname .ignore` for

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/9142#issuecomment-150201552 It's corrupted with other commits after I rebase, close it and will create another PR --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-10-22 Thread zjffdu
Github user zjffdu closed the pull request at: https://github.com/apache/spark/pull/9142 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150201234 Yes, but if you're proposing the change I believe you need to investigate the implications. This doesn't address dependency changes which Steve already alluded to. ---

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread pronix
Github user pronix commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150200410 @srowen Official documentation about upgrade http://blog.twitter4j.org/2014/03/13/69 --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8669#issuecomment-150181761 I think this is a whole lot of change, and the purpose really is just to avoid setting `SPARK_HOME` when Spark is symlinked. But that's pretty standard procedure. RIght?

[GitHub] spark pull request: [SPARK-10958] Use json4s 3.3.0. Formats is now...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8992#issuecomment-150181178 @tylerprete is this moving forward or should we close it at this point? --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark pull request: [SPARK-11125] [SQL] Uninformative exception wh...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-150181092 @zjffdu given your similar PR on this topic, can you address the question of how this occurs? The "Error" is a different lower-level thing. --- If your project is set up

[GitHub] spark pull request: Ignore NoClassDefFoundError in obtainTokenForH...

2015-10-22 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/9213#issuecomment-150181030 I presented the change to get some feedback. If the feedback is affirmative, I will create JIRA. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-11242][SQL] In conf/spark-env.sh.templa...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9201#issuecomment-150180905 **[Test build #1939 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1939/consoleFull)** for PR 9201 at commit [`5a11872`](https://git

[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...

2015-10-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9096 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-11212][Core][Streaming]Make preferred l...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9181#issuecomment-150180536 @zsxwing I merged #9096 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-150180490 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread FRosner
Github user FRosner commented on a diff in the pull request: https://github.com/apache/spark/pull/9222#discussion_r42734750 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -130,16 +130,17 @@ private[r] object SQLUtils { } def d

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150179707 **[Test build #1938 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1938/consoleFull)** for PR 9220 at commit [`396c66a`](https://git

[GitHub] spark pull request: [SPARK-8048] Partitionning of an RDD with 0 pa...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7020#issuecomment-150179696 @kmadhugit can you close this then? we can't close it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread FRosner
Github user FRosner commented on the pull request: https://github.com/apache/spark/pull/9222#issuecomment-150179591 Thanks for the reply. > Instead of creating a new testsuite in Scala, you can add a new test case in R, using callJStatic to invoke "dfToCols" on the Scala side.

[GitHub] spark pull request: Ignore NoClassDefFoundError in obtainTokenForH...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9213#issuecomment-150179060 @tedyu you know you also need to create JIRAs; I think you've been around long enough to read https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark But

[GitHub] spark pull request: Ignore NoClassDefFoundError in obtainTokenForH...

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9213#issuecomment-150178986 I have the same question as in a different change. How does this occur? it means Spark was compiled with Hive, and it wasn't present at runtime, which seems like a more f

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150178852 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150178851 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150178787 **[Test build #44144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44144/consoleFull)** for PR 8880 at commit [`135b380`](https://git

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150178770 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/9222#discussion_r42733892 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -130,16 +130,17 @@ private[r] object SQLUtils { } def d

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150177373 @pronix also you need to look at https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark and update the title, etc. --- If your project is set up for it

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150177327 Yeah, we would need more info here, to update across major releases. Because this is just an isolated 'leaf' package, we can probably tolerate more change there. But, at

[GitHub] spark pull request: [SPARK-11225]Prevent generate empty file

2015-10-22 Thread viper-kun
Github user viper-kun commented on the pull request: https://github.com/apache/spark/pull/9191#issuecomment-150176502 Hi @davies, Sorry, I do not understand Python. Can you help me fix it? --- If your project is set up for it, you can reply to this email and have your reply app

[GitHub] spark pull request: [SPARK-9622] [ML] DecisionTreeRegressor: provi...

2015-10-22 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/8866#issuecomment-150176392 @jkbradley I restricted ```varianceCol``` to only DecisionTreeRegressor and remove default value, so it's validated only when users specified. --- If your project is

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9222#issuecomment-150166755 Instead of creating a new testsuite in Scala, you can add a new test case in R, using callJStatic to invoke "dfToCols" on the Scala side. --- If your project is set

[GitHub] spark pull request: [SPARK-10903] [SPARKR] R - Simplify SQLContext...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9192#issuecomment-150165422 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-10903] [SPARKR] R - Simplify SQLContext...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9192#issuecomment-150165423 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10903] [SPARKR] R - Simplify SQLContext...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9192#issuecomment-150165301 **[Test build #44139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44139/consoleFull)** for PR 9192 at commit [`2f9e50d`](https://git

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-10-22 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9218#issuecomment-150164199 could we support both names() and colnames()? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...

2015-10-22 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-150162723 On the R side, there is a cache of created SQLContext/HiveContext, so R won't call createSQLContext() second time. See https://github.com/apache/spark/blob/master/R/pkg/

[GitHub] spark pull request: [SPARK-11256] Mark all Stage/ResultStage/Shuff...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9219#issuecomment-150159388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11256] Mark all Stage/ResultStage/Shuff...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9219#issuecomment-150159386 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11256] Mark all Stage/ResultStage/Shuff...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9219#issuecomment-150159239 **[Test build #44140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44140/consoleFull)** for PR 9219 at commit [`f9939ca`](https://git

[GitHub] spark pull request: [SPARK-10903] [SPARKR] R - Simplify SQLContext...

2015-10-22 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/9192#discussion_r42726488 --- Diff: R/pkg/R/SQLContext.R --- @@ -17,6 +17,34 @@ # SQLcontext.R: SQLContext-driven functions +#' Temporary function to reroute old S3

[GitHub] spark pull request: [SPARK-10903] [SPARKR] R - Simplify SQLContext...

2015-10-22 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/9192#discussion_r42726208 --- Diff: R/pkg/R/jobj.R --- @@ -77,6 +77,11 @@ print.jobj <- function(x, ...) { cat("Java ref type", name, "id", x$id, "\n", sep = " ") }

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150156455 **[Test build #44144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44144/consoleFull)** for PR 8880 at commit [`135b380`](https://gith

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150156202 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread pronix
Github user pronix commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150155800 @steveloughran you mean require something change ? or just potential issue ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150155724 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150155741 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2015-10-22 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8872#discussion_r42725607 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -457,34 +462,37 @@ private[spark] class MesosCluster

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150155055 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150155091 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10292][SQL] make metadata query-able by...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8461#issuecomment-150155083 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-10771: Implement the shuffle encryption ...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-150155027 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10292][SQL] make metadata query-able by...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8461#issuecomment-150155085 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150155092 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2015-10-22 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8872#discussion_r42725390 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -497,28 +505,28 @@ private[spark] class MesosCluster

[GitHub] spark pull request: [SPARK-10292][SQL] make metadata query-able by...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8461#issuecomment-150155033 **[Test build #44141 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44141/consoleFull)** for PR 8461 at commit [`eb59d63`](https://git

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2015-10-22 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8872#discussion_r42725306 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -443,9 +444,13 @@ private[spark] class MesosClusterS

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150154140 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11188][SQL][WIP] Elide stacktraces in b...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9194#issuecomment-150154176 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9222#issuecomment-150153510 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2015-10-22 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8872#discussion_r42724611 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -358,9 +358,10 @@ private[spark] class MesosClusterS

[GitHub] spark pull request: SPARK-11258 Remove quadratic runtime complexit...

2015-10-22 Thread FRosner
GitHub user FRosner opened a pull request: https://github.com/apache/spark/pull/9222 SPARK-11258 Remove quadratic runtime complexity for converting a Spark DataFrame into an R data.frame https://issues.apache.org/jira/browse/SPARK-11258 I was not able to locate an existing

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2015-10-22 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8872#discussion_r42724448 --- Diff: core/src/test/scala/org/apache/spark/scheduler/mesos/MesosClusterSchedulerSuite.scala --- @@ -72,4 +78,56 @@ class MesosClusterSchedulerSuite extend

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-10-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r42724272 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -428,6 +441,9 @@ class ReceiverTracker(ssc: StreamingCon

[GitHub] spark pull request: update twitter4j to 4.0.4 version

2015-10-22 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9221#issuecomment-150151459 Looking @ the twitter4j poms, their dependencies pull in a com.googlecode.kryo -given the kryo pain its a possible troublespot. However, that com.googlecode releas

[GitHub] spark pull request: [SPARK-6785][SQL] fix DateTimeUtils for dates ...

2015-10-22 Thread ckadner
Github user ckadner commented on a diff in the pull request: https://github.com/apache/spark/pull/6983#discussion_r42723887 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -324,20 +324,6 @@ class HiveQuerySuite extends HiveComparis

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-10-22 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-150147496 My 2 cents about this PR: is it possible to create a new `WriteAheadLog` (such as, BatchWriteAheadLog) that wraps the existing `WriteAheadLog` and implement the new logi

[GitHub] spark pull request: [SPARK-10152] [SQL] Support Init script for hi...

2015-10-22 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/8355#discussion_r42722219 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala --- @@ -67,6 +68,46 @@ object HiveThrift

[GitHub] spark pull request: [SPARK-10152] [SQL] Support Init script for hi...

2015-10-22 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/8355#discussion_r42722019 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala --- @@ -67,6 +68,46 @@ object HiveThrift

[GitHub] spark pull request: [SPARK-10292][SQL] make metadata query-able by...

2015-10-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8461#issuecomment-150142131 **[Test build #44141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44141/consoleFull)** for PR 8461 at commit [`eb59d63`](https://gith

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-10-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r42721825 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceivedBlockTracker.scala --- @@ -195,31 +285,56 @@ private[streaming] class Receive

<    2   3   4   5   6   7   8   >