[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925330 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -1,140 +0,0 @@ -/* --- End diff -- yes. These are the two

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-84933397 [Test build #28986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28986/consoleFull) for PR 4491 at commit

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-84933426 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6452] [SQL] Checks for missing attribut...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5129#issuecomment-84939647 [Test build #28991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28991/consoleFull) for PR 5129 at commit

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-23 Thread kellyzly
Github user kellyzly commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-84855188 @steveloughran: i don't understand why need make CryptoOutputStream.scala#close safe. Is there situation when multiple threads call this function at the same time?

[GitHub] spark pull request: [SPARK-6397][SQL] Check the missingInput simpl...

2015-03-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/5132#issuecomment-84855187 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-6397][SQL] Check the missingInput simpl...

2015-03-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/5132#issuecomment-84855200 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6449][YARN] Report failure status if dr...

2015-03-23 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/5130#issuecomment-84860442 Do InvocationTargetExceptions only wrap Exceptions and not all Throwables? It will wrap Error, too. Run the following codes in my machine, ```Scala

[GitHub] spark pull request: [SPARK-6397][SQL] Check the missingInput simpl...

2015-03-23 Thread watermen
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/5132#issuecomment-84897442 @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-6466][SQL] Remove unnecessary attribute...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5134#issuecomment-84930028 [Test build #28988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28988/consoleFull) for PR 5134 at commit

[GitHub] spark pull request: [SPARK-5775] BugFix: GenericRow cannot be cast...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4697#issuecomment-84936129 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5775] BugFix: GenericRow cannot be cast...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4697#issuecomment-84936098 [Test build #28985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28985/consoleFull) for PR 4697 at commit

[GitHub] spark pull request: [SPARK-6352] [SQL] Add DirectParquetOutputComm...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5042#issuecomment-84939649 [Test build #28992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28992/consoleFull) for PR 5042 at commit

[GitHub] spark pull request: [SPARK-6379][SQL] Support a functon to call us...

2015-03-23 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/5061#discussion_r26916348 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -212,6 +212,22 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [SPARK-5682] Reuse hadoop encrypted shuffle al...

2015-03-23 Thread kellyzly
Github user kellyzly commented on the pull request: https://github.com/apache/spark/pull/4491#issuecomment-84893111 @steveloughran : in hadoop, if we need add a native lib path to hadoop execution path, we need export LD_LIBRARY_PATH export LD_LIBRARY_PATH=x in hadoop,

[GitHub] spark pull request: [SPARK-6356][SQL] Support the ROLLUP/CUBE/GROU...

2015-03-23 Thread watermen
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/5080#issuecomment-84899209 @yhuai Any more comment on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5987] [MLlib] Save/load for GaussianMix...

2015-03-23 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4986#issuecomment-84913707 What would be the reason to add a Save Load Version 1.0. What are the expected changes to be done in further versions? --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-6466][SQL] Remove unnecessary attribute...

2015-03-23 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/5134 [SPARK-6466][SQL] Remove unnecessary attributes when resolving GroupingSets When resolving `GroupingSets`, we currently list all outputs of `GroupingSets`'s child plan. However, the columns that are

[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925134 --- Diff: pom.xml --- @@ -1472,6 +1474,46 @@ groupIdorg.scalatest/groupId artifactIdscalatest-maven-plugin/artifactId

[GitHub] spark pull request: [SPARK-6202] [SQL] enable variable substitutio...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4930#issuecomment-84843466 [Test build #28981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28981/consoleFull) for PR 4930 at commit

[GitHub] spark pull request: [SPARK-6449][YARN] Report failure status if dr...

2015-03-23 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/5130#issuecomment-84860777 If they wrap Errors as well, then the fix would be to replace Exception with Throwable in the match block of the InvocationTargetException cause. This has been

[GitHub] spark pull request: [SPARK-6202] [SQL] enable variable substitutio...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4930#issuecomment-84876719 [Test build #28982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28982/consoleFull) for PR 4930 at commit

[GitHub] spark pull request: [SPARK-6202] [SQL] enable variable substitutio...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4930#issuecomment-84876740 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6433 hive tests to import spark-sql test...

2015-03-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/5119#discussion_r26925089 --- Diff: pom.xml --- @@ -158,6 +158,7 @@ fasterxml.jackson.version2.4.4/fasterxml.jackson.version

[GitHub] spark pull request: [SPARK-5775] BugFix: GenericRow cannot be cast...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4697#issuecomment-84940364 [Test build #28994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28994/consoleFull) for PR 4697 at commit

[GitHub] spark pull request: Update the command to use IPython notebook

2015-03-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5111#issuecomment-85198759 OK, nevermind my question. I think it's clear you know what to do here and it's as you think it should be. I'll leave it open a bit for any other opinions but if it's

[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5144#issuecomment-85221484 [Test build #29030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29030/consoleFull) for PR 5144 at commit

[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5144#issuecomment-85220925 [Test build #29030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29030/consoleFull) for PR 5144 at commit

[GitHub] spark pull request: [SPARK-2691][Mesos] Support for Mesos DockerIn...

2015-03-23 Thread hellertime
Github user hellertime commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-85229862 @tnachen I'm stumped at the moment. I've gone so far as to exclude the explicit docker/spark-mesos/Dockerfile path, but it is still not excluded. I had put this down

[GitHub] spark pull request: [SPARK-6308] [MLlib] [Sql] Override TypeName i...

2015-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5118 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-6414: Spark driver failed with NPE on jo...

2015-03-23 Thread hunglin
Github user hunglin commented on the pull request: https://github.com/apache/spark/pull/5124#issuecomment-85186794 @JoshRosen thanks for the suggestions. Let's me work on those tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-6473] [core] Do not try to figure out S...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5143#issuecomment-85215859 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6477][Build]: Run MIMA tests before the...

2015-03-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5145#issuecomment-85223025 Agree, I like this one. Fail-fast checks should go first. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-6406] Launcher backward compatibility i...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5085#discussion_r26985197 --- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java --- @@ -47,10 +47,14 @@ * character. On Windows, the output is a command line

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade Tachyon client vers...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4867#issuecomment-85183514 The master SBT build is currently broken for a few Hadoop profiles due to dependency issues. Do you think that this patch may have been responsible? I noticed that

[GitHub] spark pull request: Update the command to use IPython notebook

2015-03-23 Thread yuecong
Github user yuecong commented on the pull request: https://github.com/apache/spark/pull/5111#issuecomment-85188711 Let me clarify my opinions more clearly. 1, change '$ PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS=notebook --pylab inline ./bin/pyspark' to $

[GitHub] spark pull request: [SPARK-5559] [Streaming] [Test] Remove oppotun...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4337#issuecomment-85192767 [Test build #29024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29024/consoleFull) for PR 4337 at commit

[GitHub] spark pull request: [SPARK-5559] [Streaming] [Test] Remove oppotun...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4337#issuecomment-85192800 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-85197419 [Test build #29026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29026/consoleFull) for PR 4435 at commit

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-85197443 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread brennonyork
Github user brennonyork commented on a diff in the pull request: https://github.com/apache/spark/pull/5093#discussion_r26983769 --- Diff: dev/tests/pr_new_dependencies.sh --- @@ -0,0 +1,85 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6477][Build]: Run MIMA tests before the...

2015-03-23 Thread brennonyork
GitHub user brennonyork opened a pull request: https://github.com/apache/spark/pull/5145 [SPARK-6477][Build]: Run MIMA tests before the Spark test suite This moves the MIMA checks to before the full Spark test suite such that, if new PR's fail the MIMA check, they will return much

[GitHub] spark pull request: [SPARK-5338][MESOS] Add cluster mode support f...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5144#issuecomment-85221495 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5750][SPARK-3441][SPARK-5836][CORE] Add...

2015-03-23 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/5074#issuecomment-85229756 @srowen I've got some more comments. Going to be fairly nitpicky on this because I think it'd benefit people to be as clear as possible. --- If your project is set up

[GitHub] spark pull request: [SPARK-6475][SQL] recognize array types when i...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5146#issuecomment-85231185 [Test build #29035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29035/consoleFull) for PR 5146 at commit

[GitHub] spark pull request: [SPARK-5750][SPARK-3441][SPARK-5836][CORE] Add...

2015-03-23 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5074#discussion_r26987263 --- Diff: docs/programming-guide.md --- @@ -1086,6 +1086,62 @@ for details. /tr /table +### Shuffle operations + +Certain operations

[GitHub] spark pull request: [SPARK-5987] [MLlib] Save/load for GaussianMix...

2015-03-23 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4986#discussion_r26976590 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/GaussianMixtureModel.scala --- @@ -83,5 +95,82 @@ class GaussianMixtureModel(

[GitHub] spark pull request: [SPARK-6473] [core] Do not try to figure out S...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5143#issuecomment-85215748 [Test build #29027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29027/consoleFull) for PR 5143 at commit

[GitHub] spark pull request: [SPARK-6406] Launcher backward compatibility i...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5085#discussion_r26985093 --- Diff: bin/spark-class --- @@ -40,36 +40,24 @@ else fi fi -# Look for the launcher. In non-release mode, add the compiled classes

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4027#issuecomment-85228821 [Test build #29034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29034/consoleFull) for PR 4027 at commit

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/4129#issuecomment-85229133 @CodingCat sorry you're right, I didn't realize CPUS_PER_TASK was configured to that flag. LGTM --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-4086][GraphX]: Fold-style aggregation f...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5142#issuecomment-85228796 [Test build #29033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29033/consoleFull) for PR 5142 at commit

[GitHub] spark pull request: [SPARK-5750][SPARK-3441][SPARK-5836][CORE] Add...

2015-03-23 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5074#discussion_r26987355 --- Diff: docs/programming-guide.md --- @@ -1086,6 +1086,62 @@ for details. /tr /table +### Shuffle operations + +Certain operations

[GitHub] spark pull request: [SPARK-6308] [MLlib] [Sql] Override TypeName i...

2015-03-23 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5118#issuecomment-85180705 Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85179771 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85179766 [Test build #29028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29028/consoleFull) for PR 5093 at commit

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85179769 [Test build #29028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29028/consoleFull) for PR 5093 at commit

[GitHub] spark pull request: [SPARK-6473] [core] Do not try to figure out S...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5143#issuecomment-85179800 [Test build #29027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29027/consoleFull) for PR 5143 at commit

[GitHub] spark pull request: [SPARK-5847] Allow for namespacing metrics by ...

2015-03-23 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/4632#issuecomment-85180665 Thanks @pwendell. I had stumbled across that [SPARK-3377](https://issues.apache.org/jira/browse/SPARK-3377) work as well. I think there are solid arguments

[GitHub] spark pull request: [SPARK-4086][GraphX]: Fold-style aggregation f...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5142#issuecomment-85182245 [Test build #29029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29029/consoleFull) for PR 5142 at commit

[GitHub] spark pull request: [SPARK-6473] [core] Do not try to figure out S...

2015-03-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5143#issuecomment-85186045 Seems fine to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-][MESOS] Add cluster mode support for M...

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/5144#issuecomment-85219844 @andrewor14 Let me know what you think! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6124] Support jdbc connection propertie...

2015-03-23 Thread vlyubin
Github user vlyubin commented on a diff in the pull request: https://github.com/apache/spark/pull/4859#discussion_r26986539 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JDBCRelation.scala --- @@ -115,18 +116,21 @@ private[sql] class DefaultSource extends

[GitHub] spark pull request: [SPARK-5750][SPARK-3441][SPARK-5836][CORE] Add...

2015-03-23 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5074#discussion_r26986920 --- Diff: docs/programming-guide.md --- @@ -1086,6 +1086,62 @@ for details. /tr /table +### Shuffle operations + +Certain operations

[GitHub] spark pull request: [SPARK-4086][GraphX]: Fold-style aggregation f...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5142#issuecomment-85220210 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4086][GraphX]: Fold-style aggregation f...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5142#issuecomment-85220152 [Test build #29029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29029/consoleFull) for PR 5142 at commit

[GitHub] spark pull request: [SPARK-6287][MESOS] Add dynamic allocation to ...

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/4984#issuecomment-85228355 @pwendell @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6124] Support jdbc connection propertie...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4859#issuecomment-85228515 [Test build #29032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29032/consoleFull) for PR 4859 at commit

[GitHub] spark pull request: [SPARK-2691][Mesos] Support for Mesos DockerIn...

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/3074#issuecomment-85228145 @hellertime are you able to figure out the RAT problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-5750][SPARK-3441][SPARK-5836][CORE] Add...

2015-03-23 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5074#discussion_r26987121 --- Diff: docs/programming-guide.md --- @@ -1086,6 +1086,62 @@ for details. /tr /table +### Shuffle operations + +Certain operations

[GitHub] spark pull request: [SPARK-6475][SQL] recognize array types when i...

2015-03-23 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/5146 [SPARK-6475][SQL] recognize array types when infer data types from JavaBeans Right now if there is a array field in a JavaBean, the user wold see an exception in `createDataFrame`. @liancheng You

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-85103963 [Test build #29003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29003/consoleFull) for PR 4435 at commit

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26957574 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala --- @@ -0,0 +1,412 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26958295 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala --- @@ -0,0 +1,412 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26958330 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEnv.scala --- @@ -0,0 +1,412 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: Update the command to use IPython notebook

2015-03-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5111#issuecomment-85116122 But does this then work with ipython 2? I wouldn't want to necessarily 'break' support, even if it's just in an example. Or are two examples called for? Ideally, one

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread brennonyork
Github user brennonyork commented on a diff in the pull request: https://github.com/apache/spark/pull/5093#discussion_r26959105 --- Diff: dev/tests/pr_new_dependencies.sh --- @@ -0,0 +1,77 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6369] [SQL] [WIP] Uses commit coordinat...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5139#issuecomment-85116031 [Test build #29004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29004/consoleFull) for PR 5139 at commit

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85118421 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-4848] Stand-alone cluster: Allow differ...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5140#issuecomment-85118446 [Test build #29005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29005/consoleFull) for PR 5140 at commit

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85118397 [Test build #29006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29006/consoleFull) for PR 5093 at commit

[GitHub] spark pull request: [SPARK-4123][Project Infra][WIP]: Show new dep...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5093#issuecomment-85118409 [Test build #29006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29006/consoleFull) for PR 5093 at commit

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26959685 --- Diff: core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala --- @@ -0,0 +1,318 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-6256] [MLlib] MLlib Python API parity c...

2015-03-23 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4997#discussion_r26953234 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -111,9 +111,11 @@ private[python] class PythonMLLibAPI extends

[GitHub] spark pull request: [SPARK-6322][SQL] CTAS should consider the cas...

2015-03-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/5014#discussion_r26953213 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -557,7 +557,6 @@

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-85098523 [Test build #29000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29000/consoleFull) for PR 4435 at commit

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26958734 --- Diff: core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala --- @@ -0,0 +1,318 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-3533][Core][PySpark] Add saveAsTextFile...

2015-03-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4895#issuecomment-85115634 My entirely personal opinion is I'm neutral on whether this is worth more API methods. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26959384 --- Diff: core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala --- @@ -0,0 +1,318 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [Spark-4848] Stand-alone cluster: Allow differ...

2015-03-23 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/5140#issuecomment-85117276 I'm not sure how mesos and yarn clusters are started/stopped (nor do I have such clusters on which to test), so I'm not sure how this will affect them. I think the

[GitHub] spark pull request: [SPARK-6468][Block Manager] Fix the race condi...

2015-03-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/5136#discussion_r26959388 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala --- @@ -91,7 +90,12 @@ private[spark] class DiskBlockManager(blockManager:

[GitHub] spark pull request: [SPARK-5987] [MLlib] Save/load for GaussianMix...

2015-03-23 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4986#issuecomment-85086722 We want to allow the model data to be extended (with defaults to allow backwards compatibility). There might be unforeseeable reasons to change the format, too. ---

[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-85096699 @sryza When creating a Mesos Task, one usually define the resources required for the execution of the task and the resources required to run the Mesos executor. Again

[GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...

2015-03-23 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/5063#issuecomment-85096806 Fractional is definitely supported, since it's just cpu shares in the end. We should make it a double --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-6308] [MLlib] [Sql] Override TypeName i...

2015-03-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5118#issuecomment-85101090 [Test build #29001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29001/consoleFull) for PR 5118 at commit

[GitHub] spark pull request: [SPARK-4848] Stand-alone cluster: Allow differ...

2015-03-23 Thread nkronenfeld
Github user nkronenfeld closed the pull request at: https://github.com/apache/spark/pull/3699 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4848] Stand-alone cluster: Allow differ...

2015-03-23 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/3699#issuecomment-85112967 I'm redoing this in the latest code, remaking the PR from scratch, to alleviate merge issues. I'll post the new PR here when it's made. --- If your project is set

[GitHub] spark pull request: [SPARK-5124][Core] A standard RPC interface an...

2015-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4588#discussion_r26958715 --- Diff: core/src/main/scala/org/apache/spark/rpc/akka/AkkaRpcEnv.scala --- @@ -0,0 +1,318 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-6369] [SQL] [WIP] Uses commit coordinat...

2015-03-23 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/5139 [SPARK-6369] [SQL] [WIP] Uses commit coordinator to help committing Hive and Parquet tables This PR leverages the output commit coordinator introduced in #4066 to help committing Hive and

[GitHub] spark pull request: [Spark-4848] Stand-alone cluster: Allow differ...

2015-03-23 Thread nkronenfeld
GitHub user nkronenfeld opened a pull request: https://github.com/apache/spark/pull/5140 [Spark-4848] Stand-alone cluster: Allow differences between workers with multiple instances This refixes #3699 with the latest code. This fixes SPARK-4848 I've changed the

<    1   2   3   4   5   6   >