[GitHub] spark pull request: [WIP][HOTFIX][SPARK-4123]: Fix bug in PR depen...

2015-04-13 Thread brennonyork
Github user brennonyork commented on the pull request: https://github.com/apache/spark/pull/5443#issuecomment-92518053 @shaneknapp can you help me understand how Jenkins is doing the checkouts? I'm seeing the PR builder outputting: ``` Building remotely on

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5463#issuecomment-92490886 [Test build #30175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30175/consoleFull) for PR 5463 at commit

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5463#issuecomment-92490912 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-92490808 [Test build #30190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30190/consoleFull) for PR 5400 at commit

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-92490825 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-13 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/5430#discussion_r28278878 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorExitCode.scala --- @@ -46,9 +46,9 @@ object ExecutorExitCode { case OOM =

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5463#discussion_r28278804 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -755,104 +769,115 @@ private[spark] class BlockManager( case _ =

[GitHub] spark pull request: [Spark-4848] Allow different Worker configurat...

2015-04-13 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/5140#issuecomment-92497626 @andrewor14 - is there anything else that needs to get done here? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5463#discussion_r28278835 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -755,104 +769,115 @@ private[spark] class BlockManager( case _ =

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

2015-04-13 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/5430#discussion_r28278769 --- Diff: core/src/main/scala/org/apache/spark/storage/OffHeapStore.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92507955 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92507937 [Test build #30196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30196/consoleFull) for PR 5497 at commit

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282519 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala --- @@ -17,15 +17,16 @@ package

[GitHub] spark pull request: [SPARK-6229] Add SASL encryption to network li...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5377#discussion_r28282594 --- Diff: network/common/src/main/java/org/apache/spark/network/sasl/SaslClientBootstrap.java --- @@ -62,13 +76,26 @@ public void doBootstrap(TransportClient

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28282545 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282554 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282538 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -55,58 +49,42 @@ class Param[T] ( */ def -(value: T):

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282550 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28282630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282559 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/SchemaUtils.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282562 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/SchemaUtils.scala --- @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SQL][SPARK-6742]: Don't push down predicates ...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5390 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5463#discussion_r28283431 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -755,104 +769,115 @@ private[spark] class BlockManager( case _ =

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5463#discussion_r28283453 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockStore.scala --- @@ -22,13 +22,18 @@ import java.nio.ByteBuffer import

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28283381 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request: [SPARK-6844][SQL] Clean up accumulators used i...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5475#issuecomment-92513734 I see, thanks for clarifying. It seems bad that we are creating accumulators per copy anyway then. Can we just pass in the accumulator with the buffers and statistics

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285427 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92493058 [Test build #30184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30184/consoleFull) for PR 5236 at commit

[GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5236#issuecomment-92493085 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-6735:[YARN] Adding properties to disable...

2015-04-13 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/5449#discussion_r28278175 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -94,6 +98,14 @@ private[yarn] class YarnAllocator( // Additional

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92502644 [Test build #30196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30196/consoleFull) for PR 5497 at commit

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4865#issuecomment-92506360 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5360] [SPARK-6606] Eliminate duplicate ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4145#issuecomment-92505669 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2669] [yarn] Distribute client configur...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4142#issuecomment-92508870 [Test build #30197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30197/consoleFull) for PR 4142 at commit

[GitHub] spark pull request: Export driver quirks

2015-04-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5498#issuecomment-92511690 This needs a JIRA -- see https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28283598 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5463#issuecomment-92512403 I talked to @squito offline and it doesn't seem like there's a less intrusive solution to this issue, so this looks OK to me. But I'm not that familiar with these code

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5431#issuecomment-92513221 @mengxr I added minor comment but don't have major ones. That's it for now! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28283974 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -139,6 +141,8 @@ private[sql] class SQLConf extends Serializable {

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28284671 --- Diff: core/src/main/scala/org/apache/spark/deploy/ExecutorDelegationTokenUpdater.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-92514829 [Test build #30191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30191/consoleFull) for PR 4435 at commit

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-92514838 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285231 --- Diff: core/src/main/scala/org/apache/spark/deploy/ExecutorDelegationTokenUpdater.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5350#issuecomment-92499420 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5350#issuecomment-92499409 [Test build #30187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30187/consoleFull) for PR 5350 at commit

[GitHub] spark pull request: [SPARK-6130] [SQL] support if not exists for i...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4865 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28282909 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-6887][SQL] ColumnBuilder misses FloatTy...

2015-04-13 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/5499 [SPARK-6887][SQL] ColumnBuilder misses FloatType https://issues.apache.org/jira/browse/SPARK-6887 You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28284885 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -201,6 +206,49 @@ class SparkHadoopUtil extends Logging { val

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285004 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -201,6 +206,49 @@ class SparkHadoopUtil extends Logging { val

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285458 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5330 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6065] [MLlib] Optimize word2vec.findSyn...

2015-04-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5467#issuecomment-92519784 Yep, that's pretty much what I had in mind, except that I'd recommend: * using MLlib's local Matrix type (and its BLAS call in mllib.linalg.BLAS) * computing and

[GitHub] spark pull request: [SPARK-5972] [MLlib] Cache residuals and gradi...

2015-04-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5330#issuecomment-92519859 Merged into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/5497 [SPARK-6368][SQL] Build a specialized serializer for Exchange operator. JIRA: https://issues.apache.org/jira/browse/SPARK-6368 You can merge this pull request into a Git repository by running: $

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282819 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -179,52 +179,96 @@ trait Params extends Identifiable with Serializable {

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92512638 Yes, let's call the shaded B artifact B'. Are you saying that the published artifact of A shades B and C? yes, then it contains B' and C' and they are inseparable. It

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28284358 --- Diff: core/src/main/scala/org/apache/spark/deploy/ExecutorDelegationTokenUpdater.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28286134 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-3454] [WIP] separate json endpoints for...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4435#issuecomment-92490236 [Test build #30191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30191/consoleFull) for PR 4435 at commit

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-92490225 [Test build #30190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30190/consoleFull) for PR 5400 at commit

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5463#issuecomment-92491757 [Test build #30180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30180/consoleFull) for PR 5463 at commit

[GitHub] spark pull request: [Spark-6839] BlockManger.dataDeserialize must ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5463#issuecomment-92491779 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28282279 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28283755 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28283689 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -325,7 +379,7 @@ class ParamMap private[ml] (private val map:

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285820 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6886] [PySpark] fix big closure with sh...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5496#issuecomment-92517832 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6886] [PySpark] fix big closure with sh...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5496#issuecomment-92517801 [Test build #30192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30192/consoleFull) for PR 5496 at commit

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92518143 Sorry for the bad example, here is (hopefully) a better explanation: Previously: `tachyon-client` depends on `tachyon` `tachyon` depends on

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-13 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-92488435 @srowen What I mean by promoting dependencies is like: Project A depends on Project B Project B depends on Project C If Project S depends on

[GitHub] spark pull request: [SPARK-6638] [SQL] Improve performance of Stri...

2015-04-13 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/5350#discussion_r28279154 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateProjection.scala --- @@ -111,36 +111,54 @@ object

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92500617 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92500606 [Test build #30194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30194/consoleFull) for PR 5497 at commit

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5497#issuecomment-92500556 [Test build #30194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30194/consoleFull) for PR 5497 at commit

[GitHub] spark pull request: [SPARK-6734] [SQL] Add UDTF.close support in G...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5383#issuecomment-92500634 [Test build #30195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30195/consoleFull) for PR 5383 at commit

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92509801 [Test build #30198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30198/consoleFull) for PR 5459 at commit

[GitHub] spark pull request: [SPARK-5957][ML] better handling of parameters

2015-04-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5431#discussion_r28282508 --- Diff: mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala --- @@ -78,23 +81,42 @@ class ParamsSuite extends FunSuite { }

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28283271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Exchange.scala --- @@ -45,6 +46,27 @@ case class Exchange(newPartitioning:

[GitHub] spark pull request: [Minor][SparkR] Minor refactor and removes red...

2015-04-13 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5495#issuecomment-92515198 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5493#issuecomment-92515235 Jenkins, add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6881][SparkR] Changes the checkpoint di...

2015-04-13 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5493#issuecomment-92515144 Jenkins, ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28284735 --- Diff: core/src/main/scala/org/apache/spark/deploy/ExecutorDelegationTokenUpdater.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285158 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -164,6 +174,9 @@ private[spark] object

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28285643 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28286065 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28286063 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28286012 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5342][YARN] Allow long running Spark ap...

2015-04-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4688#discussion_r28286027 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala --- @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6568] spark-shell.cmd --jars option doe...

2015-04-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5447#issuecomment-92485233 but why not just use File.toURI That works for me. If doing that, though, should make sure that all the call sites can handle URIs with a scheme, since

[GitHub] spark pull request: [SPARK-5277][SQL] - SparkSqlSerializer doesn't...

2015-04-13 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/5237#issuecomment-92485811 Seems `new SparkSqlSerializer(new SparkConf(false))` is also used in `org.apache.spark.sql.execution.Limit`. Can you fix that as well? --- If your project is set up for

[GitHub] spark pull request: [SPARK-6886] [PySpark] fix big closure with sh...

2015-04-13 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/5496 [SPARK-6886] [PySpark] fix big closure with shuffle Currently, the created broadcast object will have same life cycle as RDD in Python. For multistage jobs, an PythonRDD will be created in JVM and

[GitHub] spark pull request: [SPARK-6886] [PySpark] fix big closure with sh...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5496#issuecomment-92497692 [Test build #30192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30192/consoleFull) for PR 5496 at commit

[GitHub] spark pull request: [SPARK-4897] [PySpark] Python 3 support

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5173#issuecomment-92497746 [Test build #30193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30193/consoleFull) for PR 5173 at commit

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92505667 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5360] [SPARK-6606] Eliminate duplicate ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4145#issuecomment-92505635 [Test build #30189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30189/consoleFull) for PR 4145 at commit

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92505634 [Test build #30188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30188/consoleFull) for PR 5459 at commit

[GitHub] spark pull request: [SPARK-6368][SQL] Build a specialized serializ...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/5497#discussion_r28282234 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer2.scala --- @@ -0,0 +1,378 @@ +/* + * Licensed to the

[GitHub] spark pull request: WIP: re-enable flaky test to catch failure in ...

2015-04-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5459#issuecomment-92509413 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Export driver quirks

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5498#issuecomment-92511404 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

<    1   2   3   4   5   6   7   >