[GitHub] spark pull request: [WIP]Combine splits by size

2015-10-13 Thread zhichao-li
GitHub user zhichao-li opened a pull request: https://github.com/apache/spark/pull/9097 [WIP]Combine splits by size The idea is simple and it try to solve this problem by combining splits by size which has been generated by the underlying inputformat, so it would support all of

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

2015-10-13 Thread a-roberts
Github user a-roberts commented on the pull request: https://github.com/apache/spark/pull/8995#issuecomment-147657380 Josh, apologies for the late response here and making sure I'm understanding your proposal For your first comment the naive way would to be add

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-147658663 [Test build #43640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43640/consoleFull) for PR 9096 at commit

[GitHub] spark pull request: [WIP]Combine splits by size

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147658806 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10533] [SQL] handle scientific notation...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9085#issuecomment-147674049 [Test build #43634 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43634/console) for PR 9085 at commit

[GitHub] spark pull request: [SPARK-10533] [SQL] handle scientific notation...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9085#issuecomment-147674267 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11068][SQL] add callback to query execu...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9078#issuecomment-147674000 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11077] [SQL] Join elimination in Cataly...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9089#issuecomment-147679090 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11077] [SQL] Join elimination in Cataly...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9089#issuecomment-147678970 [Test build #43638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43638/console) for PR 9089 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147661007 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147661009 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147665208 CC @sryza @vanzin seems reasonable to make sure it's actually allocating what YARN said it could? Is this really the extent of the assumption though? it seems

[GitHub] spark pull request: [SPARK-11036][SQL] AttributeReference should n...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9094#issuecomment-147670866 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP]Combine splits by size

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147658822 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11009] [SQL] fix wrong result of Window...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9050#issuecomment-147667552 [Test build #1887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1887/console) for PR 9050 at commit

[GitHub] spark pull request: [SPARK-11066] Update DAGScheduler's "misbehave...

2015-10-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9076#issuecomment-147672457 (Lines are too long and have some trailing whitespace it seems) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11068][SQL] add callback to query execu...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9078#issuecomment-147673806 [Test build #43637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43637/console) for PR 9078 at commit

[GitHub] spark pull request: [SPARK-11077] [SQL] Join elimination in Cataly...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9089#issuecomment-147679091 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP]Combine splits by size

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147659252 [Test build #43641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43641/consoleFull) for PR 9097 at commit

[GitHub] spark pull request: [SPARK-11079] Post-hoc review Netty-based RPC ...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9091#issuecomment-147662403 [Test build #43629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43629/console) for PR 9091 at commit

[GitHub] spark pull request: [SPARK-11079] Post-hoc review Netty-based RPC ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9091#issuecomment-147662535 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9096#discussion_r41845779 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskLocation.scala --- @@ -62,12 +62,5 @@ private[spark] object TaskLocation { * These

[GitHub] spark pull request: [SPARK-11036][SQL] AttributeReference should n...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9094#issuecomment-147670575 [Test build #43636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43636/console) for PR 9094 at commit

[GitHub] spark pull request: [SPARK-10983] Unified memory manager

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9084#issuecomment-147670790 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10983] Unified memory manager

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9084#issuecomment-147670793 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11036][SQL] AttributeReference should n...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9094#issuecomment-147670863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10983] Unified memory manager

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9084#issuecomment-147670601 [Test build #43631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43631/console) for PR 9084 at commit

[GitHub] spark pull request: [SPARK-10533] [SQL] handle scientific notation...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9085#issuecomment-147674266 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11068][SQL] add callback to query execu...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9078#issuecomment-147673998 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11079] Post-hoc review Netty-based RPC ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9091#issuecomment-147662534 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11000][YARN]Bug fix: Derby have booted ...

2015-10-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9026#issuecomment-147664500 OK I confess I don't know this aspect well but I think you explained how this is different from just accidentally making two HiveContexts here. To my knowledge it sounds

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147668124 Gotcha. This is probably my ignorance/misunderstanding then. As long as this is the only place the fact that the requested amount wasn't the same as the granted amount.

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147660586 [Test build #43628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43628/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-11000][YARN]Bug fix: Derby have booted ...

2015-10-13 Thread SaintBacchus
Github user SaintBacchus commented on the pull request: https://github.com/apache/spark/pull/9026#issuecomment-147660393 Actually there are two metastores. In hive-1.2.1 when we use `metastoe.Hive`, it will create the metastore in static code block. As spark have two class

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147667700 @srowen , not sure what exactly you mean? From what I know in `CoarseGrainedSchedulerBackend`, it will manage the executors with cores available, this number

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/8884#discussion_r41863564 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -93,525 +94,559 @@ class LinearRegressionSuite extends

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147712151 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9045 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-10-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-147722497 @vanzin does this also fix https://github.com/apache/spark/pull/7786? From the description is sounds like it. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147726522 [Test build #43643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43643/console) for PR 8884 at commit

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147714180 +1 thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41866396 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -414,7 +418,7 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41866472 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -395,6 +395,10 @@ private[yarn] class YarnAllocator( val

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread Lewuathe
Github user Lewuathe commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147712068 @dbtsai Thank you so much for reviewing many times. Could check it again, please? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147712183 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147714575 [Test build #43643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43643/consoleFull) for PR 8884 at commit

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-147686577 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8813][SQL][WIP]Combine splits by size

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147690302 [Test build #43641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43641/console) for PR 9097 at commit

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41857332 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -395,6 +395,10 @@ private[yarn] class YarnAllocator( val

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147695726 @tgravescs @vanzin i have updated it. thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147695898 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

2015-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9075#issuecomment-147702807 The fix looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11084][ML][PYTHON] Check if index can c...

2015-10-13 Thread zero323
GitHub user zero323 opened a pull request: https://github.com/apache/spark/pull/9098 [SPARK-11084][ML][PYTHON] Check if index can contain non-zero value before binary search At this moment `SparseVector.__getitem__` executes `np.searchsorted` first and checks if result is in an

[GitHub] spark pull request: [SPARK-9596][SQL]treat hadoop classes as share...

2015-10-13 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on a diff in the pull request: https://github.com/apache/spark/pull/7931#discussion_r41853694 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -124,6 +124,7 @@ private[hive] class

[GitHub] spark pull request: [SPARK-8813][SQL][WIP]Combine splits by size

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147690432 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8813][SQL][WIP]Combine splits by size

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9097#issuecomment-147690436 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41856996 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -395,6 +395,10 @@ private[yarn] class YarnAllocator( val

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147701142 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147700926 [Test build #43642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43642/console) for PR 9045 at commit

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147701136 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9096#discussion_r41860056 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskLocation.scala --- @@ -62,12 +62,5 @@ private[spark] object TaskLocation { * These

[GitHub] spark pull request: [SPARK-11084][ML][PYTHON] Check if index can c...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9098#issuecomment-147704842 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-147686278 [Test build #43640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43640/console) for PR 9096 at commit

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/8884#discussion_r41860463 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -93,525 +94,559 @@ class LinearRegressionSuite extends

[GitHub] spark pull request: [Core]Remove useless if branch

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-147686580 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/8884#discussion_r41856388 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -130,9 +131,54 @@ class LinearRegression(override val uid:

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147695928 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [spark-11026][yarn]spark.yarn.user.classpath.f...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9045#issuecomment-147697020 [Test build #43642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43642/consoleFull) for PR 9045 at commit

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41858120 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -395,6 +395,10 @@ private[yarn] class YarnAllocator( val

[GitHub] spark pull request: [SPARK-9066][SQL] Improve cartesian performanc...

2015-10-13 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on a diff in the pull request: https://github.com/apache/spark/pull/7417#discussion_r41855351 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/CartesianProduct.scala --- @@ -28,9 +28,17 @@ import

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/8884#discussion_r41856633 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -130,9 +131,54 @@ class LinearRegression(override val uid:

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-10-13 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/5423#issuecomment-147698799 I've been thinking about this —how about I do the following # have move the history in under yarn/src/history, as discussed above # also as discussed

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147734624 Sometimes its not up to the user what scheduler they user. Like in our case cluster admins choose what its running and users just use it. They have to use whatever

[GitHub] spark pull request: SPARK-1537 publisher-side code and tests

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8744#issuecomment-147746836 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5423#issuecomment-147746919 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5423#issuecomment-147746868 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1537 publisher-side code and tests

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8744#issuecomment-147746916 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1537 publisher-side code and tests

2015-10-13 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/8744#issuecomment-147749231 I've pushed out a rebase of this against master. 1. The asynchronous publishing of events with retries and, eventually dropped events, appears to be

[GitHub] spark pull request: SPARK-1537 [WiP] Application Timeline Server i...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5423#issuecomment-147749606 [Test build #43645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43645/consoleFull) for PR 5423 at commit

[GitHub] spark pull request: SPARK-1537 publisher-side code and tests

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8744#issuecomment-147749964 [Test build #43644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43644/consoleFull) for PR 8744 at commit

[GitHub] spark pull request: [SPARK-11030] [SQL] share the SQLTab across se...

2015-10-13 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9048#issuecomment-147751626 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147726706 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-147726704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147727046 But from yarn's side actually only allocated 1 vcores, whereas in the driver side, it notified with more than 1 cores when executor get registered, this is not

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147721566 So actually against this change. It breaks backwards compatibility and I think the current behavior is what we want. @jerryshao why do you think this is a

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9095#discussion_r41872358 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -395,6 +395,10 @@ private[yarn] class YarnAllocator( val

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147734023 Actually YARN doesn't allocate any. The only reason it reports 1 is because cpu scheduling is disabled and its trying to return something reasonable.YARN does not

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147737702 Yeah, I get it, thanks a lot for your explanation, still from user' point, it may easily get confused, maybe we should document this difference. --- If your project

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147727971 If user want to set executor cores more than 1, user should choose dominant scheduler calculator, that will keep consistent both in spark and yarn side. --- If your

[GitHub] spark pull request: [SPARK-10973][ML][PYTHON] __gettitem__ method ...

2015-10-13 Thread zero323
Github user zero323 commented on the pull request: https://github.com/apache/spark/pull/9009#issuecomment-147753463 @jkbradley It's my pleasure. I've created a [JIRA](https://issues.apache.org/jira/browse/SPARK-11084) and opened a [PR](https://github.com/apache/spark/pull/9098).

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-10-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/8887#discussion_r41882345 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -459,46 +492,30 @@ private[spark] class TaskSchedulerImpl( }

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-10-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/8887#discussion_r41884442 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -781,6 +781,23 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-7402][ML] JSON SerDe for standard param...

2015-10-13 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9090#issuecomment-147756356 Making a pass now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11082][YARN] Fix wrong core number when...

2015-10-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9095#issuecomment-147767229 There's related discussion about this in https://issues.apache.org/jira/browse/SPARK-6050 and the respective PR (#4818). --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-10-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-147768213 @tgravescs this change does not replace #7786; that change makes the code stop allocating tasks on containers that will be preempted soon, while this change does that

[GitHub] spark pull request: [SPARK-10983] Unified memory manager

2015-10-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9084#issuecomment-147768208 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10983] Unified memory manager

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9084#issuecomment-147769449 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11009] [SQL] fix wrong result of Window...

2015-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9050 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-8398 hadoop input/output format advanced...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6848#issuecomment-147778564 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-8398 hadoop input/output format advanced...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6848#issuecomment-147778517 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

<    1   2   3   4   5   6   7   8   >